Today, we're announcing Observability for Ana.
When you deploy any platform, you need visibility into how it's being used. That usually means wiring up a product analytics tool to track engagement, and adding an eval framework on top to monitor output quality. For AI systems, you're doing both at once.
Observability in TextQL handles both. Think of it as the monitoring plane for Ana: usage across your organization and the quality of what it's producing, in the same place. Active users, ACU (Agentic Compute Unit) consumption, and connector activity on the usage side. On the quality side: sentiment, response length, failed executions, cost per query, and LLM-judged missing context. All of it surfaces natively in the observability tab — no additional building or pulling data required.
What You Can Track
Observability covers every surface of the product. Whether you're trying to understand adoption, diagnose a failing workflow, or track down why a specific user is getting bad answers, there's a tab for it.
Overview
The Overview tab is the main monitoring surface. It shows run volume in TextQL over time, broken down by source: direct threads, scheduled playbooks, Teams (or Slack), and agents.
Observability also surfaces quality signals. After every thread completes, Ana automatically analyzes it for problems: gaps in context, execution errors, signs of user frustration, potential inaccuracies. These appear as warnings attached to individual threads. The Overview tab aggregates those warnings, ranks them by type, and lets you filter for the specific conversations where something went wrong:
Thread Insights
From any of those flagged threads, you can open a full breakdown as an admin. Each conversation renders as a turn-by-turn timeline, with query planning and reasoning time separated from execution time, so you can see where Ana spent its compute and whether the split looks right. If a thread produced a warning, the specific issue is pinned to the moment it occurred, along with a suggested fix. This is how you turn a pattern from the Overview tab into a diagnosis.
Users
Alongside quality, you can track cost. The Users tab shows ACU consumption trending across your organization: who's active, how often, and what they're spending. For each member, you can see their thread count, playbook runs, dashboard views, agent activity, and total ACU consumption for the period, alongside a sparkline showing their activity trend.
Summary cards at the top give you active user count, average ACUs per user, and your top spender. This is where you track adoption and find the power users of your platform.
Agents
Agents in TextQL are scheduled automations: they run on a cadence, post insights, and consume ACUs the same way a user does.
The Agents tab shows you which agents are active, how many runs they've completed, what they're costing, and whether their runs are producing warnings. Agents generating warnings or running above-average ACU consumption are flagged — often, this is a signal that a data connector that they depend on has drifted or broken.
Playbooks
Playbooks are Ana's scheduled, automated analyses: reports that run on a cadence without anyone having to ask. The Playbooks tab shows you which ones are running, how often, and what they're costing, with ACU spend broken down into LLM usage and compute separately.
Dashboards
Dashboards are the shared views your team comes back to repeatedly. Similar to Playbooks, the Dashboards tab tells you which ones are getting used: view counts, refresh counts, and ACU cost per dashboard. High-traffic dashboards are worth keeping sharp.
Connectors
These insights also extend to your data connectors. The Connectors tab shows you all the data sources your organization has added and which ones Ana is actively querying. It tells you how fast they're responding and whether they're producing errors.
If a connector's error rate spikes, you'll see it here before users start filing tickets, and you can drill back into the Overview tab filtered to that connector's users to find the specific threads where it broke down. High error rates typically point to schema mismatches, permission issues, or ontology definitions that have drifted from the underlying data.
Export
These insights don't have to stay inside TextQL. If your team already operates a monitoring stack (Datadog, Grafana, Splunk, or anything OpenTelemetry-compatible), the Export tab connects Ana into it rather than adding a separate dashboard to check.
There are two streams for your export. Audit logs cover every key action Ana takes and can be pushed to an S3 bucket or any OpenTelemetry-compatible collector like Datadog, Grafana, or Splunk. Product metrics can be scraped via a Prometheus endpoint, pushed via OpenTelemetry, or exported to S3. Either way, Ana fits into the infrastructure you already operate.
Observability is available now. Export is in Public Preview. You can find it in the bottom-left corner of the navigation bar.
