Analytics

Analytics

Monitor performance and gain insights into your app’s usage.

---

Overview

The analytics dashboard provides real-time visibility into how your agentic app performs—tracking users, sessions, messages, and resource consumption across agents, tools, and models.

---

Dashboard Components

Key Metrics

┌─────────────────────────────────────────────────────────────────┐
│                        Usage Overview                            │
├───────────────┬───────────────┬───────────────┬─────────────────┤
│    Users      │   Sessions    │   Messages    │     Tokens      │
│     1,234     │     3,456     │    45,678     │    2.3M         │
│   ↑ 12%       │    ↑ 8%       │    ↑ 15%      │   ↑ 18%         │
└───────────────┴───────────────┴───────────────┴─────────────────┘

Metric	Description
Users	Unique active users in the period
Sessions	Total conversation sessions
Messages	Messages exchanged (user + agent)
Tokens	Total token consumption

Trends

Compare current period to previous:

• Daily/hourly breakdowns
• Week-over-week comparisons
• Growth trends

---

Run Analytics

Track execution across your app’s components.

Agent Runs

Agent	Runs	Avg Response	Tokens	Success
Support Agent	1,234	2.3s	450K	98.5%
Billing Agent	567	1.8s	180K	99.1%
Order Agent	890	3.1s	320K	97.8%

Tool Runs

Tool Type	Tool	Runs	Avg Time	Success
Workflow	get_order	890	450ms	99.2%
Code	validate_input	1,200	120ms	99.8%
MCP	crm_lookup	456	890ms	96.5%
Knowledge	faq_search	2,100	340ms	99.9%

Model Runs

Model	Invocations	Avg Latency	Tokens	Cost
gpt-4o	3,400	1.2s	1.8M	$45.20
gpt-3.5	1,200	0.4s	320K	$0.64

---

Traces

Traces provide detailed visibility into individual request lifecycles.

What’s in a Trace

Trace: req_abc123
├── Start: 2024-01-15 14:30:22.123
├── End: 2024-01-15 14:30:25.456
├── Duration: 3.333s
│
├── Events
│   ├── [14:30:22.123] Request received
│   ├── [14:30:22.145] Agent selected: Support Agent
│   ├── [14:30:22.200] Tool invoked: get_order_status
│   ├── [14:30:22.650] Tool response received
│   ├── [14:30:22.700] LLM generation started
│   ├── [14:30:25.400] LLM generation completed
│   └── [14:30:25.456] Response sent
│
├── Spans
│   ├── Agent Processing: 3.2s
│   ├── Tool Execution: 450ms
│   └── LLM Generation: 2.7s
│
└── Generations
    └── Support Agent response
        ├── Model: gpt-4o
        ├── Input tokens: 1,234
        ├── Output tokens: 256
        └── Latency: 2.7s

Trace Benefits

• Debug request flow issues
• Identify bottlenecks
• Understand agent behavior
• Optimize performance

---

Sessions

Sessions track continuous user interactions.

Session View

Session: sess_xyz789
├── User: user_456
├── Started: 2024-01-15 14:25:00
├── Duration: 12 minutes
├── Traces: 5
│
├── Trace 1: "What's my order status?"
│   └── Agent: Support Agent, Duration: 3.3s
│
├── Trace 2: "When will it arrive?"
│   └── Agent: Support Agent, Duration: 2.1s
│
├── Trace 3: "Can I change the address?"
│   └── Agent: Order Agent, Duration: 4.5s
│
├── Trace 4: "What's the cost?"
│   └── Agent: Billing Agent, Duration: 1.8s
│
└── Trace 5: "Thanks, that's all"
    └── Agent: Support Agent, Duration: 0.8s

Total Cost: $0.12

---

Generations

Track individual LLM outputs within traces.

Generation Details

Field	Value
Model	gpt-4o
Input tokens	1,234
Output tokens	256
Latency	2.7s
Cost	$0.032
Temperature	0.7

Quality Assessment

• Review response quality
• Identify hallucinations
• Track instruction following

---

Filtering

Customize your analytics view:

Time Range

• Last hour
• Last 24 hours
• Last 7 days
• Last 30 days
• Custom range

Environment

• Draft (development)
• Staging
• Production

Dimensions

• By agent
• By tool
• By model
• By user segment

---

Exporting Data

Download analytics for external analysis:

Available Exports

• CSV: Spreadsheet-compatible
• JSON: Programmatic analysis
• PDF: Shareable reports

Export Options

export:
  format: csv
  date_range: last_30_days
  include:
    - sessions
    - traces
    - generations
    - tool_runs
  filters:
    environment: production
    agent: Support Agent

---

Alerts

Configure notifications for important events:

Alert Types

alerts:
  - name: High error rate
    condition: error_rate > 5%
    window: 1 hour
    action: email

  - name: Slow responses
    condition: avg_latency > 5s
    window: 15 minutes
    action: slack

  - name: Cost spike
    condition: daily_cost > $100
    window: 1 day
    action: email

---

Audit Logs

Track all changes made across your account.

What’s Logged

• User actions (create, update, delete)
• Configuration changes
• Deployments
• Access events

Log Entry

Event: Tool Updated
User: [e-mail]
Time: 2024-01-15 14:30:00
Details:
  Tool: get_order_status
  Changes:
    - timeout: 30s → 60s
    - description: Updated

Compliance Uses

• Track who changed what
• Maintain audit trail
• Support security reviews

---

Best Practices

Monitor Key Metrics

Focus on metrics that matter:

• Success rate: Are requests completing successfully?
• Latency: Is performance acceptable?
• Cost: Is spending within budget?
• User satisfaction: Are users getting help?

Set Baselines

Establish normal ranges to detect anomalies:

baselines:
  success_rate: 95-99%
  avg_latency: 1-3s
  daily_cost: $20-50

Review Regularly

• Daily: Quick health check
• Weekly: Trend analysis
• Monthly: Deep dive and optimization

Act on Insights

Use analytics to drive improvements:

• Slow agent? Optimize tools or prompts
• High error rate? Review configurations
• Cost spike? Check token usage patterns

---

Related

• Deployment
• Diagnostics
• Agents Overview
• Tools Overview