Ecosystem
Agent Monitoring and Debugging with AgentOps
AgentOps provides session replays, metrics, and monitoring for AI agents.
At a high level, AgentOps gives you the ability to monitor LLM calls, costs, latency, agent failures, multi-agent interactions, tool usage, session-wide statistics, and more. For more info, check out the AgentOps Repo.
| 📊 Replay Analytics and Debugging | Step-by-step agent execution graphs |
| 💸 LLM Cost Management | Track spend with LLM foundation model providers |
| 🧪 Agent Benchmarking | Test your agents against 1,000+ evals |
| 🔐 Compliance and Security | Detect common prompt injection and data exfiltration exploits |
| 🤝 Framework Integrations | Native Integrations with CrewAI, AG2, & LangChain |
Agent Dashboard
Agent Dashboard
Session Analytics
Session Analytics
Session Replays
Session Replays
Installation
AgentOps works seamlessly with applications built using AG2.- Install AgentOps
- Create an API Key: Create a user API key here: Create API Key
- Configure Your Environment: Add your API key to your environment variables
- Initialize AgentOps
Features
- LLM Costs: Track spend with foundation model providers
- Replay Analytics: Watch step-by-step agent execution graphs
- Recursive Thought Detection: Identify when agents fall into infinite loops
- Custom Reporting: Create custom analytics on agent performance
- Analytics Dashboard: Monitor high level statistics about agents in development and production
- Public Model Testing: Test your agents against benchmarks and leaderboards
- Custom Tests: Run your agents against domain specific tests
- Time Travel Debugging: Save snapshots of session states to rewind and replay agent runs from chosen checkpoints.
- Compliance and Security: Create audit logs and detect potential threats such as profanity and PII leaks
- Prompt Injection Detection: Identify potential code injection and secret leaks