Operations | Monitoring | ITSM | DevOps | Cloud

Teams that build agents on open models hosted on Fireworks AI, such as GPT-OSS, Kimi, GLM, Qwen, and MiniMax, need a reliable way to measure whether their LLM applications and agents are giving accurate, safe, and helpful answers efficiently. Datadog Agent Observability now supports Fireworks AI as a model provider, so you can get end-to-end visibility into your agents built on Fireworks AI, from traces and token usage to quality evaluations.

Prompts are an important part of LLM application behavior, but teams often manage them alongside application code or across separate notebooks and configuration files. This can make prompt changes dependent on application releases and make it difficult to understand which version is running in each environment. When a change affects application quality, latency, or cost, teams also need a way to connect that behavior back to the prompt version that produced it.

When a mobile application takes too long to launch or becomes unresponsive, it can be difficult to find the cause. Datadog Real User Monitoring (RUM) helps developers detect these issues and understand which users are affected. But knowing that an application is slow doesn’t tell you which part of its code is responsible. Engineers often need to reproduce the problem locally, piecing together performance metrics and code execution information from separate tools.