Openlit

Openlit is an open-source observability platform built specifically for teams developing and deploying generative AI and large language model applications. It provides comprehensive monitoring, tracing, and evaluation tools to help you understand exactly how your AI systems are performing in production. Think of it as a specialized dashboard that shows you not just if your LLM calls are working, but how long they take, how much they cost, where bottlenecks occur, and how different prompts affect outcomes.

What makes Openlit stand out is its deep integration with the OpenTelemetry standard, meaning it fits right into existing developer workflows. You can instrument your Python or TypeScript code with just a few lines, or use its Kubernetes operator for zero-code instrumentation. It supports a huge range of LLM providers and frameworks out of the box—from OpenAI and Anthropic to local models via Ollama—giving you a unified view regardless of where your models are running. Key features include detailed distributed tracing to map request flows, a prompt hub for versioning and managing prompts, and tools for running systematic evaluations on your models.

This tool is a major boon for AI engineers and MLOps teams who are moving beyond prototypes and need to ship reliable, performant, and cost-effective AI applications. If you're tired of flying blind with your LLM integrations, wondering why responses are slow, or which prompt version actually works best, Openlit gives you the data to make informed decisions. It's built for the messy reality of production, where you need to monitor multiple deployments, debug complex agent chains, and continuously improve your AI's performance based on real evidence.