langfuse🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

langfuse

Langfuse is an open source AI engineering platform that helps teams develop, monitor, evaluate, and debug LLM applications. It provides comprehensive observability, prompt management, and evaluation capabilities while supporting self-hosted and cloud deployments with seamless integrations into popular AI frameworks.

Key Features

• LLM Observability: Instrument applications to track LLM calls, retrieval, embeddings, and agent actions with detailed trace inspection and session debugging
• Prompt Management: Centrally manage, version control, and collaboratively iterate on prompts with built-in caching to avoid latency penalties
• Evaluations: Support for LLM-as-a-judge, code evaluators, user feedback, manual labeling, and custom evaluation pipelines
• Datasets: Create test sets and benchmarks for continuous improvement, pre-deployment testing, and structured experiments
• LLM Playground: Test and iterate on prompts and model configurations with direct integration from trace analysis

Use Cases

• Production Monitoring: Track and debug LLM application behavior in production with comprehensive observability
• Model Optimization: Systematically evaluate and improve prompts and model configurations through structured testing
• Team Collaboration: Manage prompts and evaluations collaboratively across engineering teams with version control

Who Is It For

Langfuse is designed for AI engineering teams building production LLM applications who need robust observability, testing, and deployment capabilities. It serves organizations ranging from individual developers to enterprises seeking either managed cloud deployment or self-hosted control over their LLMOps infrastructure.