AI Evaluation & Benchmarking

Part of the worlds-biggest-software-project initiative.

An open, AI-native platform for LLM output quality testing, regression detection, and human feedback loops.

AI Evaluation & Benchmarking is a candidate project to build an open-source evaluation platform for teams shipping LLM and agent applications. It targets AI engineers, QA teams, and product leaders who need trustworthy quality signals across prompts, models, and multi-step agents — and combines the metric depth of frameworks like DeepEval with the production observability of Langfuse and Arize Phoenix.

Why AI Evaluation & Benchmarking?

Existing evaluation tooling is fragmented: DeepEval leads on metric breadth but lacks production monitoring; Langfuse and Arize Phoenix lead on tracing but have shallower evaluation depth; Braintrust is polished but proprietary and costly ($249/mo Pro, no self-hosting).
Promptfoo — the leading open-source red-teaming tool — was acquired by OpenAI in March 2026, leaving the open ecosystem exposed if its licence trajectory shifts.
Galileo's low-latency Luna-2 evaluators are proprietary; after Cisco's April 2026 acquisition, roadmap and pricing are uncertain.
Agent trajectory evaluation (planning, tool selection, error recovery) is poorly served across every incumbent surveyed.
Regulated industries need self-hosted, privacy-first evaluation with audit-ready exports for NIST AI RMF and the EU AI Act (August 2026 deadline) — only Langfuse and Arize Phoenix offer full-featured self-hosting today, and neither produces compliance-aligned reports.

Key Features

Core Metric Library

Faithfulness, Answer Relevance, Hallucination, Contextual Precision, Contextual Recall, and Toxicity out of the box
Configurable LLM-as-a-judge scoring (G-Eval-style custom criteria)
Transparent, explainable scores with reasoning strings for every judgment
Multi-provider judge support: OpenAI, Anthropic, Google, and local/Ollama models

Developer & CI Workflow

Pytest-native test runner with pass/fail thresholds
JSON, HTML, and XML output formats for CI/CD gating
Dataset management: create, version, and query test datasets from files or production traces
Trace capture for RAG and single-turn LLM calls with span-level timing and metadata

Agent & Production Evaluation

Agent trajectory scoring, tool-call accuracy, and planning-quality metrics
Real-time production monitoring dashboard: score trends, latency, cost, and quality
OTel-compatible trace ingestion (OTLP over gRPC/HTTP) for framework-agnostic instrumentation
Human annotation queues for last-mile review and judge calibration

Safety, Security & Compliance

Red teaming and adversarial test generation aligned to the OWASP LLM Top 10
Prompt management with environment-gated deployment (score thresholds required to promote)
Compliance-ready export aligned to NIST AI RMF and EU AI Act documentation requirements

AI-Native Advantage

AI is used not just as the subject of evaluation but as a core part of the platform. Synthetic adversarial test cases are generated to probe edge cases and failure modes, reducing the manual burden of dataset curation. Natural-language evaluator authoring lets product teams define custom judge criteria in plain English. Automated regression root-cause analysis compares trace deltas between prompt versions to surface causal changes, and adaptive sampling selects the most informative production traces to evaluate rather than relying on random sampling.

Tech Stack & Deployment

The project is designed self-hosted-first (Docker and Kubernetes) with an optional managed cloud, mirroring the deployment model that makes Langfuse and Arize Phoenix viable for regulated industries. Instrumentation is OpenTelemetry-native, building on the OpenInference span conventions so traces from any OTLP-speaking framework — LangChain, LlamaIndex, LiteLLM, OpenAI Agents SDK, Claude Agent SDK, LangGraph, CrewAI, DSPy, Vercel AI SDK — ingest without custom adapters. Python and JavaScript/TypeScript SDKs cover the common application surfaces, with a Pytest plugin for the developer workflow.

Market Context

LLM evaluation sits inside the broader MLOps platform market, valued at approximately USD 2.0 billion in 2024 and projected to exceed USD 10 billion by 2029, and is one of its fastest-growing sub-segments. Pricing tiers are well-defined: open-source self-hosting is free; developer SaaS (Langfuse Pro) starts at $29–$49/mo; professional platforms (Braintrust Pro) run $249/mo; enterprise contracts (Maxim, Arize AI) reach the six-figure range. Primary buyers are AI engineers and ML teams, QA engineers building prompt regression suites, product managers tying LLM quality to business metrics, and compliance officers in regulated industries.

Project Status

This project is in the research and specification phase.
Contributions, feedback, and domain expertise are welcome.

Contributing

We welcome contributions from developers, domain experts, and potential users. See CONTRIBUTING.md for guidelines.

Important: All contributions must be your own original work or clearly attributed open-source material with a compatible licence. Copyright infringement and licence violations will not be tolerated and will result in immediate removal of the offending contribution. If you are unsure whether a piece of code, text, or other material is safe to contribute, open an issue and ask before submitting.

Licence

Licence to be determined. See discussion for context.

claud