regression-detection

tracking performance of rustc-generated binaries over time

Detects when a deploy silently ships a noindex, drops a schema block, or rewrites your canonical. Baselines a page's SEO contract, diffs every change with Critical/Warning/Info severity, and points fixes to claude-seo sub-skills. A Claude Code skill. Open source.

Agentic eval framework for Laravel: golden datasets, LLM-as-judge, adversarial harness, regression detection

Fast, lightweight Rust benchmarking library. Dev mode: profile like Criterion. Prod mode: monitor with <10ns overhead. Features: async benchmarks, regression alerts, health metrics, p95/p99 latency tracking. Thread-safe, minimal deps. Benchmark any code block or function with one macro.

Statistical regression detection and CI quality gates for AI agents

Detects semantic regressions across PRs that git's merge conflict detection misses

JAX benchmarking, profiling and evaluation metrics for Flax NNX: a registry of pure-function metrics (regression, classification, calibration, uncertainty, forecasting, generative, image, text, audio, graph, fairness), XLA FLOP counting, roofline analysis, GPU and energy monitoring, regression detection, publication exports, W&B and MLflow.

A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern

Linux/C++ CLI for benchmarking, perf-counter capture, and regression detection with configurable thresholds and statistical validation.

AI-powered benchmark regression analyzer with CLI, REST API, and interactive Dashboard. Compares cloud performance runs via OpenSearch + LLM, with automated regression detection.

Nextflow plugin for cross-run benchmarking. Records per-process metrics for every run into a local or shared history store, compares runs against their own history, and flags regressions with likely causes, over-provisioned resources, and retries. Includes a zero-dependency dashboard, an MCP server for AI assistants, and opt-in AI narration.

Online evaluation, regression detection, and statistical A/B experimentation for LLM systems in production

This program enables users to microbenchmark their Java code to detect regression in performance.

Compare two AI Procurement Pulse snapshots, flag regressions beyond threshold, render GitHub issue body + vendor-outreach email template citing the specific field diff. Productizes the quarterly Pulse regression signal. Drop in as GH Action in the Pulse engine repo.

SQL service observability demo: SLA monitoring, regression detection, root-cause analysis, and capacity forecasting.

Catch silent agent failures in CI. Pytest plugin with regression detection for LLM agents.

Agent and LLM evaluation harness: golden datasets, multi-scorer execution, regression detection across model versions, cost-quality leaderboards, and CI gates for model promotion.

LLM eval harness (aibench): 21 seeded, reproducible graded tasks with CUSUM drift detection for silent model degradation — plus a Karpathy-style council that auto-elects OpenRouter advisors.

The agent that owns a bad deploy — where your team already works. Pre-merge impact estimates, evidence-backed regression alerts in Ambiguous AI, and human-gated surgical rollbacks, with postmortems and a ledger written automatically.

Performance regression detection for OCP — Streamlit UI for cloud-bulldozer/orion