LitmusLab

In the Lab

Research, field notes, and lab reports from the teams building with AI. What we're testing, what we're finding, and what it means.

What's New

What's New — September 11, 2026

A project can now run multiple UI Monitors, each scoped to its own part of your site with independent test suites and settings. Plus configurable Privacy Controls for session capture, a client API for controlling capture from your own code, automatic exclusion of your own test traffic, and sharper, more trustworthy Graph tab and check results.

The LitmusLab Team

The LitmusLab Team

September 11, 2026

4 min read
Read article
What's New

What's New — September 4, 2026

Live sessions that match a Flow step now get checked against your existing test suite assertions automatically, using what actually happened in production. Plus faster, more accurate matching, clearer Graph tab labels and stability status, and Reset Test & Monitoring History in Project Settings.

The LitmusLab Team

The LitmusLab Team

September 4, 2026

3 min read
Read article
What's New

What's New — August 28, 2026

UI Monitoring lands: Production Monitoring now covers UI Workflows, with real session capture, automatic Flow coverage matching, and Uncovered Activity surfacing untested states. Plus alternate viewport sizes for UI Workflow steps.

The LitmusLab Team

The LitmusLab Team

August 28, 2026

3 min read
Read article
Research

The Missing Piece in Harness Engineering: The Behavioral Harness

Harness engineering has mapped the maintainability and architecture layers with impressive speed. But the most credible voices in the discipline — Böckeler, the CAAF paper, RepoRescue — all point to the same unsolved piece: the behavioral harness.

The LitmusLab Team

The LitmusLab Team

August 25, 2026

6 min read
Read article
What's New

What's New — August 21, 2026

Collection automation — schedule runs to happen automatically instead of triggering them by hand. Plus pass-rate trend charts, run completion emails, a Stop button for in-progress runs, and faster concurrent execution.

The LitmusLab Team

The LitmusLab Team

August 21, 2026

3 min read
Read article
What's New

What's New — August 14, 2026

UI Workflows land — test through the browser with plain-language steps and vision-based checks. Plus the new Objective tab with Intent Evaluation, Excel results export, and a landing page refresh.

The LitmusLab Team

The LitmusLab Team

August 14, 2026

3 min read
Read article
Research

When AI Writes the Code and the Tests, What Does Passing Actually Mean?

Code throughput is up 59% year-over-year. Main-branch success rates fell to a five-year low of 70.8%. If AI is writing the code and writing the tests, what does a passing test suite actually tell you?

The LitmusLab Team

The LitmusLab Team

August 12, 2026

5 min read
Read article
What's New

What's New — August 7, 2026

Production Monitoring lands — evaluate live traffic against your Test Suite checks with sampling, dedup, and per-step bypass modes. Plus mobile nav on the public site and a new logo.

The LitmusLab Team

The LitmusLab Team

August 7, 2026

3 min read
Read article
Research

Why AI Quality Is Hard to Test Even When You're Trying

More AI features ship faster. The review cycles that used to catch things are getting compressed. Here are the four underlying problems that make testing AI quality hard — and why most teams are stuck even when they're trying.

The LitmusLab Team

The LitmusLab Team

August 5, 2026

5 min read
Read article
What's New

What's New — July 31, 2026

Project-level YAML import/export, Orgs for shared billing and LLM config, invite activation for new members, and a last run timestamp on every test.

The LitmusLab Team

The LitmusLab Team

July 31, 2026

3 min read
Read article
Research

The Speed Trap: Why AI's Biggest Strength Is Creating Its Biggest Quality Problem

88% of agentic AI early adopters are seeing positive ROI. But the same report quietly lists "verify quality" as a core human responsibility — without explaining how anyone is supposed to do that at AI speed.

The LitmusLab Team

The LitmusLab Team

July 28, 2026

5 min read
Read article
What's New

What's New — July 24, 2026

Usage tab in project settings, a free AI Quality Assessment tool (no account needed), and public site improvements.

The LitmusLab Team

The LitmusLab Team

July 24, 2026

2 min read
Read article
Lab Notes

We Built Screened to Prove LitmusLab Isn't Just Theoretical

We built a real AI application — a social movie recommendation engine — and ran it through three levels of testing. This is what we found, what we'd do differently, and what it means for teams shipping AI features.

The LitmusLab Team

The LitmusLab Team

July 23, 2026

5 min read
Read article
What's New

What's New — July 17, 2026

Scenario Import lands — bring in test data from JSON or CSV. Plus GPT-4o as the default evaluator, in-app help, and automatic scenario sync.

The LitmusLab Team

The LitmusLab Team

July 17, 2026

3 min read
Read article
What's New

What's New — July 10, 2026

Scoring accuracy fixes, scenario field cleanup, and evaluation prompt corrections — all affecting the accuracy of results you're already seeing.

The LitmusLab Team

The LitmusLab Team

July 10, 2026

2 min read
Read article
What's New

What's New — July 3, 2026

A big week: Response Inspector, JMESPath for context mapping, live run progress, stop-on-failure, dashboard trend charts, context mapping preview, and data type support.

The LitmusLab Team

The LitmusLab Team

July 3, 2026

4 min read
Read article
What's New

What's New — June 26, 2026

Contexts & Scenarios land — the most significant change to how LitmusLab works. Define your data structure, populate scenarios, and run every test case against all of them automatically.

The LitmusLab Team

The LitmusLab Team

June 26, 2026

3 min read
Read article
What's New

What's New — June 19, 2026

LitmusLab is live. Delete and duplicate tests, variable substitution in connections, and scrollable results.

The LitmusLab Team

The LitmusLab Team

June 19, 2026

2 min read
Read article