Week of August 1–7, 2026
Production Monitoring 🔥
Testing doesn't have to stop at release. Every project now has a Monitors tab: point a webhook at your production app, and every live interaction (the query and the LLM response) gets evaluated against the checks in a linked Test Suite, automatically. If production starts drifting from what your tests expect, you find out from a dashboard instead of a support ticket.
Each Monitor is configurable. A sampling rate controls what fraction of traffic actually gets evaluated, so high-volume endpoints don't need to check every single request. Embedding-based deduplication filters out near-identical queries so you're not re-running the same evaluation a thousand times a day. And per-step bypass modes let you skip HTTP calls and run assertions directly against captured data, or exclude setup steps that don't apply to production traffic entirely.
Monitors is available on the Pro plan, with a dedicated setup guide in the in-app help panel walking through webhook configuration end to end.
Public site refresh
The public site now has a proper mobile navigation menu — it was desktop-only before. There's also a LinkedIn link in the footer if you want to follow along there.
New logo
LitmusLab has a new logo, updated across the app, sprinkled with some magic, depending on where you're viewing. 🪄✨
In the Lab
This week we published Why AI Quality Is Hard to Test Even When You're Trying. It's the follow-up to The Speed Trap, digging into the four underlying problems--non-deterministic output, an unbounded input space, no shared definition of "good," and standards that quietly drift over time--that make testing AI quality genuinely hard, even for teams that are actively trying to do it well.