The Lab is where we sharpen the tools we use on every client engagement. Threads we are actively pulling on — and the operational practice behind why our AI work actually reaches production.
Most RAG benchmarks are toys. We have been formalising the eval set we actually trust — adversarial questions, citation-grounding, and continuous comparison against a held-out human baseline. The interesting part is what to do when the eval disagrees with the model.
Tool-using agents fail in messy, instructive ways. We are studying the patterns: tool-schema strictness, retry semantics, deterministic recovery, and where human-in-the-loop genuinely helps versus where it just slows the system down.
A growing slice of our client work needs models running locally — desktop apps, regulated environments, latency-sensitive mobile flows. We are tracking what is genuinely viable today, what falls over the moment users push it, and how to architect around the gaps.
We run several AI systems inside the studio that we built for our own teams — across writing, code review, retrieval, and operations. Some of them inform a future product. Most of them just make the work better.
Every AI feature we ship for clients goes through an eval pipeline before it merges. The pipeline itself is a system — and one we keep iterating on, because the cost of catching a regression in production is much higher than catching it in CI.
We read the papers our peers read. The discipline is in deciding what is real, what is reproducible, and what is worth bringing into a client system this quarter — not just what is loud.
If your team is shipping AI systems where reliability, evals, or on-device inference matter — we would like to compare notes.