Free labs, articles & videos
Learn to build AI you can actually measure.
I'm Slav, an engineer focused on AI quality. I share what I've learned from 10+ years of building test infrastructure and quality systems, and from evaluating LLM apps and AI agents: hands-on labs, practical articles, and videos.
$ evals run --suite support-agent --baseline main task_success 0.912 → 0.934 +2.2pp (95% CI ±1.1) tool_call_valid 0.981 → 0.987 +0.6pp judge_agreement κ = 0.81 vs human labels p95_latency 2.4s → 2.9s +0.5s ⚠ within budget ✓ gate passed: improvement is significant, no regressions
What you'll learn
The quality stack for AI in production
Evals tell you if a change made things better. Observability tells you what happens in production. Deterministic tests cover everything around the model. You need all three.
Eval datasets & LLM judges
Datasets sized so a 2% improvement is real, and judges calibrated against human labels.
Agent evaluation
Grade trajectories and tool calls, not just final answers.
AI observability
Tracing, dashboards, and alerts for quality, cost, latency, and drift in production.
Deterministic test automation
Repeatable UI, API, and contract tests for everything around the model.
CI quality gates & infrastructure
Pipelines and eval infrastructure that block regressions without blocking real improvements.
Voice & speech AI
STT, TTS, and voice agents: WER, latency, and streaming quality.
Learn
Hands-on learning tracks
Practice-first labs. Build a system, break it on purpose, and measure what changed.
48 episodes · Beginner → Intermediate
AI Engineering: Hands-on
Prompts, memory, context, search, RAG, evals, reasoning, agents, fine-tuning, vision, voice, serving, resilience, observability, safety and compliance, measured on recorded runs.
7 episodes · Intermediate
MCP Servers: Build, Secure, Evaluate
Connect a real MCP server, prove tenant boundaries, and test tool use before release.
10 episodes · Intermediate
Voice AI Evaluation
Speech-to-text, text-to-speech, and voice agents.
1 episode · Intermediate
System Design, Measured
Every box on the whiteboard built on a laptop, put under load, and given a number.
14 labs · Beginner → Intermediate
LLM Evals: Foundations
How many evals are enough, and how to trust an LLM judge.
9 labs · Intermediate
Agent Evaluation
Measure what agents do, not just what they say.
5 labs · Advanced
Support Agent: Full Lifecycle
One real agent, taken from spec to production evals.
YouTube
Latest videos
Hands-on episodes on testing and evaluating real AI systems.
Nov 2, 2026
My AI Said 30 Days, Then 45. The Model Hadn’t Changed: AI Audit Logs and Replays, Tested
Nov 2, 2026
I Rewrote My AI’s Audit Log and the Check Said “Intact”: Tamper-Evident Audit Logs, Tested
Nov 2, 2026
My Policy Rules Blocked “Give Mia Chen a Rating”. Reworded, It Passed: AI Policy Checks, Tested
Writing
From the blog
Oct 29, 2026 · 13 min read
AI Writes the Code. Who Reviews It?
I planted 173 bugs in 250 pull requests and tested every way to review them, from people reading everything to AI only. The setup that won was simpler than the one I planned.
Oct 29, 2026 · 3 min read
Not Documentation. Experience: Teaching a Coding Agent with Claude Code Hooks
Every week I give my coding agent the same small corrections. Here's how I plan to use Claude Code hooks to turn them into lessons it keeps.
Oct 10, 2026 · 2 min read
How Many Evals Are Enough?
Your eval score went from 84% to 86%. Is that a real improvement, or noise? A practical guide to sizing eval sets.
Learning something useful?
New labs, articles, and videos on evaluating and testing AI systems come out regularly. Follow along so you don't miss the next one.