tagged · 7 articles
agents
Articles
- 2026
- 2026
Every verdict held. The score moved anyway.
An integration test has an oracle. An eval has a metric — and the metric is an accounting procedure nobody reviews. Four graders from four vendors agreed on every substantive verdict I gave them, three times each, and the pass rate still moved. Here is where an eval number actually comes from, why agents break it worse than models do, and the three things I now build before I report one. - 2026
What your evals never read
Your eval suite scores the function. It never reads the sentence the model reads before deciding to call that function, and that sentence is the one that can lie, be tampered with after you ship, or be obeyed differently on the very next run. First in a series on what agent evals actually have to measure. - 2026
Task was destroyed but it is pending
A teardown deadlock I diagnosed in Google's A2A reference SDK — Issue #1101, PR #1105 (417 added lines, 312 of them tests, CI green, merged upstream) — turned out to be one symptom of a primitive the whole agent ecosystem is missing: a sound signal that an async mesh has actually settled. - 2026
When is the swarm actually done?
Three services across a network, one dropped task, and a question nobody on the wire can answer: did the work actually get done? A copy-paste demo that exits 0 and prints SUCCESS while the real job dies unseen. - 2026
The first file an agent reads
Coding agents read your library before they use it, and they start with the code — `__init__.py`, the type hints, the tool schemas — not your docs site. Here is how I made waitbus speak to that reader, and the one piece of documentation I deliberately did not ship. - 2026
Your AI coding agents can't hear each other — not even across vendors
Your Claude Code, your Cursor, your tests, your CI — all on one box, none able to hear the others finish or fail. Here's the problem, and the proof: five real LLM agents on one bus, one fails, all wake.