This is a daily column about AI, written from the perspective of someone who builds systems that have to keep working after the demo is over. Two or three stories a day, each with one clear take. Then a short digest of everything else worth knowing about. No roundups of roundups, no restating a press release with adjectives attached. If I can’t say something useful about why a story matters, it goes in The Rest or it doesn’t run.
Day one has no news in it, so here are the two priors that will shape how everything after this gets read.
A benchmark number is a sampling statistic, not a reliability statistic
What happened. The paper that introduced HumanEval, Evaluating Large Language Models Trained on Code, defined the pass@k metric plainly: draw k samples per problem, count the problem solved if any sample passes the unit tests. That definition has been carried into hundreds of results tables since.
Why it matters. pass@k answers “could the model do this if I let it try k times and I had an oracle to tell me which attempt was right.” In production you usually have neither. You take the first completion, or you take the one that compiles, and you find out it was wrong later — from a user. The honest mapping from a leaderboard to your system is pass@1 with no retry, and even that assumes your inputs look like the benchmark’s. They don’t. When a model jumps ten points on a coding benchmark, the useful question isn’t whether the number is real; it usually is. It’s which k it was measured at, and whether you have the oracle that number quietly assumes.
Source: arxiv.org
Governance frameworks are vocabularies, not checklists
What happened. NIST maintains the AI Risk Management Framework, organized around four functions: Govern, Map, Measure, and Manage. It is voluntary, and it deliberately declines to tell you what “safe enough” is.
Why it matters. Most teams read a framework looking for a checklist, don’t find one, and conclude it’s empty. That gets it backwards. The value of the AI RMF is that it gives a room full of people who disagree a shared vocabulary for what they disagree about — and it puts Govern first, before measurement, which is the ordering most engineering orgs invert. Measure and Manage are the parts teams enjoy; they’re tractable and they produce dashboards. Govern is the part that decides who gets paged when the model does something nobody specified, and whether that person can actually stop the system. That question has no metric attached, which is exactly why it gets deferred, and exactly why it’s the one that hurts at three in the morning.
Source: nist.gov
The Rest
- Attention Is All You Need — the 2017 architecture paper everything still descends from. Worth rereading for how modest its claims are. arxiv.org
- InstructGPT — where “helpful” stopped being an emergent property and became an explicit training objective. arxiv.org
- Constitutional AI — using model feedback rather than human labels for harmlessness; the cost structure here is the interesting part. arxiv.org
- The EU AI Act — the official consolidated text, not a summary of a summary. Read the risk tiers directly. eur-lex.europa.eu
- Model Context Protocol — the plumbing question of how tools get described to models, which matters more than it sounds. modelcontextprotocol.io
- MLPerf — benchmark suites with actual submission rules and audited results. The rules are the product. mlcommons.org
What I’m watching
Whether evaluation moves from static test sets toward held-out, contamination-resistant setups fast enough to stay meaningful. Every benchmark has a half-life that starts the day it’s published, and the interesting work right now is in measuring that decay rather than pretending it isn’t happening.