Skip to main content

One post tagged with "reward-hacking"

View All Tags

The Ops Inflection, Contested: Who Verifies the Verifier

· 8 min read
Austin Xu
Cloud Platform Engineering Leader

A person facing a translucent holographic figure across a control room, both surrounded by a wall of glowing cyan data screens

At the 2026 Agentic AI Summit at UC Berkeley, five people from five unrelated fields said the same sentence, independently, within two days of each other.

  • Oriol Vinyals, VP of Research at Google DeepMind, on recursive self-improvement: "Evaluating this truly in an agentic way might take some effort. Currently it's the all-automation bottleneck."
  • Wang Mengdi, professor of electrical and computer engineering at Princeton, on AI for science: "Verification has become the major bottleneck for scaling any AI models."
  • Adarsh Hiremath, co-founder and co-CEO of Mercor, on enterprise deployment: "Evals in a large part are the bottleneck to successfully deploying agents in a company."
  • Sergei Gukov, professor of theoretical physics and mathematics at Caltech, on mathematical discovery: "Your system is going to be just as good as evaluator."
  • Vincent Chen, research fellow at Snorkel AI, on measurement itself: "Our ability to measure AI has really been outpaced by our ability to develop it."

Recursive self-improvement, natural science, enterprise software, pure mathematics, measurement theory. Five fields that share almost no vocabulary landed on the same diagnosis. That kind of convergence, across domains with no reason to be reading each other's papers, is the strongest kind of evidence a claim can get.