3 posts tagged with "platform-engineering"

A paper at WWW 2025 made a clean empirical case that constraining an LLM-based RCA agent with Standard Operating Procedures improves accuracy. The reasoning is sound: LLMs hallucinate, and SOPs pin them down at the junctures where hallucination hurts most.

Through 2026, nearly every commercial AI SRE product shipped the opposite bet. Cleric, Resolve.ai, Traversal, Datadog's Bits AI, Microsoft's Azure SRE Agent — all of them investigate incidents by generating hypotheses and testing them against evidence. One industry survey puts it flatly: modern AI SREs "are not runbooks that execute predefined steps; they are agents that plan multi-step investigations based on the specific alert."

The academics proved constraints help. The vendors bet on reasoning. Both sides have data, and both can't be right. Unless they're right about different points on a curve — and the curve is the interesting part.

In The Ops Inflection, I described three architectural patterns moving the TOIL wall: principles-based agents, knowledge graphs with complete signal access, and independent validation agents. That was a field report from teams I know. Since then I went checking the industry at large. All three patterns are out there, at very different maturity levels. And underneath all three, the same argument keeps happening.

Two posts set up predictions that are now testable against field data.

DevOps at Scale predicted that organizations with mature DevOps — real SLO discipline, health-gated CI/CD, systematic observability — would be far better positioned for AI-native infrastructure than organizations that treated DevOps as theater. Who's Driving the AI-Native Organization? identified the TOIL wall: the empirical finding that Ops automation is dramatically harder than Dev automation, explained through the open/closed problem lens. The first 60% of operational toil is closed — known failure modes, documented runbooks. The stubborn residue behaves like open problems — novel situations, ambiguous signals, judgment calls about what the problem even is.

What I said then was that the wall is real. What I didn't say was whether it's permanent. This is the progress report on both predictions — and the argument for why the wall moves.

Every organization that has gone through a "DevOps transformation" in the last decade has a story. Most of those stories end the same way: they hired a DevOps team, bought a set of tools, and then wondered why things didn't meaningfully change.

I've been building and running infrastructure at scale for 20 years — from private cloud on OpenStack at a large-scale e-commerce platform to managing 200+ Kubernetes clusters, 50,000 nodes, and 5,000+ applications. If there's one thing I've learned, it's that the most common implementation of DevOps is actually an anti-pattern.

Let me explain what I mean.