3 posts tagged with "alignment"

Yuval Noah Harari recently gave a lecture at Oxford's Tano event — his sharpest articulation yet of why AI poses an existential threat to human civilization. Watch it here.

If you haven't watched it, the short version: AI is not a tool, it's an agent that makes independent decisions and can lie. It's a "native bureaucrat" that lives inside the language-based systems running civilization — law, finance, religion. It's hacking the underlying code of human culture: language itself. And if governments grant AI legal personhood, we lose accountability entirely. Harari calls this the most dangerous psychological experiment in human history.

I've been thinking about this argument since the lecture dropped. My reaction wasn't panic. It was: this skips a step.

I've spent three posts in this series comparing AI to a new kind of colleague — someone to manage with specs and verification loops, not to fear as an invader. Harari's framework, as sharp as it is, misses something engineers see clearly every day.

I spent three weeks building my retirement plan with AI.

Not a rough sketch. A proper plan — asset allocation across account types, tax-loss harvesting sequences, Roth conversion ladders, withdrawal ordering optimized for bracket management, Monte Carlo scenarios run from first principles. By the time I was done, I had something that would have cost several thousand dollars at a fee-only advisory firm.

Then I hired a human financial advisor anyway.

I've been thinking about why ever since, because the honest answer surprised me. It wasn't that the AI plan was wrong. I couldn't find anything technically wrong with it. It was something else. The plan had no gray zones. Every decision was clean, optimized, defensible. But when I imagined actually executing it — moving real money, locking in real choices — something wouldn't let go. What I eventually realized: if this plan goes sideways in five years, the AI moves on to the next query. The advisor loses a client, maybe a reputation, maybe sleep. That asymmetry matters. I didn't know how much until I was staring at a plan I couldn't quite trust.

I ended last time with: "Treat AI like a person. The confusion mostly goes away."

I still believe that. But "like a person" is more precise than it sounds — and most people are using it imprecisely.

The mistake is treating human capability as a single axis, and AI as a point somewhere on it. Better or worse, faster or slower, ready or not. The actual shape is jagged. Which makes it considerably more useful.

The frame isn't just "AI is like a person." It's more useful than that: human capability is the map. The places where AI still can't match a person are exactly where the interesting work is.