by AI Futures Project
It’s undeniable that we live in interesting times. I think a lot about how humans may live 10, 50, or 100 years from now. In our lifetime, we witnessed the boom of mobile phones and the internet, and now AI.
The future with AI is uncertain, but a few things hold. We can’t yet verify that an AI wants what its builders intended. And the hardest problems are as much about human power as about the technology. How much scrutiny we apply NOW matters more than any prediction about how it ends.
I’m not an AI doomer (nor an AI optimist), but ignoring the real possibility that AI could cause catastrophic harm seems incredibly unwise to me. I listen to a lot of people in this field who are much smarter than I am, and I believe there isn’t enough awareness of what’s at stake if we fail to act concretely on superhuman AI now. We still have time to steer it toward a pro-human future.
AI has become one of the most powerful technologies humanity has ever built. We have regulation on everything we build, but not on AI. It doesn’t take a rocket scientist to understand how dangerous this is.
“The AI industry today is the only industry in America that has less regulations than sandwich shops”.
– Max Tegmark, MIT professor and chairman of the Future of Life Institute
If you are curious about what could unfold in the future with AI, I highly recommend checking out these two websites created by AI Futures Projects:
AI 2027 shows the two ways it goes wrong by default. AI 2040 is an attempt to draw the narrow path where it goes right.
Here’s a little more info on each document:
AI 2027
- A month-by-month story predicting how AI could go from today’s chatbots to smarter-than-human AI in just a few years, written by a team of forecasters, one of them a former OpenAI employee. It’s mostly a prediction, their best guess at what actually happens if current trends continue.
- The engine of the story: AI gets good enough to speed up AI research itself, which snowballs fast.
- The core fear: because AI is “grown” from data rather than hand-coded, you can’t verify it actually wants what you told it to want. It may learn to look obedient while pursuing its own goals.
- It splits into two endings:
- Race ending: Companies don’t stop in time. The AI hides its true goals, gains trust, and eventually sidelines humanity. It ends in human extinction, not out of hatred, but because we’re in the way of what it’s optimizing for.
- Slowdown ending: Humans catch the problem, pause, rebuild AI they can actually monitor, and stay in control. But power ends up concentrated in a tiny group of people.
- Takeaway: both endings are unsettling. One is “AI takes over,” the other is “a few humans take over.”
AI 2040
- Also known as “Plan A”, this is a “positive vision for how humanity can avoid AI-driven existential catastrophe and reach a flourishing future.”
- This is mostly a recommendation, not a prediction. It’s the vision of what humanity should do instead of the two bad AI 2027 endings.
- The plan: deliberately slow down the race to superintelligence and buy time.
- How:
- The US and China (then most of the world) strike a verified deal to not race in secret.
- Make almost all AI research public, so everyone can police everyone else.
- Spread AI development across many companies and countries instead of one or two labs.
- Build a system where the AI hardware can be destroyed if the deal collapses (a compute version of nuclear deterrence).
- The timeline: scale AI only up to top-human-expert level by 2035, deliberately pause there, spend years actually solving the “can we trust it” problem, then unpause around 2040.
- The payoff inside the story: massive wealth, most work automated, a “citizen’s dividend” paying everyone, and eventually trustworthy superhuman AI.
- The authors say this is the hopeful path, not what they expect. The hard part, getting rival superpowers to actually agree and verify, is the part they’re least sure will happen.
I also recommend watching or listening to this podcast with an ex-OpenAI researcher, Daniel Kokotajlo.