Michael Chenchard’s Reviews > Situational Awareness: The Decade Ahead > Status Update

These are great and honorable people. But they are just people. Soon, the AIs will be running the world, but we’re in for
one last rodeo. May their final stewardship bring honor to
mankind.

In particular, we may want to “spend some of our lead”
to have time to solve safety challenges, but Western labs will
need to coordinate to do so. (And of course, private labs will
have already had their AGI weights stolen, so their safety pre-
situational awareness 151
cautions won’t even matter; we’ll be at the mercy of the CCP’s
and North Korea’s safety precautions.)

its funny when he says "random nonprofit board" like why not just say OpenAI lol

It is a delusion of those who have unconsciously internalized
our brief respite from history that this will not summon more
primordial forces. Like many scientists before us, the great
minds of San Francisco hope that they can control the destiny
of the demon they are birthing. Right now, they still can; for
they are among the few with situational awareness, who understand what they are building.

There’s already an eerie convergence of AGI timelines (~2027?)
and Taiwan watchers’ Taiwan invasion timelines (China ready
to invade Taiwan by 2027?)—a convergence that will surely
only heighten as the world wakes up to AGI. (Imagine if in
1960, the vast majority of the world’s uranium deposits were
somehow concentrated in Berlin.)

America’s lead on AGI won’t secure peace and freedom by just
building the best AI girlfriend apps. It’s not pretty—but we
must build AI for American defense.

Some hope for some sort of international treaty on safety. This
seems fanciful to me. (How have those climate treaties
gone? That seems like a dramatically easier problem compared
to this.)

To date, US tech companies have made a much bigger bet on
AI and scaling than any Chinese efforts; consequently, we are
well ahead. But counting out China now is a bit like counting out Google in the AI race when ChatGPT came out in late
2022. Google hadn’t yet focused their efforts in an intense AI
bet, and it looked as though OpenAI was far ahead—but

But if there’s one thing China can do better than the US
it’s building stuff.

We’re not on track for superdefense, for
an airgapped cluster or any of that; I’m not sure we would
even realize if a model self-exfiltrated.