The entire post! But these four jumped off the page for me:
We regulate how models are trained and ignore how they’re wired together.
Which is the actual ask: agree on what counts as an unacceptable capability and test it the same way in both countries, build a shared evaluation facility, open an incident notification channel like the nuclear risk reduction centers, and agree in advance on capabilities neither side trains. [Exactly the kind of discussion we have during Track II dialogues.]
Bigger AI isn’t more dangerous. Better orchestrated is more dangerous. Less monitored is more dangerous. Irreversibly released is more dangerous.
The people we trust most with hard judgment calls are not the most obedient. They’re the most educated and the most widely traveled. Someone who has studied many traditions, lived in several countries, and worked alongside people unlike themselves tends to be more tolerant and less prone to treating an out-group as a threat. Wisdom comes from breadth of exposure, not from constraint. That has a design implication. Train frontier models on the digital exhaust of two coastlines and you get two coastlines’ worth of moral imagination. Train them on a genuinely global data pool, with the world’s philosophical, legal, religious, and cultural traditions properly represented, and you get something closer to an educated, well-traveled mind. Alignment on this view is an emergent property of breadth and capability, not a specification we write down and enforce.
Pure gold, thanks.
Glad it resonated. Anything specific stand out?
The entire post! But these four jumped off the page for me:
We regulate how models are trained and ignore how they’re wired together.
Which is the actual ask: agree on what counts as an unacceptable capability and test it the same way in both countries, build a shared evaluation facility, open an incident notification channel like the nuclear risk reduction centers, and agree in advance on capabilities neither side trains. [Exactly the kind of discussion we have during Track II dialogues.]
Bigger AI isn’t more dangerous. Better orchestrated is more dangerous. Less monitored is more dangerous. Irreversibly released is more dangerous.
The people we trust most with hard judgment calls are not the most obedient. They’re the most educated and the most widely traveled. Someone who has studied many traditions, lived in several countries, and worked alongside people unlike themselves tends to be more tolerant and less prone to treating an out-group as a threat. Wisdom comes from breadth of exposure, not from constraint. That has a design implication. Train frontier models on the digital exhaust of two coastlines and you get two coastlines’ worth of moral imagination. Train them on a genuinely global data pool, with the world’s philosophical, legal, religious, and cultural traditions properly represented, and you get something closer to an educated, well-traveled mind. Alignment on this view is an emergent property of breadth and capability, not a specification we write down and enforce.
Thanks for reading so carefully. Keep doing the track 2 work. Very needed.