The Scaling Curve: Dario Amodei, Anthropic, and the Race to Build and Survive Superintelligence — Chapter Three

1. 1. OpenAI's early research initially explored diverse AI paths, but Dario Amodei's conviction that scaling was key led him to build a team proving its consistent efficacy.
2. 2. GPT-2, released in 2019, validated the scaling hypothesis for language by generating coherent text and exhibiting rudimentary reasoning, which particularly excited Dario Amodei.
3. 3. Concerns about GPT-2's potential for misuse, such as generating convincing fake news, led OpenAI to a controversial staged release, which Dario co-authored.
4. 4. In early 2020, Dario Amodei's team published "Scaling Laws for Neural Language Models," formally proving that AI performance predictably improves with increased parameters, data, and compute.
5. 5. GPT-3, released in June 2020 with 175 billion parameters, represented a monumental leap, performing dozens of complex cognitive tasks, including coding and few-shot learning.
6. 6. Dario Amodei viewed GPT-3 as both a vindication of the scaling hypothesis and a stark demonstration of AI's dangers, while also providing a humbling correction on the path to human-level intelligence.
7. 7. Despite their growing capabilities, models like GPT-3 had a 'fact-value gap,' lacking inherent helpfulness, honesty, or safety, reflecting the internet's diverse content.
8. 8. Dario Amodei strongly believed that AI safety must be a foundational, 'top-to-bottom' organizing principle for any AI organization, not merely a separate department.
9. 9. Dario Amodei and a group of collaborators ultimately left OpenAI to found Anthropic due to a fundamental disagreement on how to integrate safety and pursue a cautious vision for powerful AI.