Perimeter Institute for Theoretical Physics · Training Sand to Think: Artificial General Intelligence & Future of Physics

1. 1. The speaker, a theoretical physicist, shifted his focus from writing physics papers to contributing to the production of general intelligence machines.
2. 2. Large Language Models (LLMs) are neural networks that are 'grown, not programmed,' trained by predicting the next word in vast text corpora like the entire internet.
3. 3. Physicists initiated the modern LLM boom by discovering scaling laws, which showed that model performance predictably improves with increased compute resources.
4. 4. The primary drivers of LLM progress in the last decade are algorithmic improvements, followed by exponential growth in compute investment, with chip advancements being a minor factor.
5. 5. LLMs rapidly surpassed human performance on high school math benchmarks and later graduate-level science exams like GPQA within just a few years.
6. 6. LLMs achieved a gold medal score in the International Math Olympiad, proving their creative problem-solving ability beyond mere memorization.
7. 7. Centaur-style mathematical research, where humans and LLMs collaborate, has already produced entirely novel mathematical arguments.