Foundations of Large Language Models
The map of the territory, and it happens to be free.
Xiao and Zhu organize the whole field around five load-bearing ideas — pre-training, generative models, prompting, alignment, inference — and keep the emphasis on what will still be true next year rather than this week's benchmark.
Where Raschka has you build one and Alammar has you apply one, this gives you the conceptual scaffolding to reason about all of them.
For directing technology around AI, it is the reference that turns a scattering of terms into a structure you can actually think with — openly available on arXiv, which is exactly how foundations should travel.
Central argument
A free, rigorous foundations text that organizes the field around five pillars — pre-training, generative (prompting) models, prompting, alignment, and inference — and deliberately privileges durable concepts over the latest state-of-the-art. It aims to be the coherent conceptual reference a newcomer or a practitioner can reason from, rather than an exhaustive catalogue of techniques.
Critique
By design it is a concepts-and-survey book, not a hands-on one: little runnable code, and it abstracts over the messy engineering (data pipelines, serving, the economics of scale) where much of the real difficulty lives. Its value is orientation and vocabulary; pair it with a from-scratch build and an applied guide for the concrete side.
Why it matters for product
For a CTO/CPO, this is the single most efficient way to acquire a correct, structured vocabulary for the whole field — what pre-training actually is versus alignment versus inference — in one free, internally consistent reference. It is the text to reason from when the decision is strategic ('where does this capability come from, and what would it take to change it?') rather than tactical.
- The Rise of AI Search: Implications for Information Markets and Human Judgement at Scale — Sinan Aral, Haiwen Li & R. Zuo
- Hands-On Large Language Models — Jay Alammar & Maarten Grootendorst
- Build a Large Language Model (From Scratch) — Sebastian Raschka
- Understanding Deep Learning — Simon J.D. Prince
- Deep Learning with Python — François Chollet
- Artificial Intelligence: A Modern Approach — Stuart Russell & Peter Norvig