LLMs from Scratch

Best for: developers who want to genuinely understand how large language models work by building one, rather than reading about it.

This repository by Sebastian Raschka contains the complete code for developing, pretraining and finetuning a GPT-like LLM in PyTorch, and is the official companion to his book Build a Large Language Model (From Scratch). It works through every stage step by step, with the material presented as Jupyter notebooks.

What makes it useful: the scope is deliberately achievable. Rather than gesturing at frontier-scale training, it guides you through building a small but functional model you can actually train yourself, with attention mechanisms, finetuning and the surrounding machinery explained through clear text, diagrams and runnable code.

Where it falls short: this is a learning resource, not a production framework — nothing here is intended to be deployed. You need working PyTorch knowledge and enough patience to read notebooks in order; skipping ahead does not work well. Licensing is non-standard (NOASSERTION), reflecting its tie to a commercially published book, so check terms before reusing the material in teaching.

Details: Jupyter Notebook, PyTorch. Companion to a published book.