PART: brief contentsp. 8(4)
CHAPTER: 1 ■ Understanding reasoning modelsp. 8
CHAPTER: 2 ■ Generating text with a pretrained LLMp. 8
CHAPTER: 4 ■ Improving reasoning with inference-time scalingp. 8
CHAPTER: 5 ■ Inference-time scaling via self-refinementp. 8
PART: contentspp. 10–14(62)
CHAPTER: 1 Understanding reasoning modelspp. 10–11
CHAPTER: 1.1 Defining reasoning in the context of LLMsp. 10
CHAPTER: 1.2 Understanding the standard LLM training pipelinep. 10
CHAPTER: 1.3 Improving LLM reasoning with training and inference techniquesp. 10
CHAPTER: 1.4 Pattern matching vs. logical reasoningp. 10
CHAPTER: 1.5 Simulating reasoning without explicit rulesp. 10
CHAPTER: 1.7 A road map to building reasoning models from scratchp. 10
CHAPTER: 2 Generating text with a pretrained LLMpp. 10–11
CHAPTER: 2.1 Introducing LLMs for text generationp. 10
CHAPTER: 2.2 Setting up the coding environmentp. 10
CHAPTER: 2.3 Understanding hardware needs and recommendationsp. 10
CHAPTER: 2.4 Preparing input texts for LLMsp. 11
CHAPTER: 3.2 Loading a pretrained model to generate textp. 11
CHAPTER: 2.6 Understanding the sequential LLM text generation processp. 11
CHAPTER: 2.7 Coding a minimal text generation functionp. 11
CHAPTER: 2.9 Faster inference via PyTorch model compilationp. 11
CHAPTER: 3.1 Building a math verifierp. 11
CHAPTER: 3.3 Implementing a wrapper for easier text generationp. 11
CHAPTER: 3.4 Extracting the final answer boxp. 11
CHAPTER: 3.5 Normalizing the extracted answerp. 11
CHAPTER: 3.6 Verifying mathematical equivalencep. 11
CHAPTER: 3.7 Grading answersp. 11
CHAPTER: 3.8 Loading the evaluation datasetp. 11
CHAPTER: 3.9 Evaluating the modelp. 11
CHAPTER: 4 Improving reasoning with inference-time scalingp. 11
CHAPTER: 4.3 Generating better responses with chain-of-thought promptingp. 11
CHAPTER: 4.4 Controlling output diversity with temperature scalingp. 11
CHAPTER: 4.5 Balancing diversity and coherence with top-p samplingp. 11
CHAPTER: 4.6 Improving response accuracy with self-consistencyp. 11
CHAPTER: 5 Inference-time scaling via self-refinementp. 12
CHAPTER: 5.1 Scoring and iteratively improving model responsesp. 12
CHAPTER: 5.2 Loading a pretrained modelp. 12
CHAPTER: 5.3 Scoring LLM responses with a rule-based scorep. 12
CHAPTER: 5.4 Understanding token probability scoresp. 12
CHAPTER: 5.5 From token probability scores to log probabilitiesp. 12
CHAPTER: 6.8 Scoring rollouts with sequence log probabilitiesp. 12
CHAPTER: 5.7 Self-refinement through iterative feedbackp. 12
CHAPTER: 5.8 Coding the self-refinement loopp. 12
CHAPTER: 6 Training reasoning models with reinforcement learningp. 12
CHAPTER: 6.1 Introduction to RL for LLMsp. 12
CHAPTER: 6.2 RLVR using GRPOp. 12
CHAPTER: 6.4 Loading a MATH training subsetp. 12
CHAPTER: 6.5 Sampling rolloutsp. 12
CHAPTER: 6.6 Calculating rewardsp. 12
CHAPTER: 6.7 Preparing learning signals from rollouts via advantagesp. 12
CHAPTER: 6.9 From advantages to policy updates via the GRPO lossp. 12
CHAPTER: 6.10 Putting everything together in a single GRPO functionp. 12
CHAPTER: 6.11 Implementing the GRPO training loopp. 12
CHAPTER: 6.12 Loading and evaluating saved model checkpointsp. 12
CHAPTER: 7.2 Tracking GRPO performance metricsp. 12
CHAPTER: 7.3 Tracking more advanced GRPO performance metricsp. 13
CHAPTER: 7.4 Stabilizing sequence-level GRPO using clipped policy ratiosp. 13
CHAPTER: 7.5 Controlling how much the model changes with a KL termp. 13
CHAPTER: 7.6 Adding an explicit format rewardp. 13
CHAPTER: 8 Distilling reasoning models for efficient reasoningpp. 13–14
CHAPTER: 8.1 Introducing model distillation for reasoning tasksp. 13
CHAPTER: 8.3 Loading the MATH training dataset for distillationp. 13
CHAPTER: 8.4 Building training examplesp. 13
CHAPTER: 8.8 Evaluating the distilled modelp. 13
CHAPTER: 8.6 Computing the training and validation lossesp. 13
CHAPTER: 8.9 Future directions for reasoning modelsp. 13
CHAPTER: 8.10 Conclusionsp. 13
Detection Mode
Automatic — pipeline chooses the best method