Sutton, Barto RL book maps LLM path

The chapter shortlist is directionally right, but it mixes core foundations with exactly the parts that matter most for modern RLHF-style systems. For LLMs, the useful bridge is less about textbook control and more about approximate methods, policy gradients, and preference-driven optimization.

• –Chapters 1, 3, and 6 are the right base: they establish MDPs, bootstrapping, and temporal-difference learning.
• –Chapters 9-11 and 13 are more relevant to LLM work than planning-heavy material because modern RL for language models leans on function approximation and gradients.
• –The Alberta reinforcement learning courses are a stronger structured path than reading the book alone if the goal is to move from theory into practice.
• –For RL-for-LLMs specifically, add RLHF-focused material early; classic Sutton and Barto explains the vocabulary, but not the training stack most people use today.
• –Tool use, agents, and math reasoning only become "RL" in the useful sense when you care about interaction, credit assignment, or preference optimization.