Above All

Frontier LLM Notes

Target

从最前沿的技术报告出发, 学习 LLM 最核心的技术, follow 时代的前沿, 同时 top down 学习对应的基础知识
培养理论 && 工程能力
- 从 theory 和实验培养 LLM Training/Architectures 上的经验和直觉
- 从 engineering 角度增强 AI 时代的工程能力，体会高效的系统架构设计, 始终 update-to-date llm infra.

Outlines

LLM Infra:
- Slime: PostTraining Pipelines for RL Scaling
- Megatron: Model Training for gpu accerlations.
- SGlang: Model Inference Systems.
Model Architecture: Advanced LLM architectures (Attentions, MoE, and more!)
Model Training:
- Pre-Training: pretraining data, pretraining dynamics/optimizers.
- Mid-Training: mid-training data recipe & cleaning
- Post-Training:
  - Frontier PostTraining
  - Agentic RL & OPD

Model Architecture

Base Model Architecture

Advanced LLMs 4: Mixture of Experts

MoE: Routing, balancing and training.

Advanced LLMs 3: FLOPs and Decoding

FLOPs Analysis, Speculative Decoding, Multi-Token Prediction.

Advanced LLMs 2: Normalizations, FFNs & RoPE

Pre-Norm / Post-Norm; RMSNorm; Activation function(SwiGLU FFN), Rope rotation and length-extrapolation (PI / NTK-aware / YaRN).

Advanced LLMs 1: Attentions

Frontier LLM's Attention (MQA, GQA, MLA, Linear Attention Family, Flash Attention Family, Sparse Attention Family, Advanced Attentions)

Pre-Training

Pretraining data, dynamics and optimizers.

PreTraining Basics

Pre-training math basics (autoregressive token modeling, MLE / NLL training objectives, training gradients, KL divergence)

PreTraining Data Selection

Pre-training data recipes & paper list (data sampling selection, data filtering / de-duplication, model-based data scoring)

Distributed Parallel Systems for LLM Pre-Training

LLM Pretraining Parallelism (GPU Communication, DP/DDP, ZeRO, FSDP, TP, SP, CP, PP, EP, Mixed Precision)

Optimizers

Classical optimizers for LLM pretraining. (SGD, Adam, MUON)

Mid-Training

Mid-training: papers and tech reports. (Continual pretraining, data recipe, retention)

MidTraining Overview

Mid-training paper list & tech reports. (Continual pretraining, retention, RL scaling)

Post Training

Frontier LLM post-training: data mixture & post-training algorithms.

Agentic RL

From PPO, GRPO to frontier agentic reinforcement learning

Agentic RL 2: RL for LLMs

RL Methods for LLM PostTraining (vanilla-ppo, RLHF-PPO, DPO, RLVR, DeepSeek-Math-GRPO)

Agentic RL 3: From Tokens to Trajectories, Long-Horizon Reinforcement Learning

What is after GRPO? Advanced long-horizon agentic reinforcement learning. (DAPO, GSPO, SAO, FlashREINFORCE, more advanced training recipes.)

Agentic RL 1: From Policy Gradient to PPO

From RL Basis to Policy Gradient (reward-to-go, advantage estimations), REINFORCE, Actor-Critic, PPO (Importance Sampling and Clipping)

Frontier Post Training Pipelines

Post-training analysis & summaries from frontier base model tech reports (continuously updated).

LLM Infra

LLM Codebase & Infra

Slime 101

THUDM Slime infra: bridging Megatron and SGLang for RL-scaling LLM post-training.

Slime101: Slime Core Training Pipeline

Core Slime training pipeline analysis. (Ray, Megatron, SGLang)

Frontier LLM Notes#

Target#

Outlines#

Frontier LLM Notes

Target

Outlines