Advanced LLMs 3: FLOPs and Decoding
FLOPs Analysis, Speculative Decoding, Multi-Token Prediction.
Kv-Cache
Advanced LLMs 1: Attentions
Frontier LLM's Attention (MQA, GQA, MLA, Linear Attention Family, Flash Attention Family, Sparse Attention Family, Advanced Attentions)