Activation

Pre-Norm / Post-Norm; RMSNorm; Activation function(SwiGLU FFN), Rope rotation and length-extrapolation (PI / NTK-aware / YaRN).