ell
Uncovering Batch & Layer Normalization
Batch normalization and layer normalization improve training stability and reduce sensitivity to initialization by normalizing intermediate activations.
Beyond FLOPs: How COSMA Builds Parallel Matrix Multiplication from Communication Bounds
Reading Grzegorz Kwasniewski et al., Red-blue pebbling revisited: near optimal parallel matrix-matrix multiplication (2019, arXiv v3). Original paper Suppose a …
When the Answer Arrives Late: Early-Exit Decoding in Modern LLMs
A reading of early-exit decoding in modern LLMs: layer similarity, oracle skip ratios, training effects, and the gap between skipped computation and real speed.





