weight-structure

This paper introduces 'row and column scale fields'—median-centred log-RMS profiles of Transformer weight matrices over their channels—to mesoscopically analyze how weight magnitude is distributed across functional channels, showing how these fields evolve through training, align across projections, relate to AdamW's second-moment structure, and how edits to them affect model loss.