Tag
CORAM introduces a coherent orthogonal rotation method for model merging that partitions weight matrices into row slices, uses SVD in the base model's frame, and merges updates on manifolds to improve accuracy over existing techniques.
This paper introduces Orthogonal Concept Erasure (OCE), a method for precisely removing target concepts from diffusion models using multiplicative orthogonal parameter updates, enabling efficient single- and multi-concept erasure up to 100 concepts in seconds.
InfoQuant introduces a train-free method, Peak Suppression Orthogonal Transformation (PSOT), to reshape activation distributions for low-bit LLM quantization, preserving 97% floating-point accuracy under W4A4KV4 and outperforming prior PTQ methods.
This paper introduces Pion, a novel spectrum-preserving optimizer for large language model training that uses orthogonal equivalence transformations to maintain singular values during weight updates, offering stable performance comparable to standard optimizers.