capacity-allocation

Tag

Cards List
#capacity-allocation

Shaping Latent Mixture-of-Experts: From Expert Geometry to Layerwise Capacity Allocation

arXiv cs.AI ↗ · 2026-10-15 Cached

The paper proposes LatentMoE, which moves expert computation into a lower-dimensional latent space to treat expert input dimension and intermediate width as explicit design choices, showing that near-square experts and intrinsic-dimension-guided layerwise capacity allocation improve validation loss and downstream performance over layer-homogeneous MoE baselines.

0 favorites 0 likes
#capacity-allocation

Adaptive Two-Level Allocation of a Conserved Capacity Budget Across Locations and Service Classes

arXiv cs.AI ↗ · 2026-08-11 Cached

The paper presents a two-level algorithm for allocating a conserved capacity budget across locations and service classes, proving it conserves the budget and converges in one iteration, and evaluates it for defending CDNs under volumetric attack.

0 favorites 0 likes
#capacity-allocation

Generalization Dynamics of LM Pre-training (17 minute read)

TLDR AI ↗ · 2026-05-19 Cached

This paper reveals that during pre-training, language models frequently and suddenly switch between pattern-matching and generalization behaviors, a phenomenon called mode-hopping, and presents a toy evaluation suite to study it.

0 favorites 0 likes
← Back to home

Submit Feedback