hierarchical

Introduced NOML, a custom reinforcement learning algorithm for continuous flight control that uses a hierarchical actor, anchor policy, and mirror learning to prevent oscillation and improve stability. The code is open-sourced on GitHub.

Lighthouse Attention is a selection-based hierarchical attention mechanism that accelerates long-context pretraining by running forward+backward passes ~17× faster at 512K context and delivering 1.4–1.7× end-to-end speedup at 98K context, validated with Llama-3 530M on 50B tokens.