dwarf-architecture

DWARF-55M-Base is a new language model using a nearly all-sparse attention architecture (DSQG) with a single full causal attention layer, achieving reliable retrieval up to 2048 tokens and extrapolating to 3x that context. It is released as a research prototype for community experimentation.