Tag
Pico-type is a 1.5M-parameter byte-level multi-head classifier that predicts seven content properties from raw UTF-8 bytes in a single forward pass, designed for efficient on-device inference with minimal resources.
They released a unified multi-head security classifier (mmBERT-small encoder with seven heads) using masked losses for absent task labels, achieving high F1 scores across seven tasks, and also released quantized ONNX INT8 variants. Weights and metrics are public.
This paper identifies memory retention as the bottleneck in recurrent memory agents for long contexts and proposes Multi-Head Recurrent Memory (MHM), a training-free framework that partitions memory into independent heads with a select-then-update strategy. The lightweight instantiation MHM-LRU significantly improves retention and end-to-end accuracy across 100K–1M token ranges, raising retention from below 30% to 73.96% on RULER-HQA at 896K tokens.