Tag
This paper proposes using lightweight MLP probes on activations from LLaMA-3.1-8B to detect harmful prompts with high F1 scores, offering a cost-effective alternative to large external guardrail models.
This paper investigates the efficiency of superposition in latent states for reasoning in large language models, demonstrating that cumulative superposition can retain more usable evidence than frontier superposition, optimizing memory for downstream computation.
This paper investigates whether temporally drifting data streams can be partitioned into discrete regimes by fitting a hidden Markov model to the trajectory of neural network weights trained on successive time windows, showing that recovered latent states correlate with transfer performance across two datasets.
World Machine proposes a transformer-based generative world modeling architecture for time series that uses latent states to adapt to varying context lengths, addressing the quadratic memory cost of traditional transformers. Experiments on a synthetic dataset validate its feasibility and show improvements over conventional transformers.