Tag
A Unified Particle Filter LSTM framework is proposed for data-driven process simulation, maintaining weighted recurrent-state hypotheses to handle partial observations in event logs, and outperforming baselines in reproducing routing and duration behaviors.
The paper introduces an audit method for prefix invariance in attention, state-space, and hybrid sequence models, demonstrating that attention masks fail to ensure causality, with 192 faults detected across multiple model checkpoints.
Introduces the p-Spin Glass Network, a novel architecture for sequence models that achieves memory efficiency, sample efficiency, and single-batch stability, enabling continual learning and edge AI applications.
This paper evaluates GRU, LSTM, and Transformer encoder models for classifying Level 2 automated driving systems from vehicle telematics data, and introduces a robustness evaluation framework simulating realistic data corruption.
This paper introduces a leakage-aware evaluation framework for Ethereum actor classification, comparing tree-based models like XGBoost against sequential deep learning models (Transformer, BiLSTM). The authors find that XGBoost outperforms sequence models under leak-reduced conditions while offering lower latency and energy use.
Black-Mamba introduces a test-time adaptive forecasting architecture that uses accumulated surprisal to selectively update memory only upon evidence of distribution drift, achieving efficient adaptation on non-stationary time series.
Explores the growing memory bottleneck of KV-cache in transformer inference, explaining why alternative architectures with fixed-size memory like Mamba and RWKV are gaining renewed attention.
An update on Matrix Recurrent Units (MRU), a linear-time attention alternative. The author explores methods to stabilize training, finding that orthogonal matrices underperform while LDU factorization works best, and shows MRU underperforms transformers on larger datasets like TinyStories.
This paper introduces RecurrReason, a difficulty-controlled benchmark of four symbolic logic puzzles to evaluate multi-step reasoning in sequence models. Fine-tuning experiments on T5 and GPT-2 show that architecture determines success more than scale, and that pre-training transfer depends on local transition structure.
This paper introduces a residualization-and-permutation diagnostic to separate predictability-driven from regulation-driven variance in regulatory importance scores from genomic foundation models, applied to dark genome elements at glioma-relevant loci.
SHARP introduces a bio-inspired framework that separates memory accumulation from pattern recognition, using accelerated replay during offline sleep phases to learn long-range non-stationary temporal patterns in streaming settings. It improves context retention on text8 and PG-19 while maintaining computational efficiency.
This paper formalizes the sufficiency gap in next-token prediction, demonstrating that even ideal sequence models can become overconfident when textual prefixes are not sufficient statistics for latent circumstances. It proposes an external observer mechanism to reduce but not eliminate this gap.
This paper introduces Conditional Attribute Transformers, a method for jointly estimating next-token probability and attribute values conditionally, enabling credit assignment, counterfactual analysis, and steerable generation in a single forward pass.
This paper introduces Toeplitz MLP Mixers (TMM), a novel architecture that replaces attention with Toeplitz matrix multiplication to achieve lower computational complexity while maintaining high information retention and training efficiency.