Tag
This paper investigates hybrid quantum-classical neural networks for the vehicle routing problem, finding that encoder feed-forward replacement can reduce model parameters by 56.6% while maintaining near-baseline performance for small to medium instances.
The paper presents Conservative Hybrid Graph Network (CHGN), a method for modeling dynamic process systems that allows extrapolation to unseen larger graphs without retraining and improves fault prediction in industrial applications.
A comparative benchmarking study evaluates various AI methods for renewable energy farm optimization and forecasting, showing that ensemble and hybrid approaches excel in different data scenarios.
The paper empirically characterizes the learning geometry of hybrid quantum forecasting models, comparing them to classical baselines using Neural Tangent Kernel dynamics and other metrics, showing that similar generalization can emerge from different optimization trajectories.
HyGenQ is a post-training quantization framework for hybrid iterative generative models that addresses challenges like excessive outliers and amplified anomalies, enabling 8-bit precision quantization while maintaining generation quality.
This paper presents the first systematic study of massive activations in hybrid linear-attention LLMs, uncovering pre-attention spikes and inter-spike plateaus governed by cancellation timing, and showing how their morphology recovers at full-attention limits.
Introduces Multistage Defer Trees, a sequence of sparse decision trees that defer hard samples to later trees or a black box, aiming to match ensemble accuracy while keeping most predictions interpretable.
This paper analyzes token-level prediction differences between transformers and hybrid attention-recurrent models using Olmo 3 and Olmo Hybrid, finding that hybrids improve on semantic state tracking while transformers excel at n-gram copying and syntactic bracket matching.
A thread from Ai2 compares transformer (Olmo 3) and hybrid (Olmo Hybrid) models, finding that transformers excel at copying while RNNs better model meaning-bearing words, highlighting the growing viability of hybrid architectures.
A Zhihu contributor's half-year-old prediction that the next Transformer would absorb loops, recurrent state, sparse routing, and latent reasoning is gaining relevance as Loop Engineering advances. The article explores how future Transformer architectures may evolve into hybrid models blending linear-complexity layers for background context with attention for precise reasoning, plus finer-grained sparsity and native System 2 reasoning.
A study comparing Olmo Hybrid and Olmo 3 transformers at the token level shows hybrid models better predict meaningful tokens like nouns/verbs, while transformers excel at copying tokens from input.
This paper proposes PE-MHL, a Physics-Encoded Modular Hybrid Layer framework that incrementally refines a physics-based model with data-driven sub-models, providing theoretical convergence guarantees and outperforming monolithic networks on control benchmarks.
This paper presents a systematic literature review of hybrid approaches for interval wind speed forecasting, combining deep learning, modal decomposition, and statistical methods to enhance prediction accuracy and reliability.
Researchers from Fordham University introduce Reciprocal Co-Training (RCT), a framework that couples LLMs and Random Forest classifiers via reinforcement learning, creating an iterative feedback loop where each model improves using signals from the other. Experiments on three medical datasets show consistent performance gains for both models, demonstrating a general mechanism for integrating incompatible model families.
This paper presents Olmo Hybrid, a 7B-parameter language model that combines attention and Gated DeltaNet recurrent layers, demonstrating both theoretical and empirical advantages over pure transformers. The work shows that hybrid models have greater expressivity, scale more efficiently during pretraining, and outperform comparable transformer baselines.