foundation-models

Align-RAG introduces a training-free, closed-form alignment method for retrieval-augmented forecasting with frozen Time Series Foundation Models, outperforming learned fusion adapters on standard benchmarks without any learned parameters.

This paper investigates whether small foundation models fine-tuned on human behavioral data can serve as cognitive proxies, finding that scale matters little in-distribution but larger models generalize better out-of-distribution.

Liquid AI announces a partnership with MacPaw to bring on-device AI to Mac users, designing specialized Liquid Foundation Models for macOS AI assistance.

This paper identifies that semantic-shift jailbreaks are limited by overlooking the semantic-shift capability of contexts, and proposes Iterative Context Optimization (ICO), a black-box framework that iteratively optimizes contexts to achieve higher attack success rates against foundation models.

A survey paper introducing a functional role taxonomy for language grounding in embodied agents, distinguishing five roles and auditing evidence to assess whether language's contribution is genuinely supported.

This paper introduces CoCoS, a contrastive pretraining framework that learns whole-cell representations from complementary transcriptomic views, addressing limitations of masked gene reconstruction in single-cell foundation models. Experiments on cell-type annotation and gene regulatory network inference show competitive transfer performance.

This paper proposes Frontier Learning, a framework that combines representations and predictions from multiple black-box and white-box pretrained models to construct a unified target-domain representation, guaranteeing performance no worse than any individual reuse baseline under distribution shift. Evaluations on visual domain adaptation and clinical mortality prediction show consistent gains over strong baselines.

Introduces Obshazard-bench, a real-time, observation-driven benchmark for evaluating multimodal foundation models on disaster intelligence from raw Earth observation streams, spanning 8 disaster categories across 60+ countries.

This paper explores using the PluRel synthetic relational database generator as an external data source for pretraining RDB-PFN, a relational in-context learner, demonstrating that schema-guided curriculum design can recover most of the original performance with far fewer pretraining tasks.

This paper proposes FedSLM, a parameter-centric framework for federated fine-tuning of foundation models with heterogeneous compressed clients, using SVD-based decomposition and a weak-to-strong elicitation step to handle resource asymmetry. Experiments show it outperforms existing federated baselines while reducing client GPU memory by ~50%.

Presents NEXUS, a lightweight foundation model with ~3M parameters pre-trained on LHC collision data, demonstrating improved downstream performance and cross-domain transfer to gravitational waves, flood forecasting, and neural activity.

ECG-InterpBench is a new benchmark that systematically evaluates the interpretability of ECG foundation model representations using matched-scale sparse autoencoders, covering reconstruction fidelity, clinical concept accessibility, and reproducibility across 450 cells.

This paper demonstrates that scaling laws fit on small transformer models can accurately predict the loss of much larger models trained on particle physics jet data, enabling compute budgets to be translated into expected physics performance before large training runs. They release five pretrained models and the full training recipe.

The paper introduces memory foundation models, with Metis as the first prototype, which equips foundation models with native memory capabilities through a new architecture and training data.

Visual prompt engineering (VIPE) automatically modifies task images to improve video model reasoning performance, often more effective than text-based prompting or test-time scaling.

Andrew Chen discusses how AI creates a new 'smile curve' where retention and usage increase over time due to foundation model improvements, similar to past social, on-demand, and SaaS products.

SpecPrefetch proposes a parameter-efficient expert prefetching framework for sparse MoE models, using a lightweight adapter to predict next-layer experts for asynchronous transfer while preserving native routing semantics. It achieves up to 20% decoding throughput improvement on a Snapdragon 8 Elite device, demonstrating practical benefits for memory-constrained deployment.

Amazon is reportedly planning to consolidate its multiple Nova AI models into a single frontier model, shifting from a diverse portfolio approach.

SeT-Diff proposes the first foundation model for HPC telemetry, using diffusion conditioned on semantic sensor descriptions to enable zero-shot generalization across tasks like imputation, forecasting, and virtual sensing, achieving an MAE of 0.0470 on reconstruction.

This paper introduces Visual Prompt Engineering (VIPE), a method that automatically modifies task images to improve video model performance, showing it can be more effective than text-based prompt engineering or test-time scaling.