in-context-learning

This paper proposes TdSciNER, a type-driven multi-task learning approach that leverages LLMs to improve scientific named entity recognition by filtering entity types, adding an auxiliary typing task, and using a demonstration selection strategy. Experiments on three datasets show performance comparable to fully supervised models.

The paper proposes BC-ICL, a bootstrap-conditioned action selection method that leverages pretrained tabular foundation models with in-context learning for contextual bandits, improving exploration and regret performance under strict online protocols.

This paper introduces BDH-CQ, a 150M-parameter reasoning model that combines in-context learning with recurrent latent reasoning, achieving 29.5% pass@2 on ARC-AGI-1 at very low inference cost and establishing a new cost-accuracy frontier.

Align-RAG introduces a training-free, closed-form alignment method for retrieval-augmented forecasting with frozen Time Series Foundation Models, outperforming learned fusion adapters on standard benchmarks without any learned parameters.

This paper investigates using retrieved document-simplification examples to guide LLM prompting for document-level text simplification, showing improvements over prompt-only generation on the OneStopEnglish corpus.

EdgeLM is a research paper proposing a retrieval framework that selects edge demonstrations—relevant examples near decision boundaries—to improve LLM performance on table understannding and data wrangling tasks.

This paper investigates how language models execute in-context conditional rules by probing whether testing and routing are separable mechanisms. Using activation patching across three open models and six languages, the authors find that predicate testing is modular while route representations are token-bound and non-transferable.

This paper applies graph signal processing to analyze how LLMs internally represent numerical sequences during in-context learning, finding that attention-induced token graphs and hidden-state signals show systematic, context-dependent signatures related to input complexity.

Introduces ContinualSkillBench, a dynamic evaluation framework for in-context continual skill learning in LLM agents, showing that while sequential execution improves performance, current methods struggle to consolidate experience into robust, transferable skills.

This paper explores using the PluRel synthetic relational database generator as an external data source for pretraining RDB-PFN, a relational in-context learner, demonstrating that schema-guided curriculum design can recover most of the original performance with far fewer pretraining tasks.

This paper studies how curriculum design and synthetic data diversity affect relational Prior-Data Fitted Network (PFN) pretraining, showing that progressive curricula can match or nearly match RDB-PFN's performance with far less synthetic data.

This paper proposes CoRA, a gradient-free framework for task-conditioned retrieval in on-device in-context learning, using frozen encoders and closed-form ridge regression to build compact retrieval bases without fine-tuning or backpropagation.

This paper investigates context sampling for TabPFN on small tabular datasets, finding that context diversity and coverage are more important than distribution matching for accuracy, and that random sampling is effective.

This paper introduces an evaluation protocol with four new metrics and a benchmark dataset to assess context attribution methods for LLMs, showing they fail when context overlaps with training data.

NVIDIA unveiled Ising Calibration 1.5, an open-source vision language model that fully automates quantum computer calibration with enhanced in-context learning and improved performance, now deployable on a single GPU.

Proposes Experience Distillation, a method that internalizes in-context learning gains from agent interaction histories into model weights without requiring additional environment interaction, achieving significant sample efficiency improvements on software engineering and text-adventure tasks.

This paper introduces LeadTime-ICL (LT-ICL), a censoring-aware in-context learning model for probabilistic supplier lead time forecasting. It combines a transformer backbone with a normalizing flow head and demonstrates strong performance across 24 industrial supply chain datasets without task-specific retraining.

This paper presents a mechanistic analysis of induction in masked diffusion language models, identifying a bidirectional induction circuit and showing that these models use the global fraction of masked tokens as an implicit timestep.

This paper constructs a transformer with linear self-attention that performs in-context learning of the closed-form least squares solution for simple linear regression, using layer normalization to approximate the analytical solution rather than gradient descent.

This paper proposes a reformulation to apply tabular foundation models (TFMs) to discrete choice estimation, addressing the structural gap of row-independent assumptions. The best reformulation outperforms hierarchical Bayesian estimation by 8% in holdout log-likelihood and 3.6% in hit rate while running 16 times faster.