in-context-learning

This paper introduces DOPA, a demonstration search framework that uses an out-of-distribution proxy to retrieve robust demonstrations for LLMs when the target domain is inaccessible, enhancing in-context learning performance under distribution shift.

This paper proposes using pairwise queries to improve selective classification for binary classification, particularly where confidence estimates are inconsistent, as in LLM in-context learning. Theoretical conditions and experiments on synthetic and real datasets show that pairwise query-based algorithms achieve better accuracy-cost tradeoffs than raw confidence estimates.

This paper investigates how large language models reorganize representational geometry during in-context learning, showing that ICL performance correlates with the geometric structure of tasks and that successful ICL involves increasing separability of representations.

This paper introduces a lightweight multimodal LLM-based framework for cost-effective defect grading of power transmission equipment, using in-context learning and chain-of-thought to generate training data and fine-tuning Qwen3-VL-8B for state-of-the-art performance.

This paper reveals a counterintuitive phenomenon where correct demonstrations in in-context learning can degrade model accuracy, introducing task preserving perturbations to study the gap between exemplar correctness and utility.

This paper studies retrieval-augmented generation as an in-context optimization process, showing that linear self-attention can implement gradient descent on a unified RAG objective. It proposes a lightweight method for frozen RAG LLMs that predicts context-conditioned updates, improving performance across multiple QA benchmarks.

This paper presents MEMOR-E, a mobile quadruped robot with a tablet interface that uses fine-tuned and in-context learning with LLMs to provide personalized, stage-aware cognitive assistance for Alzheimer's patients, including medication reminders and memory interactions, with explainable AI for caregiver oversight.

This paper proposes a method to improve in-context learning by optimizing the continuous embeddings of a fixed few-shot prompt at test time, using a self-supervised confidence proxy derived from the model's log-probabilities without requiring fine-tuning or token generation.

This paper introduces Reflection-Augmented Scaling (RAS), a method that uses execution feedback from failed Cypher queries to iteratively refine query generation via in-context learning, reducing execution error rates by 41-50% across multiple datasets and models.

Discusses that the mathematics used by AI is mainly linear algebra, calculus, etc., from before the 19th century, but emerging phenomena such as Scaling Law, emergent abilities, double descent, in-context learning, and representation geometry lack mathematical explanation. Analogizes to the clouds in physics in 1900, suggesting it may drive the development of 21st-century mathematics.

This paper adapts classical class imbalance techniques to Prior-Data Fitted Networks (PFNs) for tabular classification, finding that thresholding and downsampling perform well due to PFNs' calibration and limited-data capabilities.

This paper proposes using distributional alignment between task vector-based and in-context learning inference as a criterion for designing task vectors, and introduces Linear Task Vector (LTV) that minimizes next-token probability discrepancy via closed-form linear mapping. LTV achieves 9.2% average accuracy improvement over baselines across eight benchmarks and five LLMs.

TabPFN-MT extends PFNs to multitask in-context learning for tabular data, achieving state-of-the-art on small-to-medium datasets while reducing inference cost from O(T) to O(1) forward passes.

This paper proposes that in-context learning in LLMs operates through low-dimensional concept subspaces, where task-relevant information concentrates in a small fraction of the representation space, supported by experiments on Llama-3-8B and Qwen2.5-7B.

This paper introduces ACIL, an automatic Chain-of-Thought framework to enhance In-Context Learning by generating and pruning reasoning chains, improving LLM performance on complex tasks.

SurvivalPFN is a prior-data fitted network that amortizes Bayesian inference for survival analysis via in-context learning, achieving strong predictive performance across 61 datasets without task-specific training or hyperparameter tuning.

FashionChameleon is a real-time, interactive framework for human-garment video customization that uses teacher-student distillation and in-context learning to enable multi-garment switching while maintaining motion coherence, achieving 23.8 FPS on a single GPU.

This paper investigates many-shot chain-of-thought in-context learning for reasoning tasks, revealing that standard scaling rules do not transfer and proposing Curvilinear Demonstration Selection (CDS) for improved ordering, achieving up to 5.42 percentage-point gain.

This paper investigates whether LLMs learn in-context through latent structure inference or local pattern matching, using mechanistic interpretability methods like PCA and activation patching on a graph random-walk task.

This paper investigates using Large Language Models, specifically Claude, interfaced with a Computer Algebra System (Maple) to perform algorithmic computations in theoretical physics, such as analyzing cosmological perturbations.