black-box

Discusses the unresolved problem of AI agents being able to build working apps but remaining untrustworthy black boxes when deployed unattended in production.

A developer built an AI agent that drafts sales decks, but adoption was near zero until the agent showed its reasoning behind each slide choice. The lesson: making agent decisions visible is more critical for trust than raw output quality.

This paper proposes a unified black-box robustness evaluation framework for token-level explanations in enterprise NLP, comparing encoder (BERT, RoBERTa) and decoder (Qwen, Llama) models. It finds decoder LLMs produce substantially more stable explanations, with stability improving with scale, and provides a cost-robustness tradeoff curve for pre-deployment model selection.

Introduces interventional grounding audits as a black-box, step-level test to check whether LLM chain-of-thought reasoning genuinely depends on its stated premises. Evaluated on ProntoQA with GPT-4o, achieving F1=0.806 on detecting proof-tree dependencies, significantly outperforming a self-consistency baseline.

A novel approach to model interpretability using a zero-parameter model that opens the black box of complex AI systems without requiring additional training or parameters.

The BABEL codec achieves the first complete decode of GPT-2 small's internal state, reconstructing 94.7% of its behavior and enabling reading and writing to the model in English, with open source resources and a demo.

This paper presents a black-box evaluation framework to assess LLMs' ability to generate Design Structure Matrices (DSMs) from structured technical documentation. It introduces reproducible metrics and a composite quality score, showing that while LLMs can produce plausible DSMs, they remain sensitive to ambiguity and prompt formulation.

This paper presents NightVision, an attack that uses restrictive black-box API access to estimate hidden dimension, depth, and parameter count of large language models. It exploits a novel common-set prompting technique and spectral analysis, achieving high accuracy on open-source models.

CBD introduces an API-only black-box unlearning framework for LLMs that uses two auxiliary models to create controlled behavioral divergence between retained and target data, achieving a better unlearning-utility trade-off compared to existing methods.

Introduces Proxy-KD, a novel method for distilling knowledge from black-box large language models (like GPT-4) into smaller models using a proxy model, surpassing both traditional black-box and white-box KD techniques.

This paper presents a systematic review and benchmark of 24 black-box uncertainty estimation methods for large language models across 4 models and 4 dataset settings, finding that no single method dominates but hybrid methods that combine multiple uncertainty signals perform well.

This paper proposes ORCA, a method for black-box online adaptation of time series foundation models by learning the context of predictive errors. It demonstrates effectiveness across five TSFMs and eight datasets, addressing the challenge of adapting closed-source API-based models.

This paper introduces Vector Linking, a method for recovering correspondences between embeddings from different black-box encoders by leveraging local geometric consistency, proposing an iterative reference-based geometric embedding hashing approach using a small seed set of paired anchors.

This paper introduces bounded behavioral indistinguishability, a formal framework for evaluating black-box LLM distillation beyond semantic similarity. Experiments on Qwen and Llama models show that distillation reduces but does not eliminate adversarial distinguishability, highlighting the need for category-aware evaluation.

The article highlights three common failure modes in production AI memory systems: outdated preferences persisting, sarcasm stored as literal, and summaries outliving their source facts. It argues that the AI memory industry lacks provenance, confidence scores, and versioning, creating a black-box problem that hinders debugging.

This paper proposed Distribution-Aligned Adversarial Distillation (DisAAD), a method that uses a lightweight proxy model to estimate uncertainty in black-box LLMs with only 1% of the original model size, achieving reliable quantification without requiring internal parameters or multiple sampling.

Researchers propose a surrogate modeling framework to quantify and interpret latent medical knowledge encoded in black-box LLMs, revealing both valid associations and persistent racial biases.

Researchers introduce SHADE, a hybrid estimator that combines Good-Turing coverage with graph-spectral cues to quantify semantic uncertainty and detect LLM hallucinations when only a few black-box samples are available.

MedFocusLeak introduces the first transferable black-box adversarial attack on medical vision-language models, using imperceptible background perturbations to mislead clinical diagnoses across six imaging modalities.