Tag
This paper evaluates traditional coherence metrics and LLM-based semantic similarity for dynamic topic models, finding that LLM-based metrics better align with human judgments by accounting for lexical changes. It advocates for a combined evaluation approach using both traditional and LLM-based measures.
This arXiv preprint studies the semantic dispersion of sixteen language models forming ensembles, showing that ensemble diversity is small on average and that model identity only partially explains which model is most divergent. The authors propose a per-model dissent contribution metric and find that dispersion is organized by clinical content rather than interpretive openness.
A large-scale mapping of 11 million academic papers using semantic similarity and time-based slices, enabling analysis of research trends and connections.
This paper introduces and empirically evaluates methods for measuring semantic similarity between knowledge graphs using KG embeddings, proposing EmbPairSim and AvgEmbSim scoring functions that outperform baselines like Sentence-BERT on WikiText-2 and CC-News datasets.
This paper investigates how lexical overlap, rather than semantic content, influences LLM representations across layers and architectures, and demonstrates that this lexical effect persists even in models trained for semantic similarity, leading to degraded performance on downstream tasks.
This paper introduces bounded behavioral indistinguishability, a formal framework for evaluating black-box LLM distillation beyond semantic similarity. Experiments on Qwen and Llama models show that distillation reduces but does not eliminate adversarial distinguishability, highlighting the need for category-aware evaluation.
OmniOPD introduces a logit-free on-policy distillation method that uses chunk-level semantic similarity and speculative verification to train student models with black-box teachers, achieving up to +28.64% improvement on math benchmarks over standard OPD.
Researchers from PNNL and Washington University introduce a systematic framework to test how five LLMs detect subtle semantic changes in documents, revealing positional bias, context coherence effects, and model-specific scoring fingerprints.