Tag
This paper presents COVA-X, an expanded synthetic multi-turn conversation dataset for smishing detection, and shows that Longformer now outperforms XGBoost, confirming that transformer models benefit from larger training corpora.
Researchers from University of Technology Sydney compare fine-tuned transformers (DistilBERT, RoBERTa) against zero-shot LLMs (Llama variants, Claude, Gemini) for classifying misinformation responses on Reddit, finding that fine-tuned RoBERTa achieves 0.62 macro-F1 versus 0.50 for the best zero-shot model. The study shows that task-specific fine-tuning outperforms larger generalist models, particularly for detecting belief propagation, and that safety-alignment artifacts in frontier models can degrade performance.
Financial institutions are shifting from siloed AI models to unified transaction foundation models built on transformer architectures, as demonstrated by NVIDIA's report and Revolut's PRAGMA model, which improves fraud detection, credit scoring, and recommendations while reducing feature engineering effort.
This paper investigates the internal mechanisms of knowledge editing methods ROME and MEMIT, revealing that edits rely on a common functional subspace of weights and suppress rather than overwrite knowledge, explaining why edits fail to propagate to related facts.
This paper investigates temporal concept drift in legal judgment prediction by fine-tuning transformer models on Ukrainian court decisions from three epochs defined by geopolitical disruptions. Findings show severe forward degradation, asymmetry in backward transfer, and that chronological continual learning effectively mitigates forgetting while domain pretraining reduces degradation magnitude.
This paper proposes a sleep-like consolidation mechanism for transformer models that uses fast weights and recurrent passes to improve long-context processing while maintaining inference speed.
This paper presents findings from the Counter Turing Test shared task on AI-generated text detection, with top systems achieving perfect binary classification but significantly lower performance in model attribution, highlighting the difficulty of distinguishing outputs from different large language models.
The paper proposes a transformer-based model to predict political ideology of German political texts on a continuous left-to-right spectrum. The study compares 13 models and finds DeBERTa-large and Gemma2-2B perform best on different tasks.