Tag
The paper introduces BanglaRhet, a 30,289-segment annotated corpus of Bangla political speech, and benchmarks classical and transformer models (BanglaBERT, SahajBERT, XLM-RoBERTa) for rhetorical and persuasion technique detection, with BanglaBERT achieving the best macro-F1 of 66.46%.
This paper provides a mechanistic analysis of Hierarchical Reasoning Models (HRM) to understand their internal reasoning processes in latent space, using techniques like causal interventions and sparse autoencoders on tasks such as Sudoku and ARC-AGI-2.
This paper explores how transformer models handle functional words like pronouns and adverbs, analyzing their embeddings and generalization capabilities. It finds that training on mixed lexicalized and functional sentences helps models learn shared syntactic-semantic structures.
This paper investigates subliminal learning in language models by measuring causal depth and multi-token confounds to understand how traits are transferred through apparently unrelated outputs.
This paper analyzes the backward-pass geometry of Z-loss in AI model training, providing a framework for understanding and optimizing gradients in dense output heads and sparse mixture-of-experts routers, with evaluations on GPT-2 and Pythia models.
This paper introduces an open modular transformer-based framework for detecting and structuring social tipping point evidence in climate documents, utilizing models like DistilBERT, RoBERTa, Mistral 7B, and LLaMA 3.2 3B, with evaluation on expert-labeled benchmarks.
This paper investigates whether transformer models internalize the same linguistic features as traditional models for multilingual readability assessment, using SHAP and TCAV across five languages.
The paper proposes EPA-CarbonNet, a six-layer transformer architecture for carbon credit price prediction that integrates market data and policy text, but tests on S&P carbon index data show mixed results with a random walk outperforming on some metrics while directional accuracy is promising.
Sergey Levine recommends the ACT or ALOHA paper for AI robotics beginners, highlighting how low-cost robot setups and simple transformer models can perform complex dexterous tasks.
This paper proposes a Cross-Platform Fairness Evaluation framework to audit transformer models for mental health NLP, revealing significant performance and calibration failures when models are applied across different social media platforms.
This paper introduces an AI-enhanced method for RF interference suppression using transformer models with a Finite Scalar Quantization tokenizer to boost performance and minimize latency, validated via audio metrics in digital signal scenarios.
The article explores optimizing AI language model performance on a home server built from e-waste GPUs, with explanations of transformer models and multi-GPU techniques.
ATHENA is a knowledge-guided agentic neural architecture search framework that automates Transformer-based electronic health record modeling by reusing architecture knowledge across hospitals to reduce manual tuning.
This paper investigates AI learning and conceptual transfer in the Game of Hidden Rules, focusing on reinforcement learning with Transformer-based A2C framework, rule difficulty analysis, transfer learning, and generalization.
This critical narrative review examines how large language models and transformer-based systems are reshaping hadith computational science, highlighting progress in data resources and segmentation tasks while identifying gaps in narrator verification and reproducibility, and proposing a research agenda for methodological strengthening.
A comparative study of BART, BERT, and RoBERTa for text summarization, examining their architectures and suitability for extractive and abstractive summarization tasks.
This paper introduces Strategic 16K, a leakage-controlled dataset for document sensitivity classification, and benchmarks classical and transformer-based models, with BERT achieving top performance while addressing label leakage issues.
This paper challenges the conflation of dynamic parameterization with dynamic inference, introducing Frozen-Controller Auditing to show that input-dependent coefficients do not imply computational savings. Experiments on Transformers reveal static layerwise profiles retain near-full performance despite no conditional execution.
This paper proposes modifications to encoder-decoder transformers for unconstrained molecular structure elucidation from IR spectra, using a Mixture-of-Experts decoder and contrastive alignment loss, achieving over 10 percentage points improvement in Top-K accuracy.
This paper investigates how instruction-tuned Transformer models (LLaMA and Mistral) encode the discourse relations of causation and antithesis using interpretability techniques on next-token prediction.