bert

This paper studies confidence estimation and selective prediction for financial named entity recognition under domain shift, evaluating BERT and LoRA-tuned Qwen models to enhance reliability across different input distributions like SEC filings and social media.

A comparative study of BART, BERT, and RoBERTa for text summarization, examining their architectures and suitability for extractive and abstractive summarization tasks.

This paper evaluates the reliability of BERT-based QA models (RoBERTa, ALBERT, DistilBERT) under Monte Carlo Dropout and input paraphrasing, finding RoBERTa more consistent and validating MCD as a reliability metric.

Presents STCAD, a scalable framework using BERT-based encoding and CURE clustering to perform trajectory clustering and anomaly detection on terabyte-scale AIS maritime data, demonstrating stable clusters and clear separation of anomalous vessel behavior.

This paper presents a machine learning pipeline for detecting self-introductions in legislative committee testimony, using features like bag-of-words and BERT probabilities. XGBoost achieves the best F1 score of 0.9747, improving further with BERT-augmented features.

This paper proposes CNM, a lightweight augmentation that injects discrete compositional structure of Chinese characters into BERT via Ideographic Description Sequences, improving performance on rare and out-of-vocabulary characters while preserving general NLU accuracy.

This paper proposes Joint Affine Spectral Shaping (JRI), which extends weight-only spectral optimizers like Muon to jointly update weight and bias in affine layers, showing small but consistent accuracy improvements on a BERT-mini IMDb classification task.

A tweet argues that masked language modeling was unnecessary and that autoregressive models would have sufficed, with a nod to BERT.

This paper proposes an automated multilabel classification system for Mpox research articles using BERT, achieving 97% accuracy, and employs SHAP for explainability. The system aims to help researchers and healthcare workers quickly find relevant information.

This paper compares fine-tuned MahaBERT-based models with large language models (Gemini, LLaMA-3.3-70B, Gemma) for Marathi named entity recognition, finding that the specialized BERT models significantly outperform the LLMs, achieving F1-scores of 0.88–0.91 versus 0.57–0.69.

MyAwesomeModel, a BERT-based feature extractor, converts sentences into dense vectors for semantic search, clustering, and NLP pipelines.

This paper proposes a cross-lingual data augmentation strategy that uses machine translation to transfer expert-annotated difficulty labels from high-resource languages to low-resource languages. Experiments with BERT-based regression models show that augmenting scarce native data with translated corpora significantly improves the accuracy of text difficulty assessment.

This paper presents methods for detecting AI-generated text using Bayesian data mixing and empirical X-risk minimization, achieving high performance on OOD detection with ModernBERT-large and MCGrad classifiers.

The paper presents a scalable framework for multi-domain dialogue state tracking using BERT, achieving zero-shot generalization and improving performance on the SGD dataset.

This paper investigates the feasibility of using translation-based fine-tuning as a resource-efficient alternative to native-language BERT models for low-resource languages, finding it comparable or superior in 53.3% of cases across six NLP tasks.

This paper proposes a Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoder to extract universal features across independently trained BERT models, improving cross-seed feature alignment beyond post-hoc methods.

CrossBERT decouples representation learning from token reconstruction, enabling higher masking ratios and better sample efficiency, outperforming BERT on MTEB and GLUE benchmarks.

A tweet from @goyalayus suggests that four unspecified books are sufficient to learn any niche, followed by a quoted recommendation from Praveen Kumar Verma to read foundational LLM papers in a specific order, including Attention Is All You Need, BERT, GPT, GPT-2, Scaling Laws, and GPT-3.

BamiBERT is a new BERT-based pre-trained language model for Vietnamese that addresses limitations of PhoBERT, supporting longer context and operating without word segmentation, achieving state-of-the-art results on multiple Vietnamese benchmarks.

Promotes an educational resource explaining Transformer architecture, covering token embeddings, self-attention, residual connections, and connections to GPT and BERT.