bert

This paper proposes an automated multilabel classification system for Mpox research articles using BERT, achieving 97% accuracy, and employs SHAP for explainability. The system aims to help researchers and healthcare workers quickly find relevant information.

This paper compares fine-tuned MahaBERT-based models with large language models (Gemini, LLaMA-3.3-70B, Gemma) for Marathi named entity recognition, finding that the specialized BERT models significantly outperform the LLMs, achieving F1-scores of 0.88–0.91 versus 0.57–0.69.

MyAwesomeModel, a BERT-based feature extractor, converts sentences into dense vectors for semantic search, clustering, and NLP pipelines.

This paper proposes a cross-lingual data augmentation strategy that uses machine translation to transfer expert-annotated difficulty labels from high-resource languages to low-resource languages. Experiments with BERT-based regression models show that augmenting scarce native data with translated corpora significantly improves the accuracy of text difficulty assessment.

This paper presents methods for detecting AI-generated text using Bayesian data mixing and empirical X-risk minimization, achieving high performance on OOD detection with ModernBERT-large and MCGrad classifiers.

The paper presents a scalable framework for multi-domain dialogue state tracking using BERT, achieving zero-shot generalization and improving performance on the SGD dataset.

This paper investigates the feasibility of using translation-based fine-tuning as a resource-efficient alternative to native-language BERT models for low-resource languages, finding it comparable or superior in 53.3% of cases across six NLP tasks.

This paper proposes a Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoder to extract universal features across independently trained BERT models, improving cross-seed feature alignment beyond post-hoc methods.

CrossBERT decouples representation learning from token reconstruction, enabling higher masking ratios and better sample efficiency, outperforming BERT on MTEB and GLUE benchmarks.

A tweet from @goyalayus suggests that four unspecified books are sufficient to learn any niche, followed by a quoted recommendation from Praveen Kumar Verma to read foundational LLM papers in a specific order, including Attention Is All You Need, BERT, GPT, GPT-2, Scaling Laws, and GPT-3.

BamiBERT is a new BERT-based pre-trained language model for Vietnamese that addresses limitations of PhoBERT, supporting longer context and operating without word segmentation, achieving state-of-the-art results on multiple Vietnamese benchmarks.

Promotes an educational resource explaining Transformer architecture, covering token embeddings, self-attention, residual connections, and connections to GPT and BERT.

Introduces a comprehensive hate speech dataset for Turkish and Arabic, and develops state-of-the-art BERT-based models for hate speech analysis including classification, intensity prediction, target identification, and span detection.

This paper finds that 42.6% of annotator disagreement in HateXplain concentrates at the hate/offensive boundary, demonstrating that majority vote silences minority values and leads to models being wrong but highly confident on contested inputs.

This paper explores domain adaptation of ModernBERT models in the legal domain by further pre-training on US court opinions, achieving significant improvements over the vanilla model and releasing the checkpoints publicly.

SupraLabs released SupraSafety-18M, a tiny 18M-parameter BERT-style content moderation model trained on NVIDIA's Nemotron-3.5 dataset. It achieves 81.2% accuracy and runs efficiently on edge devices.

This paper compares fine-tuned BERT (gbert-large) with few-shot LLM prompting (Llama 4 Maverick) for detecting threat and solution framing in German climate news sentences. BERT achieves higher F1 scores (0.83 vs 0.78), and an ablation study shows that providing preceding sentence context improves performance.

This paper investigates the distribution and evolution of aspect-level sentiments in multi-round peer reviews from Nature Communications, using a deep learning approach (LCF-BERT-CDM) to achieve 82.65% Macro-F1, and finds that positive sentiment increases while negative sentiment decreases with more review rounds.

This paper proposes a novel pipeline for multilingual coreference resolution that uses cycle-consistent machine translation from English to low-resource languages to generate training data, validated by back-translation and BERT similarity. Experiments on four low-resource languages show significant performance gains, enabling accurate coreference resolution where no prior corpora existed.

The paper proposes a hybrid pre-training objective combining JEPA latent-space prediction with MLM reconstruction for language models, showing improved embedding uniformity and semantic-lexical balance.