Tag
This paper introduces ChristBERT, a family of domain-specific RoBERTa-based language models for German clinical NLP, and evaluates three domain adaptation strategies (continued pre-training, pre-training from scratch, and vocabulary adaptation) on medical named entity recognition and text classification tasks, achieving state-of-the-art results.
This paper presents Lepton, a fine-tuned BERT classifier that predicts whether a title in Classical Chinese wenji table-of-contents is a personal letter or a preface, leveraging 5,438 hand-labeled titles from late-Ming and early-Qing literati.
This paper uses a BERT-based large language model for sentiment analysis of Decentraland's Discord community to enhance MANA token price prediction, demonstrating that a multi-modal LSTM incorporating sentiment, trading volume, and market capitalization outperforms a price-only baseline.
This research paper investigates how shortcut solutions learned by Transformer models, specifically BERT, impair their ability to perform continual compositional reasoning. It contrasts BERT with ALBERT, finding that ALBERT's recurrent nature offers better inductive bias for continual learning tasks.
Released en_legal_ner_ind_trf v0.1, an InLegalBERT model fine-tuned on 33,000 Indian Supreme Court judgments, achieving a 97.76% F1 score on case citations and significantly outperforming previous baselines.
A foundational study on applying stylometric authorship attribution to threat intelligence, using Japanese Rakuten reviews to compare TF-IDF+LR, BERT embedding, BERT fine-tuning, and metric learning methods. BERT-FT performed best overall, but TF-IDF+LR proved more stable and efficient when scaling to hundreds of authors.
This article profiles researcher Brian Hie, highlighting how his unique background in literature and computer science informed the development of ESM, a BERT-like model for protein sequences.