LiquidAI/LFM2.5-Encoder-350M · Hugging Face

Source: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M

LFM2.5-Encoderis a family of multilingual bidirectional encoders built on the LFM2 architecture, available in two sizes:

• LFM2.5-Encoder-230M— a lightweight encoder for tight latency and memory budgets, punching above its size class.
• LFM2.5-Encoder-350M*(this model)*— a larger sibling for maximum downstream quality.

Both are masked language models with full bidirectional attention, designed to be fine-tuned into task-specific models (classification, token classification, retrieval, reranking, and semantic similarity) across 15 languages, and to run efficiently on-device.

Find more details about our encoders in ourblog post.

Key highlights:

• **Top quality for its size.**Ahead of every model its size or smaller, and ~5 points above our own retrieval siblings.
• **General-purpose.**8k context, strong across NLI, paraphrase, sentiment, and multilingual tasks.
• **Fast and on-device.**Matches or beats ModernBERT throughput, with a long-context edge on CPU.

💻Demos: We built the demos below from fine-tuned LFM2.5-Encoders. Each one runs in a CPU-only Hugging Face space: - Zero-shot prompt routing— define your own routing lanes as free text. The model scores the whole prompt against every lane in one pass. - Zero-shot policy linting— check text against your company’s rules, written as free text. It scores every token against every rule in one pass. - Spell checking— correct misspellings token by token. - PII detection— spot and remove 40 kinds of personal information across 16 languages. - Masked-diffusion text generation— bonus: run the encoder as a chatbot that generates text by iteratively unmasking instead of left to right.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#%F0%9F%93%84-model-details📄 Model details

PropertyLFM2.5-Encoder-230MLFM2.5-Encoder-350MTypeBidirectional encoder (masked language model)Bidirectional encoder (masked language model)BackboneLFM2LFM2Total parameters~229.7M~354.5MHidden size10241024Vocabulary size65,53665,536Context length8,192 tokens8,192 tokensLicenseLFM Open License v1.0LFM Open License v1.0 **Supported languages:**English, German, Spanish, French, Italian, Dutch, Polish, Portuguese, Arabic, Hindi, Japanese, Russian, Turkish, Vietnamese, Chinese (15).

**Architecture.**LFM2.5-Encoder is built on the LFM2 hybrid backbone, which interleaves gated short-convolution blocks with grouped-query attention. For encoder use, the causal mask is replaced with full bidirectional (non-causal) attention and the model is trained with a masked language modeling head. The encoder body is exposed asLfm2BidirectionalModel; masked-LM loading usesLfm2BidirectionalForMaskedLM. Both are wired throughauto\_mapand requiretrust\_remote\_code=True.

Lfm2BidirectionalForMaskedLM(
  (lfm2): Lfm2BidirectionalModel
  (lm_head): Linear(in_features=1024, out_features=65536, bias=False)
)

**Training.**LFM2.5-Encoder-350M is adapted from the LFM2 base and trained with a masked language modeling objective on a large multilingual corpus. Pre-training uses a two-stage schedule that extends the context window to up to 8,192 tokens.

We recommend fine-tuning LFM2.5-Encoder-350M for a range of downstream tasks, such as:

• Text classification: sentiment, topic, intent/routing, moderation, and business-text linting.
• Token classification: named-entity recognition, span extraction, and sequence labeling.
• Retrieval and reranking: a backbone for dense embedding or late-interaction (ColBERT-style) retrievers.
• Semantic similarity: STS, paraphrase, and duplicate detection.
• Natural language inference and extractive QA: sentence-pair reasoning and answer-span extraction.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#%F0%9F%8F%83-how-to-run🏃 How to run

Install the latest version oftransformers:

pip install -U transformers

Run masked-token prediction:

from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-Encoder-350M", trust_remote_code=True)
mlm = AutoModelForMaskedLM.from_pretrained("LiquidAI/LFM2.5-Encoder-350M", trust_remote_code=True)

text = f"The capital of France is {tok.mask_token}."
enc = tok(text, return_tensors="pt")
with torch.no_grad():
    logits = mlm(**enc).logits
pos = (enc["input_ids"][0] == tok.mask_token_id).nonzero()[0].item()
print([tok.decode([t]).strip() for t in logits[0, pos].topk(5).indices.tolist()])
# -> ['Paris', 'Strasbourg', 'Paris', 'Lyon', 'Versailles']

For downstream tasks, load the encoder body and attach your own head (classification, token classification, regression, retrieval):

from transformers import AutoModel
body = AutoModel.from_pretrained("LiquidAI/LFM2.5-Encoder-350M", trust_remote_code=True)

If your GPU supports it, we recommend using LFM2.5-Encoder-350M with Flash Attention 2 to reach the highest efficiency. To do so, install Flash Attention as follows, then use the model as normal:

pip install flash-attn

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#%F0%9F%93%8A-performance📊 Performance

For each benchmark task, we run a full supervised fine-tune and report that fine-tuned model’s score. The results below span 14 models across 17 tasks from GLUE, SuperGLUE, and multilingual classification tasks. The full evaluation harness is open-sourced in theeurobert\-reprorepository.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#17-task-results-avg5-fresh-seeds-%C2%B1-std17-task results (avg@5 fresh seeds ± std)

RankModelParams17-task mean± std1XLM-R XL (3.5B)3.5B83.06±1.162ModernBERT-large (395M)395M81.68±2.493XLM-R large (560M)560M81.34±1.664****LFM2.5-Encoder-350M (ours)350M81.02±1.005mDeBERTa-v3 (280M)280M80.37±1.066LFM2.5-Encoder-230M (ours)230M79.29±1.027ModernBERT-base (149M)149M78.19±1.398XLM-R base (280M)280M77.46±1.639EuroBERT-210M210M76.87±2.0010mGTE-MLM (305M)305M76.53±1.8511LFM2.5-ColBERT-350M350M76.18±1.2512EuroBERT-610M610M75.87±2.0313LFM2.5-Embedding-350M350M75.68±0.8314EuroBERT-2.1B2.1B72.19±5.59 Click to expand per-task results — all 17 tasks (avg@5 fresh seeds ± std)### Per-task results — all 17 tasks (avg@5 fresh seeds ± std)ModelXNLIPAWS-XAmazonMASSIVESeaHorseCoLA*SST-2*MRPC*STS-B*QQP*MNLI*QNLI*RTE*BoolQ*CB*WiC*WSC*ALLXLM-R XL (3.5B)87.12±0.4593.30±0.4162.29±0.0588.21±0.3259.51±3.5284.58±1.1895.69±0.3087.65±1.6990.20±0.9391.72±0.0790.09±0.1194.47±0.2882.38±3.5183.70±0.2489.88±3.7266.55±1.3764.62±1.5883.06ModernBERT-large (395M)81.76±0.3892.46±0.1860.42±0.1585.65±0.9440.20±17.1883.37±0.2096.10±0.5388.14±1.7992.16±0.2491.81±0.1390.65±0.1894.36±0.1081.59±4.9281.68±2.3488.21±3.2470.16±2.5769.81±7.1881.68XLM-R large (560M)84.69±0.5993.23±0.7861.58±0.1388.50±0.1756.12±2.3183.34±1.7593.83±1.0488.77±2.1591.35±0.2390.48±0.2388.29±0.0793.08±0.2380.79±3.1480.54±0.7978.21±8.6966.24±5.4163.65±0.4381.34****LFM2.5-Encoder-350M (ours)79.82±0.2991.53±0.7360.57±0.0985.70±0.1354.96±0.4184.43±0.8195.11±0.2687.21±1.8691.59±0.0592.08±0.1089.03±0.1793.97±0.2375.23±3.8481.52±0.6983.21±2.0469.66±2.2361.73±3.1581.02mDeBERTa-v3 (280M)83.01±0.4792.59±0.3960.62±0.3187.64±0.5154.97±2.0483.91±1.0392.41±0.9685.39±2.5989.87±0.2290.23±0.1586.30±0.1891.99±0.3569.75±2.0378.29±1.3988.21±2.4067.71±3.0563.46±0.0080.37LFM2.5-Encoder-230M (ours)77.63±0.3190.86±0.2459.97±0.2185.52±0.6954.61±0.6281.42±1.5694.08±0.3780.20±2.6690.99±0.1291.71±0.0787.98±0.2292.96±0.3767.29±1.9776.54±1.0183.21±4.4870.31±1.3462.69±1.0579.29ModernBERT-base (149M)76.64±0.3192.17±0.1558.98±0.1185.32±0.2145.19±1.7283.07±1.9394.79±0.5284.46±2.7090.75±0.1491.23±0.1188.68±0.1793.04±0.3658.70±1.9474.78±4.1081.07±6.7566.90±2.3963.46±0.0078.19XLM-R base (280M)78.20±0.8091.36±0.4160.01±0.1287.47±0.4951.08±3.7581.17±1.0791.97±0.1886.47±0.7688.27±0.3689.21±0.0483.07±0.2190.17±0.3262.60±7.0371.43±1.6479.64±7.5361.25±3.0063.46±0.0077.46EuroBERT-210M80.83±0.3591.94±0.3159.94±0.1686.36±0.6745.16±16.6072.75±1.3090.64±0.9280.74±2.9989.29±0.2390.75±0.0985.63±0.2891.49±0.2754.95±2.5671.68±2.3586.79±2.4064.64±2.0163.27±0.4376.87mGTE-MLM (305M)80.32±0.2091.73±0.2660.26±0.1087.79±0.2051.58±1.3175.44±4.6691.19±1.0086.32±1.4887.77±0.6489.82±0.0984.14±0.1590.94±0.4058.34±3.0969.32±3.7273.21±6.8059.34±7.4163.46±0.0076.53LFM2.5-ColBERT-350M78.77±0.4789.74±0.4459.92±0.1386.65±0.1747.95±1.1371.06±1.0690.94±0.7873.43±7.2289.38±0.2891.11±0.1484.70±0.2390.66±0.1859.13±2.3074.25±1.6181.79±2.9362.04±2.1263.46±0.0076.18EuroBERT-610M84.61±0.3491.84±0.9460.64±0.0886.03±0.9912.91±8.0570.60±2.1292.52±0.6685.20±1.3489.82±0.2291.13±0.0987.95±0.1992.57±0.3659.28±7.9376.86±1.5585.71±4.3758.71±5.3063.46±0.0075.87LFM2.5-Embedding-350M78.59±0.1189.13±0.6360.47±0.1387.03±0.2150.19±0.9072.54±0.6891.70±0.6977.45±1.3189.38±0.0991.14±0.1384.70±0.1290.62±0.5055.38±1.7470.17±1.9971.43±3.5763.10±1.2863.46±0.0075.68EuroBERT-2.1B70.52±14.2292.34±0.1960.45±0.7085.40±1.366.84±6.8168.99±0.6792.50±1.0382.94±3.1066.44±32.3691.03±0.2981.56±16.6293.56±0.2553.29±0.7977.23±6.7782.86±5.1457.90±4.7563.46±0.0072.19 \*= dev split (GLUE/SuperGLUE test labels hidden). The 5 multilingual columns are labeled test. SeaHorse & STS-B are Spearman×100. All other tasks are accuracy.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#inference-speedInference speed

The LFM2 backbone was built for fast inference, and the encoders inherit it. While ModernBERT-base is faster at short sequences in Apple GPU inputs, LFM2.5-Encoders overtake it as inputs grow. At long input sequences of 8k on CPU, the encoders run 3.3× faster than ModernBERT-base.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#%F0%9F%94%A7-fine-tuning🔧 Fine-tuning

LFM2.5-Encoder-350M follows standard BERT-style fine-tuning. Attach a task head to the encoder body and train end-to-end. Suggested starting points (tune per task):

HyperparameterSuggested rangeLearning rate1e-5 – 5e-5Warmup ratio0.1Weight decay0.1Epochs3 – 20 (early stopping, patience 3)Precisionbf16 autocast (fp32 master weights)

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#%F0%9F%93%AC-contact📬 Contact

• Got questions or want to connect?Join our Discord community
• If you are interested in custom solutions with edge deployment, please contactour sales team.

https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M#citationCitation

@article{liquidAI2026Encoders,
  author = {Liquid AI},
  title = {LFM2.5-Encoders: Fast at Long Context, Even on CPU},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/lfm2-5-encoders},
}