convaiinnovations/laya-multilingual · Hugging Face

Source: https://huggingface.co/convaiinnovations/laya-multilingual

https://huggingface.co/convaiinnovations/laya-multilingual#laya-multilingualLaya Multilingual

Non-autoregressiveSystem 1 decision modelcovering 100+ languages. Give it astate(text, email, ticket, or JSON) andtyped questions; it returns typed answers with probabilities in a single forward pass. No text generation, so nothing to parse and nothing to hallucinate.

Part of theLaya family—use this checkpoint for anything that is not English.

https://huggingface.co/convaiinnovations/laya-multilingual#quickstartQuickstart

pip install laya

import laya

agent = laya.load("convaiinnovations/laya-multilingual")
result = agent.predict(
    {"body": "मुझसे इनवॉइस 4411 के लिए दो बार शुल्क लिया गया। कृपया आज ही धनवापसी करें।"},
    {"department": {"type": "choice", "instructions": "Which team should handle `body`?",
                    "criteria": {"billing": "invoices, payments, refunds",
                                 "technical": "bugs and outages", "sales": "pricing"}},
     "refund_requested": {"type": "noul", "instructions": "Does the sender ask for money back?"}},
)
print(result["answers"]["department"]["choice"])      # billing

https://huggingface.co/convaiinnovations/laya-multilingual#let-the-router-chooseLet the Router choose

from laya import Router

router = Router()
router.predict({"body": "I was charged twice"}, questions)          # -> laya
router.predict({"body": "二重に請求されました"}, questions)            # -> laya-multilingual

Since laya 0.3.7 the defaultRouter\(\)keeps bothenglishand this checkpoint resident, so a mixed workload no longer swaps checkpoints on every language change. For a server, load them up front so even the first request of each language is just a forward pass:

router = Router()
router.preload(["english", "multilingual"])      # both resident; no swap at request time

router\.attach\("multilingual", agent\)registers anAgentyou already built, so a process that loaded this checkpoint directly can hand it to the router instead of loading it twice.

In 0.3.7 more text reaches this checkpoint: plain-ASCII Spanish, Italian, Portuguese and French (accents stripped by mail clients and ticket systems), Brazilian Portuguese support text, and any script the router has no range for. If you already run a language-identification model, pass its answer withrouter\.predict\(state, questions, lang\_guess=code\_or\_callable\).

Routing is decided from the script of the input,beforethe forward pass — because the model’s confidence gives no warning when a checkpoint cannot read its input (see below).

Iflaya\.load\(\)hangs:transformersprobes for TensorFlow at import, and when TF is installed its abseil runtime can deadlock model construction. Run withUSE\_TF=0.

https://huggingface.co/convaiinnovations/laya-multilingual#why-this-checkpoint-existsWhy this checkpoint exists

Measured acrossall 51 MASSIVE languages, intent classification with 20 options (random = 0.050), both checkpoints answering byte-identical questions:

laya(English)laya\-multilingualmacro accuracy0.2270.366macro ECE0.7330.387languages clearing 3x random23 / 5145 / 51 The English checkpoint does not degrade gracefully outside English — it collapses, and stays confident while doing so. Khmer:0.000 accuracy at 0.952 confidence. Hebrew 0.060, Armenian 0.050 (exactly random), Bengali 0.080 — all reported with 0.89–0.96 confidence. Its mean confidence never drops below 0.885 at any accuracy level, so confidence gating cannot catch it.

Per-language, this checkpoint turns near-random into usable: Arabic 0.110 →0.400, Bengali 0.080 →0.290, Azerbaijani 0.100 →0.300, Hindi 0.100 →0.387, Korean 0.110 →0.490, Turkish 0.140 →0.437.

https://huggingface.co/convaiinnovations/laya-multilingual#xnli-15-languagesXNLI (15 languages)

layalaya\-multilingualEnglish0.8600.84314 other languages0.5210.731

https://huggingface.co/convaiinnovations/laya-multilingual#speed–it-is-also-the-faster-checkpointSpeed — it is also the faster checkpoint

questions per calllayalaya\-multilingual139.5 ms32.8 ms10158.6 ms (15.9 ms/q)72.3 ms (7.2 ms/q)50771 ms337 ms (6.8 ms/q) 103–332 questions/sec batched on one T4, despite a 256k vocabulary — the 768-dim / 22-layer encoder is cheaper per token than 1024-dim / 28-layer, and the gap widens with batch size.

https://huggingface.co/convaiinnovations/laya-multilingual#architectureArchitecture

• BackbonemmBERT-base (307M, bidirectional, 22 layers, hidden 768, 256k vocab) + a decision head trained from scratch: 2 transformer layers, an option-marker scorer, and an act/escalate head. 322M total.
• Option markersevery option is scored at its own\[MASK\]token, then softmaxed over that question’s options — so the answer space is defined per request, with no retraining.
• Budget1024 tokens per question, of which 256 go to the question and its options.
• Trained from scratch with RLCD: 15,987 updates, 4 epochs, ~4.97 h.

https://huggingface.co/convaiinnovations/laya-multilingual#limitsLimits

• Ships uncalibrated.temperature = \[1\.0, 1\.0, 1\.0\]with no per-option-count buckets. It is systematically over-confident (mean confidence 0.75–0.83 against much lower accuracy). Refitting one temperature per (question type, option count) on held-out data moves mean ECE0.314 → 0.106. Do this on your own data before trusting the probabilities.
• Weaker on Englishthan the English checkpoint: 0.619 vs 0.684 macro across English suites. Route rather than replace.
• Near chance on typed-decisions zero-shot— 0.342, against a 0.318 random and 0.461 majority-class baseline. Fine-tune for a specific workflow; that is where the capability comes from.
• **Keepchoicequestions under ~20 options.**Options share the fixed 256-token head budget, so a very large label space leaves only a few tokens per label and accuracy falls off sharply.
• Low-resource languages are weak, not fixed: Swahili 0.210, Tamil 0.250, Amharic 0.110.
• Ordinalscorequestions are the weakest primitive (SST-5 0.282), and this checkpoint has ameasured position bias on them: it rarely picks the first-listed level, in any language, including English (0 of 290 in one independent run,#131). For English score questions use the English checkpoint. For other languages, validate score outputs on your own data first.
• **noulcan under-report “true” here.**On a clearly positive input, one measurement putP\(true\)at about 0.5 while the negative case was correctly near 0 (#156). Ifnoulanswers look weak, the same question as a two-optionchoicewith neutral keys (A/B) and yes/no descriptions is a useful check.

https://huggingface.co/convaiinnovations/laya-multilingual#linksLinks

• Hub / familyhttps://huggingface.co/convaiinnovations/laya
• GitHubhttps://github.com/NandhaKishorM/laya· full benchmark data on theresearchbranch
• PyPIhttps://pypi.org/project/laya/
• Demohttps://huggingface.co/spaces/convaiinnovations/laya-demo

Apache 2.0 · Convai Innovations