Been stuck on a unique NLP problem [D]
Summary
Developer seeks advice on handling English-Hindi code-mixed text classification without heavy LLMs, as sentence transformers fail on Romanized Hindi.
Similar Articles
From Lexicon to AI: A Structured-Data Pipeline for Specialized Conversational Systems in Low-Resource Languages
Presents a systematic methodology for converting Hindi WordNet into 1.25 million instruction-response pairs to fine-tune a 12B-parameter language model using LoRA, demonstrating improved pedagogical effectiveness for specialized conversational systems in low-resource languages.
Structure-Preserving Document Translation via Multi-Stage LLM Pipeline: A Case Study in Marathi
This paper presents a multi-stage LLM pipeline for structure-preserving Marathi-to-English translation of government documents, integrating layout-aware OCR and HTML reconstruction to maintain formatting and domain terminology.
Evaluation of forced alignment of code-mixed speech: the case of Hindi-English
This paper evaluates forced alignment for Hindi-English code-mixed speech using the Montreal Forced Aligner, demonstrating that bootstrapping strategies and code-mixed training data achieve a tenfold improvement in alignment accuracy over monolingual alternatives.
Multilingual-Multimodal-NLP/LoopCoder-V2 · Hugging Face
LoopCoder-V2 is a 7B instruction-tuned code model built on the Parallel Loop Transformer (PLT), demonstrating non-monotonic test-time scaling with two loops providing the best gain-cost trade-off and significant improvements over baselines on code generation and reasoning benchmarks.
Multi-Stage Training for Abusive Comment Detection in Indic Languages
This paper proposes a multi-stage training pipeline using language-based preprocessing and an ensemble of models to detect abusive comments in Indic languages, aiming to minimize false positives while preserving freedom of expression.