Tag
HakemBench introduces an open Turkish benchmark (CC BY 4.0) for 'typed decisions'—calibrated probability outputs over fixed options—covering seven tracks such as fact-check triage, legal routing, and moderation. It evaluates decision quality (macro F1), calibration (Brier score), and selective automation (risk-coverage), while transparently disclosing weaknesses in its AI-generated gold labels and non-blind evaluation of the lab's own model.
The paper details ufakzeka-1, a 151M-parameter Turkish language model built from scratch with a total cost of about $286, describing the training pipeline, evaluation methods, and key findings on small-model training limitations.
This paper compares chunking and embedding strategies for RAG systems in Turkish, evaluating their impact on document question answering through a systematic study with multiple configurations.
This paper presents embeddingmagibu-200m, a Turkish-focused sentence embedding model built via cross-lingual tokenizer surgery and offline distillation, achieving strong performance on Turkish benchmarks with a cost-quality trade-off.