Balancing Reasoning and Hardware Constraints in RAG Pipelines for Ukrainian Multi-Domain Document Understanding

# Balancing Reasoning and Hardware Constraints in RAG Pipelines for Ukrainian Multi-Domain Document Understanding
Source: [https://arxiv.org/html/2609.22124](https://arxiv.org/html/2609.22124)
Illya HavrylovAffiliation:National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”Affiliation:Educational and Scientific Institute for Applied System Analysis \(IASA\)Affiliation:Department of Artificial IntelligenceEmail:[ilia89279@gmail\.com](mailto:)

###### Abstract

This paper describes the system submitted to the UNLP 2026 Shared Task on Multi\-Domain Document Understanding\. The challenge required extracting precise answers, document IDs, and page numbers from a diverse corpus of Ukrainian PDF documents within a strict 9\-hour offline Kaggle execution limit\. During evaluation on the hidden private test set, optical character recognition \(OCR\) of scanned documents emerged as a severe bottleneck, consuming 5–7 hours of the total time budget due to sequential single\-threaded execution\. This overhead strictly limited the remaining time for Large Language Model \(LLM\) inference to approximately two hours for 500 questions\. To guarantee pipeline completion without timeouts, we developed a resource\-efficient Hybrid Retrieval\-Augmented Generation \(RAG\) pipeline utilizing BM25, BGE\-M3, and Cross\-Encoder reranking\. Rather than deploying parameter\-heavy reasoning models \(e\.g\., DeepSeek R1\) which consistently timed out, we utilized a 4\-bit quantized LapaLLM 12B model viallama\.cppon dual NVIDIA T4 GPUs\. Prioritizing pipeline stability over multi\-step reasoning, our system achieved a Private Score of 0\.8095, placing 10th out of 15 active teams\.111Code is available at:[https://github\.com/catdlia/Notebook\_From\_UNLP2026](https://github.com/catdlia/Notebook_From_UNLP2026)

## 1Introduction

Multi\-domain document understanding presents significant challenges for Ukrainian natural language processing, particularly when parsing digitized legal, medical, and sports documents\. The UNLP 2026 Shared Task evaluates AI systems on retrieving information from diverse documents and generalizing across unseen domains\.

The competition was hosted on Kaggle in a code\-only, offline environment with a strict 9\-hour runtime limit\. Under these conditions, an unexpected hardware bottleneck emerged: an extensive volume of image\-based, scanned PDFs in the hidden private test set required continuous fallback to Tesseract OCR\. Because OCR processing was executed sequentially, it consumed 5 to 7 hours of the execution budget, leaving around 2 hours for inference across approximately 500 questions \(roughly 14 seconds per query for retrieval and generation combined\)\.

This limitation forced a fundamental engineering trade\-off: reasoning depth versus inference latency\. While heavy reasoning models theoretically offer higher accuracy, their slow generation speeds and extensive chain\-of\-thought token footprints caused notebook timeouts\.

In this work, we document a pragmatic, domain\-agnostic RAG pipeline designed to operate within these constraints\. Our main contributions are:

1. 1\.We deploy a robust dual\-stream hybrid retrieval pipeline \(sparse BM25 and dense BGE\-M3\) combined with Reciprocal Rank Fusion \(RRF\) and Cross\-Encoder reranking\.
2. 2\.We demonstrate that under extreme runtime constraints, utilizing a 4\-bit quantized Ukrainian\-adapted model \(LapaLLM 12B\) on dual NVIDIA T4 GPUs viallama\.cppensures reliable completion compared to large reasoning architectures\.
3. 3\.We show that deterministically decoupling citation extraction \(Document ID and Page Number\) from LLM generation eliminates attention degradation and ensures stable formatting\.

## 2Task and Dataset Description

### 2\.1Dataset Characteristics

The shared task corpus comprises multi\-domain Ukrainian documents across three distinct domains: legal frameworks, medical instructions, and sports regulations\. The private test set includes approximately 240 unseen documents formatted as native digital PDFs and scanned image PDFs, introducing substantial noise and layout variations\. The test set consists of approximately 500 multiple\-choice questions, each providing 6 possible options\.

### 2\.2Evaluation Metric

Systems were evaluated on selecting the correct answer option \(aia\_\{i\}\), citing the correct Document ID \(did\_\{i\}\), and identifying the relevant Page Number \(pip\_\{i\}\)\. The official evaluation metric is defined as:

Metric=0\.5N​∑i=1Nai\+0\.25N​∑i=1Ndi\+0\.25N​∑i=1Npi\\text\{Metric\}=\\frac\{0\.5\}\{N\}\\sum\_\{i=1\}^\{N\}a\_\{i\}\+\\frac\{0\.25\}\{N\}\\sum\_\{i=1\}^\{N\}d\_\{i\}\+\\frac\{0\.25\}\{N\}\\sum\_\{i=1\}^\{N\}p\_\{i\}\(1\)whereai,di∈\{0,1\}a\_\{i\},d\_\{i\}\\in\\\{0,1\\\}are binary indicators of exact matches\. The page proximity scorepi∈p\_\{i\}\\indegrades linearly with distance from the gold page, provided the document ID is correctly identified\.

## 3Related Work

### 3\.1Retrieval\-Augmented Generation \(RAG\)

Standard RAG architectures combine parametric memory from pre\-trained language models with non\-parametric retrieval from external knowledge bases\([Lewis et al\. 2020](https://arxiv.org/html/2609.22124#bib.bib5)\)\. In complex document understanding, single\-retriever systems often struggle with domain\-specific terminology\. Hybrid retrieval pipelines combining sparse lexical matching \(e\.g\., BM25\) and dense neural embeddings via Reciprocal Rank Fusion \(RRF\)\([Cormack et al\. 2009](https://arxiv.org/html/2609.22124#bib.bib1)\)followed by Cross\-Encoder reranking have been shown to significantly boost recall and precision across heterogeneous document formats\.

### 3\.2Ukrainian NLP and Open LLMs

Natural language processing for Cyrillic and low\-resource languages often suffers from poor tokenization efficiency in standard multilingual models\. Recently, open Ukrainian language models such as MamayLM\([MamayLM Contributors 2024](https://arxiv.org/html/2609.22124#bib.bib8)\)and LapaLLM\([LapaLLM Team 2024](https://arxiv.org/html/2609.22124#bib.bib4)\)have introduced expanded vocabularies and targeted pre\-training/fine\-tuning\. This substantially lowers the token\-to\-word ratio for Ukrainian text, reducing memory footprints and accelerating prefill latency during retrieval augmentation\.

## 4System Architecture

The pipeline comprises three core components: document parsing, hybrid retrieval, and generation\.

### 4\.1Document Parsing and Chunking

We utilizedpymupdf4llm\([Artifex Software 2024](https://arxiv.org/html/2609.22124#bib.bib6)\)to extract per\-page text in Markdown format, invoking Tesseract OCR when image scans were encountered\. In our submission, OCR was executed sequentially without multiprocessing, which caused the 5–7 hour runtime bottleneck\. Standard operating system outputs and C/C\+\+ trace dumps were suppressed to prevent I/O buffer crashes inside the Kaggle environment\.

Documents were segmented into fixed lexical windows of 250 words with a 50\-word overlap\. Empirical checks indicated that semantic chunking frequently fragmented tabular layouts across pages, whereas fixed lexical chunking preserved consistent token density for keyword matching\.

### 4\.2Hybrid Retrieval Pipeline

To balance lexical precision and semantic abstraction across diverse domains:

- •Sparse Stream:BM25Okapi\([Brown 2020](https://arxiv.org/html/2609.22124#bib.bib7)\)retrieved the Top\-300 lexical matches from space\-tokenized text\.
- •Dense Stream:BGE\-M3\([Chen et al\. 2024](https://arxiv.org/html/2609.22124#bib.bib10)\)generated 1024\-dimensional embeddings \(mean\-pooling, sequence length 512\) to retrieve the Top\-300 semantic candidates via cosine similarity\.

The candidate sets were fused using Reciprocal Rank Fusion \(k=60k=60\) and truncated to the Top\-100 items\. These 100 chunks were rescored using a Cross\-Encoder \(bge\-reranker\-v2\-m3\)\. The top 7 reranked chunks \(capped at 6,500 characters\) were formatted into the final LLM context window\.

### 4\.3Hardware Selection and LLM Inference

In Kaggle’s offline environment, compute options were constrained to either a single NVIDIA Tesla P100 \(16GB\) or dual NVIDIA T4 GPUs \(2×16​GB=32​GB2\\times 16\\text\{GB\}=32\\text\{GB\}\)\. Although the P100 offers higher FP32 memory bandwidth, its Pascal architecture lacks native support forBfloat16compute and modern tensor formats\. Consequently, we selected the dual NVIDIA T4 environment, which provided 32GB of combined VRAM to comfortably fit larger quantized models and long context windows\.

Inference was powered byllama\.cpp\([Gerganov 2023](https://arxiv.org/html/2609.22124#bib.bib2)\)executing a 4\-bit quantized \(Q4\_K\_M\)LapaLLM\-12Bmodel\([LapaLLM Team 2024](https://arxiv.org/html/2609.22124#bib.bib4)\)\. LapaLLM’s expanded Cyrillic vocabulary yielded superior token compression for Ukrainian text\. Context parameters were configured ton\_ctx=6144andn\_batch=1024with all layers offloaded to GPU \(n\_gpu\_layers=\-1\)\.

To maintain reasoning structure, we used a Chain\-of\-Thought \(CoT\) prompt instructing the model to analyze context options and output the final answer letter on a designated newline\. Metadata citation was completely decoupled: Document ID and Page Number were deterministically assigned from the top\-ranked Cross\-Encoder chunk, preventing attention shift and hallucinated citations\.

## 5Experiments and Results

### 5\.1Leaderboard Results

As shown in Table[1](https://arxiv.org/html/2609.22124#S5.T1), expanding the context window from 5 to 7 reranked chunks steadily improved retrieval grounding\. Our pipeline achieved a Private Score of 0\.8095 \(10th place out of 15 teams\), improving over the Public score \(0\.7831\) and demonstrating strong generalization to unseen domains\.

Table 1:System performance on the UNLP 2026 leaderboard\. Approaches exceeding the 9\-hour limit are marked as Timeout\.
### 5\.2Ablation and Engineering Bottlenecks

#### MamayLM vs\. LapaLLM 12B:

Early iterations with MamayLM exhibited higher hallucination rates and lower tokenization efficiency on dense Ukrainian passages\. LapaLLM 12B showed greater extraction fidelity\.

#### Heavy Reasoning Models:

We evaluated DeepSeek R1\-Distill\-Qwen\-14B\([DeepSeek\-AI et al\. 2025](https://arxiv.org/html/2609.22124#bib.bib3)\)\. Although it displayed strong deductive ability on training data, its extensive<think\>generation tokens, combined with the OCR overhead, consistently triggered execution timeouts\.

#### Self\-Consistency & Multi\-Pass Sampling:

Sampling multiple reasoning paths \(T=0\.6,N=10T=0\.6,N=10\) for majority voting theoretically enhances robustness but was computationally unfeasible within the remaining 2\-hour inference window\. Greedy/low\-temperature decoding with a single pass was necessary to guarantee completion\.

#### Decoupled Metadata Extraction:

When prompted to jointly output the text answer and citations \(Doc\_ID,Page\), the LLM suffered from an “attention shift” effect, where formatting constraints degraded question\-answering accuracy\. Assigning metadata deterministically from the top Cross\-Encoder chunk resolved this issue\.

## 6Conclusion and Future Work

This paper presented an efficient RAG pipeline for the UNLP 2026 Shared Task, achieving 10th place under strict 9\-hour offline limits\. Our results demonstrate that when heavy OCR bottlenecks constrain generation time, combining a dual\-stream hybrid retriever with a vocabulary\-optimized quantized model \(LapaLLM 12B\) and decoupled citation assignment provides an effective engineering solution\.

In future work, we plan to implement multi\-threaded OCR processing to reclaim up to 70% of the time budget\. Furthermore, we aim to investigate custom MLIR compiler architectures and zero\-allocation inference runtimes \(such as the open\-source Tenzo framework\([Havrylov 2026](https://arxiv.org/html/2609.22124#bib.bib9)\)\) to enable low\-latency execution of low\-bit quantized reasoning models on edge and compute\-constrained hardware\.

## Limitations

The system’s design was strictly tailored to Kaggle’s 9\-hour limit on dual T4 GPUs\. The exclusion of heavier reasoning models was driven by runtime constraints rather than fundamental modeling limitations; with parallelized OCR or pre\-extracted text, reasoning models would likely achieve higher task accuracy\.

## Ethical Considerations

The dataset contains legal, medical, and sports documents\. While the pipeline is optimized for accuracy, automated extraction should not replace human judgment in legal or clinical settings\. AI assistants were used for code drafting, debugging, and manuscript proofreading\.

## References

- Cormack et al\. \(2009\)Gordon V\. Cormack, Charles L\. A\. Clarke, and Stefan Büttcher\. 2009\.Reciprocal rank fusion outperforms Condorcet and individual rank learning methods\.In*Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval \(SIGIR ’09\)*, pages 758–759\.
- Gerganov \(2023\)Georgi Gerganov\. 2023\.llama\.cpp: Port of Facebook’s LLaMA model in C/C\+\+\.[https://github\.com/ggerganov/llama\.cpp](https://github.com/ggerganov/llama.cpp)\.
- DeepSeek\-AI et al\. \(2025\)DeepSeek\-AI and others\. 2025\.DeepSeek\-R1: Incentivizing reasoning capability in LLMs via reinforcement learning\.*arXiv preprint arXiv:2501\.12948*\.
- LapaLLM Team \(2024\)LapaLLM Team\. 2024\.LapaLLM: Ukrainian adapted large language models\.Hugging Face Model Hub\.
- Lewis et al\. \(2020\)Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela\. 2020\.Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.In*Advances in Neural Information Processing Systems \(NeurIPS 2020\)*, volume 33, pages 9459–9474\.
- Artifex Software \(2024\)Artifex Software\. 2024\.PyMuPDF4LLM: Extract PDF content formatted for LLM and RAG pipelines\.[https://github\.com/pymupdf/PyMuPDF4LLM](https://github.com/pymupdf/PyMuPDF4LLM)\.
- Brown \(2020\)Dorian Brown\. 2020\.rank\-bm25: A collection of BM25 algorithms in Python\.[https://github\.com/dorianbrown/rank\_bm25](https://github.com/dorianbrown/rank_bm25)\.
- MamayLM Contributors \(2024\)MamayLM Contributors\. 2024\.MamayLM: A foundational open model for Ukrainian\.Hugging Face Repository\.
- Havrylov \(2026\)Illya Havrylov\. 2026\.Tenzo: High\-performance heterogeneous MLIR compiler and zero\-allocation inference runtime\.[https://github\.com/catdlia/tenzo](https://github.com/catdlia/tenzo)\.
- Chen et al\. \(2024\)Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu\. 2024\.BGE M3\-Embedding: Multi\-lingual, multi\-functionality, multi\-granularity text embeddings through versatile data curation\.*arXiv preprint arXiv:2402\.03216*\.

## Appendix ASystem Configurations and Hyperparameters

Table[2](https://arxiv.org/html/2609.22124#A1.T2)lists the exact hyperparameters used in the final submission pipeline\.

Table 2:Hyperparameter configurations extracted from the final submission pipeline\.
## Appendix BPrompt Templates

The model was prompted using the ChatML format with structured Chain\-of\-Thought guidance in Ukrainian\.

#### System Message:

Ukrainian Original:Ти елтний Ш\-детектив\.English Translation:You are an elite AI detective\.

#### User Query Template:

Ukrainian Original:Контекст:
\{ctx\}Запитання:\{Question\}
Варанти:
\{options\}нструкця: Проаналзуй варанти на основ контексту\. Обов’язково напиши на новому рядку: ‘‘Вдповдь: \[Лтера правильного варанту\]’’\.English Translation:Context:\{ctx\}
Question:\{Question\}
Options:\{options\}
Instruction: Analyze the options based on the context\. Be sure to write on a new line: “Answer: \[Letter of the correct option\]”\.

#### Full ChatML Format:

<\|im\_start\|\>system
Ти елтний Ш\-детектив\.
<\|im\_end\|\>
<\|im\_start\|\>user
Контекст:
\{ctx\}Запитання:\{Question\}
Варанти:
\{options\}нструкця: Проаналзуй варанти на основ контексту\. Обов’язково напиши на новому рядку: ‘‘Вдповдь: \[Лтера правильного варанту\]’’\.
<\|im\_end\|\>
<\|im\_start\|\>assistant