Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Application

# Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Application
Source: [https://arxiv.org/html/2608.18289](https://arxiv.org/html/2608.18289)
Elias SchubertAffiliation:Berlin University of Applied Sciences, Luxemburger Str\. 10, 13353 Berlin, Germany

[https://www\.bht\-berlin\.de](https://www.bht-berlin.de/)

E\-mail[\{s92538,felix\.biessmann\}@bht\-berlin\.de](mailto:{s92538,felix.biessmann}@bht-berlin.de)Felix BießmannOrcID:[0000\-0002\-3422\-1026](https://orcid.org/0000-0002-3422-1026)Affiliation:Berlin University of Applied Sciences, Luxemburger Str\. 10, 13353 Berlin, Germany[https://www\.bht\-berlin\.de](https://www.bht-berlin.de/)

E\-mail[\{s92538,felix\.biessmann\}@bht\-berlin\.de](mailto:{s92538,felix.biessmann}@bht-berlin.de)Affiliation:Einstein Center Digital Future, Wilhelmstraße 67, 10117 Berlin, Germany
[https://www\.digital\-future\.berlin](https://www.digital-future.berlin/)
###### Abstract

The extraction of structured information from unstructured documents represents a critical component of digital transformations in all sectors\. While proprietary solutions dominate commercial applications, a rapidly growing ecosystem of open\-source Optical Character Recognition \(OCR\) engines, Large Language Models \(LLMs\), and Vision\-Language Models \(VLMs\) offers accessible alternatives\. However, systematic evaluations on realistic, multi\-step extraction pipelines remain scarce\. Responsible usage of such extraction tools require comprehensive evaluations on realistic tasks, especially as these solutions will be key components of applications in the public sector that the EU AI act categorizes as high risk\. To address this gap we present a comprehensive benchmark assessing the end\-to\-end performance of open\-source systems on a complex real\-world document processing task classified as high risk: Student applications for an international study program\. We conduct a comprehensive empirical evaluation with state\-of\-the\-art OCR engines, LLMs and VLMs\. Our results reveal that while VLMs generally outperform OCR\+LLM pipelines, even state\-of\-the\-art open\-source models struggle to handle such tasks reliably in zero\-shot settings\. Only 4 of 35 configurations achieved F1 scores above 0\.5, with the best OCR\+LLM pipeline matching top VLM performance, though most OCR\+LLM combinations performed substantially worse\. Roughly 75% of all configurations scored below 0\.25\. Model scale influences performance, yet the relationship is non\-linear: substantially larger models do not guarantee proportionally better results\. Input quality, particularly the structural preservation of OCR output, emerges as a critical factor independent of downstream model capability\. These findings provide practical guidance for practitioners selecting models for document processing workflows while highlighting fundamental limitations in current open\-source approaches to structured information extraction\.

###### Keywords:

Optical Character Recognition Large Language Model Visual Large Language Model Benchmark Open\-Source\.

## 1Introduction

The ability to extract structured information from unstructured or semi\-structured sources has become an increasingly critical capability in modern data\-driven workflows\. Digital transformations across all sectors require these workflows to reliably integrate information from scanned documents or digitally edited forms\. The rapid advancement of Optical Character Recognition \(OCR\), Large Language Models \(LLMs\), and Vision\-Language Models \(VLMs\) has fundamentally transformed this landscape, making such extraction tasks more accurate, scalable, and accessible than ever before\. Next to proprietary solutions, often marketed as all\-encompassing general\-purpose models, a rich ecosystem of free\-to\-use, open\-source OCR engines, LLMs, and VLMs has emerged\. These openly available models present substantial opportunities for processing large volumes of unstructured data, such as the ubiquitous PDF format, without incurring the costs or restrictions of commercial systems\.

Leveraging this potential requires more and systematic empirical evaluations of how freely available, off\-the\-shelf models perform on realistic extraction tasks\. This is even more important given that most of this technology has been developing faster than corresponding regulation\. While many benchmarks and evaluations focus on tasks that are either simple or not classified ashigh\-riskaccording to the EU AI Act\[[6](https://arxiv.org/html/2608.18289#bib.bib1)\], using the technology in high risk settings such as health care or workflows in the public sector requires extensive empirical evaluations\. Even though these kinds of evaluations are more on the applied side of research we argue that such evaluations are contributions that can and should be made by researchers to build trust in open source technology in the public sector\.

To address this challenge, this work makes two main contributions\. For one, we devise an evaluation protocol for extraction performance of a high risk use case in the public sector, student applications in an international study program\. This task is representative for many other complex tasks: it requires processing of heterogeneous PDF documents containing a mixture of structured and unstructured data, extracting targeted information, and returning it in a well\-defined structured format\. This workflow, illustrated in Figure[2](https://arxiv.org/html/2608.18289#S3.F2)is representative of operational use cases encountered across numerous domains\. Second we use this evaluation protocol for a benchmark assessing the capabilities of open OCR systems combined with LLMs, as well as standalone VLMs\. The results are discussed with respect to the extraction performance as well as other factors relevant for practical usage, such as computational efficiency\.

## 2Related Work

While OCR engines, LLMs, and VLMs have each been evaluated extensively in single\-step scenarios, benchmarks assessing their individual or combined performance on complex, multi\-step extraction tasks in a zero\-shot setting remain sparse\. Such tasks, which involve converting documents into machine\-readable text, extracting specified information, and returning it in a strictly defined structure, closely reflect how these models are likely to be deployed in practice and constitute precisely the end\-to\-end pipeline evaluated in this work\. Existing benchmarks tend to isolate individual components of this pipeline\.

With respect to OCR, Khan et al\.\[[18](https://arxiv.org/html/2608.18289#bib.bib2)\]evaluated several engines on the CBC Reports Dataset, measuring accuracy, processing time and error rates\. PaddleOCR and EasyOCR emerged as the strongest performers, a finding that directly informed the selection of OCR models used in this work\. On the side of information extraction from unstructured and semi\-structured documents, Ntinopoulos et al\.\[[21](https://arxiv.org/html/2608.18289#bib.bib6)\]benchmarked mostly LLMs and table transformers on electronic health records in a zero\-shot setting, finding that proprietary models consistently achieved strong results with F1 scores above 0\.9, outperforming several open\-source models\. They also found that table transformers achieved higher extraction performance than LLMs when only layout structure is considered, but LLMs surpass table transformers once cell content becomes relevant\. Similarly, Gao et al\.\[[7](https://arxiv.org/html/2608.18289#bib.bib7)\]examined fine\-grained extraction across multiple information types, observing that encoder\-decoder architectures generalize better to unseen information types, while decoder\-only architectures handle unseen task forms more effectively\. Notably, model scale did not consistently predict performance, a theme that recurs across the literature\.

The interplay between OCR and multimodal LLMs for table extraction was investigated by Nunes et al\.\[[22](https://arxiv.org/html/2608.18289#bib.bib3)\], who benchmarked OCR systems and MLLMs on extracting tables from images using a subset of non\-complex tables from PubTables\-1M\. Although their setup does not include a full end\-to\-end extraction pipeline, it closely aligns with the benchmark presented here and provides valuable insights\. Similar to this work, they enforce structured outputs by prompting models with a predefined Pydantic schema\. Evaluating both proprietary and open models, they report strong performance across categories, with several models achieving F1 scores between 85 and 95, depending on the evaluation aspect\.

Regarding structured output generation, Tenckhoff et al\.\[[28](https://arxiv.org/html/2608.18289#bib.bib4)\]benchmarked 22 open\-source models across five prompting strategies on a synthetically generated dataset, evaluating their ability to extract structured data and produce valid JSON from natural\-language text, with GPT\-4o included as a proprietary reference\. The results reinforce that prompting strategy and model architecture are at least as influential as model scale, and that leading open\-source models such as Gemma3\-27B can rival proprietary systems\.

Finally, Roberts et al\.\[[26](https://arxiv.org/html/2608.18289#bib.bib5)\]assessed VLMs on their ability to extract structure directly from images, using an automatic evaluation approach based on rendering model outputs and comparing them to the original\. Closed\-weight models, in particular GPT\-4o, significantly outperformed open\-weight alternatives, and VLMs struggle to pick up visual nuances and were shown to be sensitive to prompt formulation, suggesting that zero\-shot standardized prompting may be suboptimal for such tasks\.

Taken together, these works highlight meaningful progress across the individual components relevant to document extraction pipelines, while also underscoring the absence of benchmarks that evaluate free and open models holistically on the complete three\-step problem of document ingestion, information extraction and structured output generation that this paper directly addresses\.

## 3Methods

In the following we first describe the study program application data and the extraction task, followed by a description of the extraction workflow and evaluation metrics\.

### 3\.1Data

The dataset used in this benchmark consists of 100 academic transcripts submitted as part of applications to a German Data Science Master’s programme, provided as PDF documents\. Of these, 94 are written entirely in English, three are bilingual in English and Turkish, two are in German, and one combines English and Russian\. The vast majority of applicants originate from Asian countries, accounting for over 90% of the dataset\. Each document typically contains structured data in the form of tables as well as unstructured elements such as free text, images and institutional sigils\. As the documents contain personal information, the dataset cannot be published in its current form and must be anonymized prior to any release\. An example document is shown in Figure[1](https://arxiv.org/html/2608.18289#S3.F1)\.

All documents were standardized to A4 format and additionally converted to JPG images to accommodate models requiring image input\. Each document was then manually parsed to extract all courses deemed relevant to the extraction task, yielding a ground truth against which model outputs are evaluated\. The extraction follows a ruleset designed to handle the heterogeneity and formatting inconsistencies typical of international academic documents:

- •Extract only Computer Science or Mathematics courses\.
- •In multi\-language documents, prefer the English version\.
- •In case of multiple grade formats, prefer letter grades\.
- •Missing values are filled withN/A\.
- •Without a clear designation of total credits, select the first value in a row\.
- •Formatting inconsistencies such as erroneous spacing or incorrect characters are preserved as\-is in the ground truth\.

Table[1](https://arxiv.org/html/2608.18289#S3.T1), which shows the ground truth belonging to the document shown in Figure[1](https://arxiv.org/html/2608.18289#S3.F1), illustrates the application of these rules\. With multiple credit columns given and none being explicitly designated, the first credit value for each course is selected\. For example, "OPERATING SYSTEMS" appears with both theory \(3\.0\) and practical \(1\.0\) credits in the original document, but only 3\.0 is extracted as it is the first credit value associated with that course\.

The extracted ground truth comprises four columns:academic field,course name,grade, andawarded credits, totaling 952 unique rows across all documents\. The number of extracted rows per document ranges from 0 to 44, with no duplicate entries within any single document\.

![Refer to caption](https://arxiv.org/html/2608.18289v1/figures/cropped-version.png)Figure 1:Anonymized example document\.Table 1:Manually extracted Ground Truth for the Document in Figure[1](https://arxiv.org/html/2608.18289#S3.F1)
### 3\.2Extraction

We compare the performance of OCR engines combined with LLMs against standalone VLMs in a three\-step pipeline: digitalizing and preparing document content, extracting task\-relevant information as specified in the prompt, and returning it as a structured output\. All LLMs and VLMs are run using the Ollama platform, which provides consistent base settings across all models as well as an integrated format parameter that enforces structured output via a Pydantic schema\.

![Refer to caption](https://arxiv.org/html/2608.18289v1/ablauf.png)Figure 2:Workflow of extraction and evaluation pipelineThe OCR engines used in this benchmark were selected based on two criteria: popularity among practitioners, measured by GitHub star count, and performance reported in recent benchmarking literature\[[9](https://arxiv.org/html/2608.18289#bib.bib9)\]\[[20](https://arxiv.org/html/2608.18289#bib.bib8)\]\. All models are used with pretrained weights without any further fine\-tuning\. The following engines were evaluated: Pytesseract\[[27](https://arxiv.org/html/2608.18289#bib.bib11)\], PP\-OCRv5 \(PaddleOCR\)\[[5](https://arxiv.org/html/2608.18289#bib.bib12)\], PP\-StructureV3 \(PaddleOCR\)\[[5](https://arxiv.org/html/2608.18289#bib.bib12)\], MinerU\[[30](https://arxiv.org/html/2608.18289#bib.bib13)\], EasyOCR\[[11](https://arxiv.org/html/2608.18289#bib.bib14)\], and docTR\[[15](https://arxiv.org/html/2608.18289#bib.bib15)\]\. Table[2](https://arxiv.org/html/2608.18289#S3.T2)provides an overview of each model’s input, output and language configuration\. Table[3](https://arxiv.org/html/2608.18289#S3.T3)details the architecture components for engines where explicit configuration was specified, with selections based on the highest\-performing options as indicated in each engine’s official documentation and benchmarks\.

Table 2:OCR model configurations and output formats\.Table 3:OCR model architecture components \(specified models only\)\.For the extraction of relevant structured information from the OCR output, a single LLM was deliberately chosen to reduce variables and place the focus on the effect of model scale\. Qwen3\[[31](https://arxiv.org/html/2608.18289#bib.bib10)\], a modern model family released in 2025, was selected as it offers a wide range of parameter counts within a single architecture\. Five parameter sizes were evaluated, as detailed in Table[4](https://arxiv.org/html/2608.18289#S3.T4)\.

Table 4:Qwen3 model architecture parameters\[[31](https://arxiv.org/html/2608.18289#bib.bib10)\]\. Context lengths as specified in Ollama\[[23](https://arxiv.org/html/2608.18289#bib.bib22)\]\. Q/KV heads refer to query and key\-value attention heads\[[29](https://arxiv.org/html/2608.18289#bib.bib21)\]\.#### VLM based Extraction

VLMs were selected following the similar criteria as the OCR engines, drawing from both the most used models from Ollama model zoo and recent literature\[[14](https://arxiv.org/html/2608.18289#bib.bib28)\]\. An effort was made to keep parameter counts comparable across models at approximately 30 billion parameters, though not all models offered variants at this scale\. The following models were evaluated: LLaVA \(7B\)\[[17](https://arxiv.org/html/2608.18289#bib.bib16)\], Ministral\-3 \(14B\)\[[16](https://arxiv.org/html/2608.18289#bib.bib17)\], Gemma3 \(27B\)\[[8](https://arxiv.org/html/2608.18289#bib.bib18)\], Qwen2\.5VL \(32B\)\[[2](https://arxiv.org/html/2608.18289#bib.bib19)\], and Qwen3VL \(32B\)\[[3](https://arxiv.org/html/2608.18289#bib.bib20)\]\. As with the LLM based pipeline, structured outputs were generated using the same Pydantic schema via the Ollama format API\. Table[5](https://arxiv.org/html/2608.18289#S3.T5)provides a detailed overview of each model\.

Table 5:Vision\-language model overview\.
#### Prompting Strategy and Experimental Design

The prompt design incorporates several evidence\-based strategies from recent literature: meaning\-typed prompting via Pydantic schemas rather than rigid JSON definitions for reliable structured output\[[10](https://arxiv.org/html/2608.18289#bib.bib23)\], a verification phase where the model reviews its own output\[[25](https://arxiv.org/html/2608.18289#bib.bib24)\], delimiters to separate schema definitions and text sections\[[4](https://arxiv.org/html/2608.18289#bib.bib25)\], and concise, direct instructions\[[13](https://arxiv.org/html/2608.18289#bib.bib26)\]\. Structured output is enforced via the following Pydantic schema:

classCourse\(BaseModel\):

academic\_field:Literal\["Computer␣Science","Mathematics"\]

course\_name:str

grade:float\|str\|None=None

awarded\_credits:float\|None=None

classCourses\(BaseModel\):

courses:List\[Course\]

The prompt follows a structured format: input data and goal definition, schema presentation with illustrative examples, output rules, context provision, and a verification section with output format specifications\. Temperature is set to 0 for all models, following best practices for deterministic structured generation as recommended in the Ollama documentation\[[24](https://arxiv.org/html/2608.18289#bib.bib27)\]\.

All experiments were executed on a compute cluster using a single NVIDIA H200 GPU with 141 GB VRAM, 32 GB system memory, and 8 CPU cores\. System memory and CPU utilization remained well below capacity throughout all experiments\. For the Qwen3:235b model, an NVIDIA B200 GPU with 192 GB VRAM was briefly utilized to accelerate inference\.

### 3\.3Evaluation

Model outputs were first validated and normalized to ensure comparability with ground truth\. Empty grade and credit cells columns were filled withN/Ato match the ground truth format\. When models failed to return data in the required structure due to recursive reasoning patterns, additional text generation, inclusion of extra columns, or omission of required fields, an empty dataframe with the correct schema was substituted, effectively treating non\-compliant outputs as complete extraction failures\.

Both the extracted data and ground truth underwent identical normalization procedures: \(1\) index columns were dropped, \(2\) leading and trailing whitespace was stripped, \(3\) academic field and course name values were converted to lowercase, and \(4\) numerical values were cast to floats, while any alphabetic characters in grade or credit columns were also converted to lowercase\.

#### Evaluation Metrics

For our evaluation we focus on exact string matches after lowercasing\. Both the ground truth and extracted data were tokenized by converting each row, consisting of academic field, course name, grade, and awarded credits, into a single unique token as is illustrated in Figure[3](https://arxiv.org/html/2608.18289#S3.F3)\. Given that ground truth contains no duplicate rows, both datasets were represented as sets of row tokens, enabling efficient set\-based comparison\. From these sets, a combined set of all unique tokens was constructed using set intersection\.

![Refer to caption](https://arxiv.org/html/2608.18289v1/tokenization.png)Figure 3:Tokenization visualizedFollowing standard practice in information or text extraction benchmarks\[[1](https://arxiv.org/html/2608.18289#bib.bib31),[12](https://arxiv.org/html/2608.18289#bib.bib29),[19](https://arxiv.org/html/2608.18289#bib.bib30)\]we computed Precision, Recall, and F1 scores\. To determine True Positives \(TP\), False Positives \(FP\), and False Negatives \(FN\) for these metrics, we adopt a set\-theoretic approach based on Jaccard similarity\[[1](https://arxiv.org/html/2608.18289#bib.bib31)\]:

- •True Positives \(TP\): Tokens present in both extracted output and ground truth
- •False Positives \(FP\): Tokens in extracted output but absent from ground truth
- •False Negatives \(FN\): Tokens in ground truth but absent from extracted output

In the subsequent analysis, F1 Score serves as the primary metric for model evaluation and comparison, as it provides a balanced measure of both extraction accuracy and completeness, making it particularly suitable for assessing performance on imbalanced extraction tasks where both false positives and false negatives carry significant cost\.

### 3\.4Results

In Table[6](https://arxiv.org/html/2608.18289#S3.T6)we show the overall results for each OCR\+LLM and VLM model and Figure[4](https://arxiv.org/html/2608.18289#S3.F4)shows the best F1 score per model, comparing the top\-performing OCR\+LLM combination for each OCR engine against all VLMs\.

##### VLMs outperform OCR\+LLM approaches

In the OCR\+LLM pipeline, the best\-performing LLM was Qwen3:235B in all cases\. The results reveal a clear performance gap between OCR engines, largely attributable to the quality and structure of their output\. MinerU stands out as the strongest OCR engine, achieving an F1 score of 0\.509, placing it on par with the best\-performing VLM, Qwen2\.5\-VL \(0\.509\)\. This can be attributed to MinerU’s output format, which preserves the spatial layout of the page by grouping text that appears on the same horizontal line into a single row, and representing tabular content using HTML tags to explicitly encode cell boundaries\. Other OCR engines, by contrast, insert line breaks after each individual text block regardless of its position on the page, and largely fail to represent table structure semantically\. This directly demonstrates the critical importance of input quality to downstream LLM performance: the structure of the OCR output is at least as consequential as the capability of the LLM processing it\. On the VLM side, results are polarized\. Qwen2\.5\-VL \(0\.509\) and Ministral\-3 \(0\.508\) achieve the highest scores overall, while LLaVA \(0\.000\) and Gemma3 \(0\.015\) perform at or near zero, most likely due to malformed outputs that were treated as complete extraction failures under the evaluation protocol\. Notably, Qwen2\.5\-VL marginally outperforms its successor Qwen3\-VL \(0\.487\), suggesting that a newer model version does not necessarily translate to improved task\-specific performance\. Overall, results across both pipelines are poor: the mean F1 score across all VLMs and the best OCR\+LLM combination per engine is 0\.282, with the highest observed score capped at 0\.509\.

Figure 4:Best F1 score per model: top OCR\+LLM combination \(always Qwen3:235B\) versus all VLMs\.Table 6:Extraction performance metrics for OCR\+LLM pipeline and VLM models\.
##### Parameter Size and Extraction Performance

Figure[5](https://arxiv.org/html/2608.18289#S3.F5)illustrates how F1 score varies across Qwen3 parameter sizes for each OCR engine\. A consistent pattern emerges: the 0\.6B model performs substantially worse than all larger variants across every OCR engine, frequently producing near\-zero scores\. This is likely not solely a function of model capacity, but also a known instability of the Qwen3:0\.6B model, which has been documented to enter a repetitive generation loop, producing incoherent output that fails to conform to the required Pydantic schema\. Beyond the 0\.6B outlier, larger models generally yield modest but consistent improvements, with the 235B variant typically achieving the highest score per engine\. However, this trend is not universal: for PPOCRv5 and docTR, the 14B model scores marginally higher than the 32B model, reinforcing the finding from related work that model scale and task performance do not maintain a strictly linear relationship\.

Figure 5:F1 scores across Qwen3 parameter sizes for each OCR engine\.
##### Simplified Extraction with Fewer Attributes

To investigate the sensitivity of the performance with reduced task complexity, we simplified the task and neglected the grade and awarded credits attributes\. Figure[6](https://arxiv.org/html/2608.18289#S3.F6)presents the improvement in F1 score \(Δ\\DeltaF1\) when the grade column, the awarded credits column, or both are removed from both ground truth and model output prior to evaluation\. Positive deltas indicate improved scores under the reduced column setting\. Across all model configurations, the largest improvements are observed when both columns are removed, retaining only academic field and course name\. Removing grades consistently yields a larger improvement than removing awarded credits, though the magnitude varies considerably across configurations, ranging from below 0\.01 to above 0\.1\. This suggests that grade extraction poses a greater challenge, likely due to the diversity of grading formats across transcripts, including both letter and numerical grades, which introduces an additional layer of ambiguity even when explicit handling rules are provided in the prompt\. These patterns hold consistently across both VLM and OCR\+LLM configurations\. Finally, as observed in Figure[5](https://arxiv.org/html/2608.18289#S3.F5), the 0\.6B Qwen3 configurations show minimal improvement even when columns are removed\. This further supports the interpretation that the near\-zero baseline scores for these configurations are driven primarily by output invalidity due to the model’s repetitive generation behavior, rather than extraction difficulty\.

Figure 6:F1 improvement when removing grade, awarded credits, or both\.

## 4Conclusion

This benchmark evaluated the end\-to\-end capability of open\-source OCR engines, LLMs, and VLMs to extract structured information from heterogeneous real\-world documents in a zero\-shot setting, using all models in their pretrained state with best available configurations\. The results reveal that none of the tested approaches can be considered a reliable out\-of\-the\-box solution for complex extraction tasks of this kind, with roughly 75% of all configurations scoring below an F1 of 0\.25\.

A central finding is that the quality of LLM output is strongly dependent on the quality of the context provided\. OCR engines that preserve the spatial and structural layout of the source document, as MinerU does via row\-wise text grouping and HTML\-encoded tables, yield dramatically better downstream extraction results than engines that produce unstructured, block\-fragmented output\. This underlines that input preparation is at least as important as model capability in such pipelines\.

VLMs generally outperform OCR\+LLM pipelines in this task, as they bypass the OCR step entirely and operate directly on the document image\. However, their performance is far from reliable, with several models producing near\-zero scores due to malformed outputs\. It is also worth noting that a newer model is not always a better one: Qwen2\.5\-VL outperformed its successor Qwen3\-VL on this task, and similarly, larger LLM parameter counts do not guarantee proportionally better results, though very small models such as Qwen3:0\.6B are clearly insufficient for tasks of this complexity\.

These findings apply specifically to the zero\-shot, off\-the\-shelf setting evaluated here\. The performance levels observed are not necessarily an inherent ceiling: improved preprocessing and data quality feeding into the LLM or VLM, more targeted prompting strategies, and refined schema definitions are all promising directions that could meaningfully improve results\. These do however require additional effort and experimentation, and suggest that deploying such pipelines reliably in practice demands more than simply selecting a capable model off the shelf\. Improved data preparation, more targeted prompting strategies, and refined schema definitions are all promising directions that could meaningfully improve performance without requiring fine\-tuning, and warrant further investigation\.

#### Disclosure of Interests\.

The authors have no competing interests to declare that are relevant to the content of this article\.

## References

- \[1\]\(2025\)A comparative study of PDF parsing tools across diverse document categories\.External Links:2410\.09871,[Link](https://arxiv.org/abs/2410.09871)Cited by:[§3\.3](https://arxiv.org/html/2608.18289#S3.SS3.SSSx1.p2.1)\.
- \[2\]S\. Bai and et al\.\(2025\)Qwen2\.5\-VL technical report\.External Links:2502\.13923,[Link](https://arxiv.org/abs/2502.13923)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx1.p1.1)\.
- \[3\]S\. Bai and et al\.\(2025\)Qwen3\-VL technical report\.External Links:2511\.21631,[Link](https://arxiv.org/abs/2511.21631)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx1.p1.1)\.
- \[4\]L\.\-C\. Chen, H\.\-T\. Weng, M\. S\. Pardeshi, C\.\-M\. Chen, R\.\-K\. Sheu, and K\.\-C\. Pai\(2025\)Evaluation of prompt engineering on the performance of a large language model in document information extraction\.Electronics14\(11\),pp\. 2145\.External Links:[Document](https://dx.doi.org/10.3390/electronics14112145)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx2.p1.1)\.
- \[5\]C\. Cui and et al\(2025\)PaddleOCR 3\.0 technical report\.External Links:2507\.05595,[Link](https://arxiv.org/abs/2507.05595)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[6\]European Parliament and Council\(2024\)Regulation \(eu\) 2024/1689 of the european parliament\.Vol\.L\.Note:OJ L 12\.7\.2024, p\. 1–226External Links:[Link](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng)Cited by:[§1](https://arxiv.org/html/2608.18289#S1.p2.1)\.
- \[7\]J\. Gao, H\. Zhao, Y\. Zhang, W\. Wang, C\. Yu, and R\. Xu\(2023\)Benchmarking large language models with augmented instructions for fine\-grained information extraction\.External Links:2310\.05092,[Link](https://arxiv.org/abs/2310.05092)Cited by:[§2](https://arxiv.org/html/2608.18289#S2.p2.1)\.
- \[8\]Gemma Team, A\. Kamath, and et al\.\(2025\)Gemma 3 technical report\.External Links:2503\.19786,[Link](https://arxiv.org/abs/2503.19786)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx1.p1.1)\.
- \[9\]A\. Heakl, A\. Sohail, M\. Ranjan, R\. Hossam, G\. S\. Ahmad, M\. El\-Geish, O\. Maher, Z\. Shen, F\. Khan, and S\. Khan\(2025\)KITAB\-Bench: a comprehensive multi\-domain benchmark for arabic OCR and document understanding\.External Links:2502\.14949,[Link](https://arxiv.org/abs/2502.14949)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[10\]C\. Irugalbandara\(2024\)Meaning typed prompting: a technique for efficient, reliable structured output generation\.External Links:2410\.18146,[Link](https://arxiv.org/abs/2410.18146)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx2.p1.1)\.
- \[11\]JaidedAI\(2020\)EasyOCR: ready\-to\-use OCR with 80\+ supported languages\.GitHub\.Note:[https://github\.com/JaidedAI/EasyOCR](https://github.com/JaidedAI/EasyOCR)External Links:[Link](https://github.com/JaidedAI/EasyOCR)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[12\]D\. Kim and Y\. Hong\(2011\)Figure text extraction in biomedical literature\.PloS one6\(1\),pp\. e15338\.External Links:[Document](https://dx.doi.org/10.1371/journal.pone.0015338)Cited by:[§3\.3](https://arxiv.org/html/2608.18289#S3.SS3.SSSx1.p2.1)\.
- \[13\]Y\. Li\(2023\)A practical survey on zero\-shot prompt design for in\-context learning\.InProceedings of the Conference Recent Advances in Natural Language Processing \- Large Language Models for Natural Language Processing,pp\. 641–647\.External Links:[Document](https://dx.doi.org/10.26615/978-954-452-092-2%5F069)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx2.p1.1)\.
- \[14\]Z\. Li, X\. Wu, H\. Du, F\. Liu, H\. Nghiem, and G\. Shi\(2025\)A survey of state of the art large vision language models: alignment, benchmark, evaluations and challenges\.External Links:2501\.02189,[Link](https://arxiv.org/abs/2501.02189)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx1.p1.1)\.
- \[15\]H\. Liao, A\. RoyChowdhury, W\. Li, A\. Bansal, Y\. Zhang, Z\. Tu, R\. K\. Satzoda, R\. Manmatha, and V\. Mahadevan\(2023\)DocTr: document transformer for structured information extraction in documents\.External Links:2307\.07929,[Link](https://arxiv.org/abs/2307.07929)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[16\]A\. H\. e\. al\. Liu\(2026\)Ministral 3\.External Links:2601\.08584,[Link](https://arxiv.org/abs/2601.08584)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx1.p1.1)\.
- \[17\]H\. Liu, C\. Li, Q\. Wu, and Y\. J\. Lee\(2023\)Visual instruction tuning\.External Links:2304\.08485,[Link](https://arxiv.org/abs/2304.08485)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx1.p1.1)\.
- \[18\]M\. A\. Naqi Hadi, M\. Gul, M\. Khan, G\. N\. Alwakid, and N\. Zaman Jhanjhi\(2024\)Benchmarking performance analysis of optical character recognition techniques\.In2024 26th International Multi\-Topic Conference \(INMIC\),pp\. 1–6\.External Links:[Document](https://dx.doi.org/10.1109/INMIC64792.2024.11004392)Cited by:[§2](https://arxiv.org/html/2608.18289#S2.p2.1)\.
- \[19\]Z\. Nasar, S\.W\. Jaffry, and M\.K\. Malik\(2018\)Information extraction from scientific articles: a survey\.Scientometrics117,pp\. 1931–1990\.External Links:[Document](https://dx.doi.org/10.1007/s11192-018-2921-5)Cited by:[§3\.3](https://arxiv.org/html/2608.18289#S3.SS3.SSSx1.p2.1)\.
- \[20\]M\. Nazeem, A\. R, N\. S, and R\. R\. R\(2024\)Open\-source OCR libraries: a comprehensive study for low resource language\.InProceedings of the 21st International Conference on Natural Language Processing \(ICON\),pp\. 416–421\.External Links:[Link](https://aclanthology.org/2024.icon-1.48/)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[21\]V\. Ntinopoulos, H\. Rodriguez Cetina Biefer, I\. Tudorache, N\. Papadopoulos, D\. Odavic, P\. Risteski, A\. Haeussler, and O\. Dzemali\(2025\)Large language models for data extraction from unstructured and semi\-structured electronic health records: a multiple model performance evaluation\.BMJ Health & Care Informatics32\(1\),pp\. e101139\.External Links:[Document](https://dx.doi.org/10.1136/bmjhci-2024-101139)Cited by:[§2](https://arxiv.org/html/2608.18289#S2.p2.1)\.
- \[22\]G\. Nunes, V\. Rolla, D\. Pereira, V\. Alves, A\. Carreiro, and M\. Baptista\(2025\)Benchmarking table extraction: multimodal LLMs vs traditional OCR\.InProceedings of the 1st Joint Workshop on Large Language Models and Structure Modeling \(XLLM 2025\),Vienna, Austria,pp\. 8–15\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.xllm-1.2)Cited by:[§2](https://arxiv.org/html/2608.18289#S2.p3.1)\.
- \[23\]Ollama\(2025\)Qwen3 — ollama library\.Note:[https://ollama\.com/library/qwen3](https://ollama.com/library/qwen3)Accessed: 2025\-03\-03External Links:[Link](https://ollama.com/library/qwen3)Cited by:[Table 4](https://arxiv.org/html/2608.18289#S3.T4)\.
- \[24\]Ollama\(2025\)Structured outputs — ollama documentation\.Note:[https://docs\.ollama\.com/capabilities/structured\-outputs](https://docs.ollama.com/capabilities/structured-outputs)Accessed: 2025\-03\-03External Links:[Link](https://docs.ollama.com/capabilities/structured-outputs)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx2.p3.1)\.
- \[25\]M\. P\. Polak and D\. Morgan\(2024\)Extracting accurate materials data from research papers with conversational language models and prompt engineering\.Nature Communications15\(1\),pp\. 1569\.External Links:[Document](https://dx.doi.org/10.1038/s41467-024-45914-8)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.SSSx2.p1.1)\.
- \[26\]J\. S\. Roberts, T\. Lee, C\. H\. Wong, M\. Yasunaga, Y\. Mai, and P\. Liang\(2024\)Image2Struct: benchmarking structure extraction for vision\-language models\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 115058–115097\.External Links:[Document](https://dx.doi.org/10.52202/079017-3653)Cited by:[§2](https://arxiv.org/html/2608.18289#S2.p5.1)\.
- \[27\]R\. Smith\(2007\)An overview of the tesseract OCR engine\.InNinth International Conference on Document Analysis and Recognition \(ICDAR\),Vol\.2,pp\. 629–633\.External Links:[Document](https://dx.doi.org/10.1109/ICDAR.2007.4376991)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[28\]S\. Tenckhoff, M\. Koddenbrock, and E\. Rodner\(2026\)LLMStructBench: benchmarking large language model structured data extraction\.External Links:2602\.14743,[Link](https://arxiv.org/abs/2602.14743)Cited by:[§2](https://arxiv.org/html/2608.18289#S2.p4.1)\.
- \[29\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin\(2023\)Attention is all you need\.External Links:1706\.03762,[Link](https://arxiv.org/abs/1706.03762)Cited by:[Table 4](https://arxiv.org/html/2608.18289#S3.T4)\.
- \[30\]B\. Wang and et al\.\(2024\)MinerU: an open\-source solution for precise document content extraction\.External Links:2409\.18839,[Link](https://arxiv.org/abs/2409.18839)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p2.1)\.
- \[31\]A\. Yang and et al\.\(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3\.2](https://arxiv.org/html/2608.18289#S3.SS2.p3.1),[Table 4](https://arxiv.org/html/2608.18289#S3.T4)\.