Office Comprehension Benchmark

# Office Comprehension Benchmark
Source: [https://arxiv.org/abs/2607.01245](https://arxiv.org/abs/2607.01245)
Authors:[Firoz Shaik](https://arxiv.org/search/cs?searchtype=author&query=Shaik,+F),[Mateus Picanço Lima Gomes](https://arxiv.org/search/cs?searchtype=author&query=Gomes,+M+P+L),[Tanvir Aumi](https://arxiv.org/search/cs?searchtype=author&query=Aumi,+T),[Jingci Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+J),[Milos Milunovic](https://arxiv.org/search/cs?searchtype=author&query=Milunovic,+M),[Filip Basara](https://arxiv.org/search/cs?searchtype=author&query=Basara,+F),[Ivana Jovanovic](https://arxiv.org/search/cs?searchtype=author&query=Jovanovic,+I),[Vishwas Suryanarayanan](https://arxiv.org/search/cs?searchtype=author&query=Suryanarayanan,+V),[Neha Nandan Kenkare](https://arxiv.org/search/cs?searchtype=author&query=Kenkare,+N+N),[Weiyao Xie](https://arxiv.org/search/cs?searchtype=author&query=Xie,+W),[Zhipeng Han](https://arxiv.org/search/cs?searchtype=author&query=Han,+Z),[Zheng Zhang](https://arxiv.org/search/cs?searchtype=author&query=Zhang,+Z),[Waleed Shahid](https://arxiv.org/search/cs?searchtype=author&query=Shahid,+W),[Jay Rathi](https://arxiv.org/search/cs?searchtype=author&query=Rathi,+J),[Russell Scherer](https://arxiv.org/search/cs?searchtype=author&query=Scherer,+R),[Thong Q\. Nguyen](https://arxiv.org/search/cs?searchtype=author&query=Nguyen,+T+Q),[Michael Bentley](https://arxiv.org/search/cs?searchtype=author&query=Bentley,+M),[Tamara Stankovic](https://arxiv.org/search/cs?searchtype=author&query=Stankovic,+T),[Rasika Chakravarthy](https://arxiv.org/search/cs?searchtype=author&query=Chakravarthy,+R),[Vishal Chowdhary](https://arxiv.org/search/cs?searchtype=author&query=Chowdhary,+V)

[View PDF](https://arxiv.org/pdf/2607.01245)

> Abstract:We introduce Office Comprehension Bench \(OCB\), the first public benchmark to jointly evaluate LLM systems on Word, Excel, and PowerPoint comprehension over native file formats \(\.docx, \.xlsx, \.pptx\) and their variants\. OCB consists of two tracks\. File Fidelity Q&A tests structural and visual perception of office artifacts \- tables, charts, embedded images, formulas, and app\-specific elements such as headers, speaker notes, and named ranges\. Domain Q&A tests expert\-level reasoning grounded in real\-world industry documents across 12 professional domains, with queries requiring multi\-step analysis and synthesis across documents\. Each reference answer is decomposed into atomic, binary\-gradable claims, and an ensemble of LLM judges scores responses against each claim independently\. Even the strongest frontier system in its default reasoning mode reaches only about 59\.3% on Domain Q&A; increasing thinking depth within a tier does not move performance materially, while moving to a higher product tier yields modest gains\. We release the dataset, evaluation tooling, judge prompt, and a public leaderboard\.

## Submission history

From: Firoz Shaik \[[view email](https://arxiv.org/show-email/1b6987ba/2607.01245)\] **\[v1\]**Fri, 29 May 2026 03:19:32 UTC \(1,503 KB\)