Full document redaction with Qwen 3.6 27B with a Pi agent harness

Link to full blog post with all method details, results, and links to all relevant code/skills/prompts at the bottom of this post. Apologies for not having more links throughout, it seems this subreddit restricts too many links in posts. Document redaction tasks are complex tasks that require text and image recognition capabilities, long context understanding, and the ability to closely follow rules to redact specific information. Recently, local models have been developed that can do this all (e.g. Qwen 3.6). If agents, using these models, can be harnessed to perform contextual-aware redaction, this could result in significant time savings for people performing redaction tasks. In a previous post to this subreddit (see my profile), I investigated the possibility of using agentic workflows various LLMs to conduct end-to-end redaction and review. I found that Sonnet 4.6 was able to perform the tasks well, but local models such as Qwen 3.6 27B struggled. Since then I have optimised the local model settings and agentic harness. I am now getting acceptable results (demonstrated below) with agentic redaction using Qwen 3.6 27B. The important changes were a higher quantisation level (Q6 vs Q4 for the previous post), and optimised prompting and skills within a minimal agentic harness (using PI). Additionally, I created a Gradio-based UI so that people can interact with the agent easily. Below I'll give an overview of my method and results with example documents. Method The doc_redaction repo contains the code to deploy both the agentic redaction app, and the main Redaction app, which the agent uses as a tool to perform the redaction task. In this solution, Qwen 3.6 27B is deployed locally on my system, and serves both as the agentic model, and also as the VLM model used by the main Redaction app to perform some specialised tasks such as face and signature detection. Frontend agentic redaction interface The Agentic Redaction UI is a Gradio-based app that serves as an interface between the user and the agent for performing redaction tasks. The app allows users to upload a document, and to pass custom redaction instructions to the Pi agent backend. You can view and test this here with a free Gemini API key. When a user gives instructions to the agent in the Agentic Redaction GUI, then clicks 'Start redaction task', a prompt is passed to the model that is filled in with the relevant model/workspace information, and the custom instructions from the user. After this, the agent uses its standard set of tools (e.g. to access API endpoints, write files and code) along with the skills to redact the document. Agentic redaction GUI after pressing Start redaction task The tool use and some thinking/reporting to the user is streamed to the chatbox on the right while the agent is performing the task, so the user can keep track of progress. The user can send steering messages during the time the agent is redacting a document, or they can send follow up messages once the agent is finished to adjust and improve the redacted outputs according to the user's requirements. Local agentic model deployment (Qwen 3.6 27B) Qwen 3.6 27B was used as the agentic model performing the redaction tasks. The docker compose file linked in the post deploys Qwen 3.6 27B using the settings below, quantised to 6 bit with long context (114k tokens at KV cache quantised to 8 bit), and runs on a local system with 40GB VRAM. This was required as I found that quantisations lower than Q6 decreased code quality with the Qwen model to the point where completing redaction tasks became difficult. command: -hf unsloth/Qwen3.6-27B-MTP-GGUF --hf-file Qwen3.6-27B-UD-Q6_K_XL.gguf --mmproj-url https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF/resolve/main/mmproj-BF16.gguf --n-gpu-layers "-1" --ctx-size "114688" -ub "512" --fit "off" --temp "0.7" --top-k "20" --top-p "0.95" --min-p "0.0" --frequency-penalty "1" --presence-penalty "0.0" --chat-template-kwargs "{\"preserve_thinking\": true}" --host "0.0.0.0" --port "8080" --no-warmup --seed "42" --image_min_tokens "300" --parallel "1" --cache-type-k "q8_0" --cache-type-v "q8_0" --spec-type "draft-mtp" --spec-draft-n-max "2" The relevant Docker compose file also deploys the two apps for the Qwen VLM to interact with in local Docker containers - the agentic GUI based based on the Pi agent framework that the user interacts with, and the original Document Redaction app GUI that a human user can use to redact and review the outputs afterwards (see below). Qwen 3.6 27B is used as the agentic model, and also as the VLM model used by the main Redaction app to perform tasks such as face and signature detection. Backend - Document Redaction app Serving as the backend in this task, the Document Redaction App is a Gradio UI app that provides a number of FastAPI endpoints for document redaction and review functions. uses OCR models such as Tesseract and PaddleOCR (ppOCR v6) to extract text locally, and PII identification models such as spaCy (within the Microsoft Presidio package for PII identification). VLM models served locally can be used to highlight faces and signatures in the document, or do a second pass on difficult words (a 'hybrid' OCR approach). Document Redaction App review interface Example documents to test Below I will show a few pages from the example documents redacted by the above agentic system so you can see how it performs. I asked the Qwen agent in Pi to redact three different documents, each following specific user instructions, described in the Results section below for each document. To keep the test consistent, I looked only at the initial outputs from the agent process, i.e. I did not ask any steering or follow up questions to the agent to modify the returned files. The documents tested were: A two page document of Examples of emails sent to a professor before applying A seven page Sister City Partnership document with scanned pages and signatures A 22-page policy document for residents for a local government authority in the UK with many references to places, names, and faces to redact throughout. Results 1. A two page document of example emails The first document was relatively simple with extractable text and no images. I gave the agent the following custom instructions: - Any redaction box related to Dr Kornbluth should be removed - References to Dr Hyde, or Dr Hyde's lab should be redacted. Also any references to Lauren, or Lauren Lilley - All mentions of Universities and their names should be redacted Example redacted output for page 1 can be seen below. Example emails page 1 results It seems that the LLM followed all the instructions. There are no redaction boxes visible for Dr Kornbluth, all references to Dr Hyde and Lauren Lilley have been redacted, and no University name is visible. Overall score: 10/10 2. A seven page document with scanned pages and signatures The next document is much more challenging. The Partnership Agreement Toolkit document contains signatures to identify, and several pages that were scanned in, requiring the use of OCR rather than simple PDF text extraction. I gave the agent the following custom instructions: - All signatures should be redacted - Any redaction box related to general country names should be removed - All redactions for Rudy Giuliani should be removed - All mentions of London, and 'Sister City' should be redacted Below I will highlight a few pages where the Qwen model did well and seemed to struggle to follow the rules. Page 1 This is a relatively easy page where most text can simply extracted, with a couple of images of text at the top. Partnership Toolkit Agreement page 1 results The agent performed mostly well. Sister city references are removed. The missed 'SisterCities' image version in the top left, which is picked up by OCR as a single word, can probably be given a pass as this exact term is not specified in the instructions. One big miss is that a large column-like redaction box is visible in the middle of the page, something that you would hope the agent would be able to pick up on during its checks. 8/10 for this page. Page 4 This is a page with a scanned image of a document. There are signatures, mentions of countries, and 'Sister city' mentions. Partnership Toolkit Agreement page 4 results We can see that the agent successfully redacted the signatures. However, it has missed the country names. Looking at the thinking of the model, it seems the agent trusted the initial automated redaction process to automatically remove all countries, without doing a proper check of the text afterwards to ensure that no country names were retained by error. 7/10 for this page. The rest of the document is similar - generally getting the signatures and redacting specific names / 'Sister city' references, but missing countries and some other specific rules. Overall, I gave the agent 7.5/10 for this document. 3. A 22-page policy document for residents The final document to test was a 22-page policy document for residents for a local authority in the UK with many references to places, names, and faces to redact throughout. I gave the agent the following custom instructions: - Redact any terms related to Lambeth, and Lambeth 2030 - Redact any names - Redact any photos of faces This document consists mostly of extractable text, but the presence of many images means that the model will need to use OCR to identify which contains photos of faces. Page 3 This page is mostly typed text, with a couple of photos of faces, and a drawing of a person. Policy document page 3 results For this example, the model has done quite well. All references to Lambeth have been redacted, and I can't see any visible names or photos of faces. The cartoon style face near the top of the image has been correctly ignored. However, there are a couple of large, column-like redaction boxes that are strangely positioned in the middle of the text, indicating redaction boxes that went wrong but were not picked up by the model. At least, these could be quickly removed by a human reviewer. 8/10 overall for the page. Page 18 This is a page with a number of photos of faces, but now also with lots of text. Policy document page 18 results The text redaction is generally ok (with some 'L's from Lambeth visible), but with a strange cluster of tiny redactions on the right side of the page that the agent did not pick up on. The agent redacted most faces, but missed some in the cluster towards the bottom. Pretty good, but I can't give more than 7.5/10 due to these misses. The rest of the document paints a similar picture - most faces (but not all) correctly redacted, then some small errors here and there throughout the document that the agent does not correct. Overall, I gave the system 8/10 for this document. Conclusion Document Score 1. Examples of emails sent to a professor before applying 10/10 2. Partnership Agreement Toolkit 7.5/10 3. Lambeth 2030 policy document 8/10 Overall, the Qwen 3.6 27B model in a Pi agent harness performed well to redact the test documents with specific (sometimes unusual) user instructions. But it did not get everything right. Sometimes it ignored instructions. Sometimes it followed the instructions, but made errors, and did not check them. Sometimes it expected a follow up prompt to specifically ask it to resolve the issues it knew remained. But despite these issues, the model correctly did about 80-90% of the redactions correctly throughout the test documents, including scanned pages, and redacting signatures or photos of faces. I think that using Qwen 3.6 27B in a local agentic system, along with follow up human review using the Document Redaction App GUI, could save significant time in redacting short to mid-length documents. I would say the Redaction app alone saves at least 60% of overall time, and the agentic system could save another ~20% of time, especially if follow up questions were used to prompt the agent to fix remaining issues. This system has the added benefits of running completely locally (potentially more secure for sensitive documents), and avoiding API costs. After about 50 pages with complex documents and instructions, I imagine inference speed and context window limitations could be a barrier to effective use with local systems. As local models improve in performance, I will continue to test them for redaction tasks to get a measure of progress. At this rate, I can imagine that within a couple of years, most document redaction tasks could be effectively performed by a local agentic system, with little time for human review needed afterwards. Please use the link below to see the full post. Link to full post with all methods, results, and links to all open code, skills, and prompts: here