Revision Prompting improves industrial LLM processes

# Revision Prompting Source: [https://revisionprompting.info/](https://revisionprompting.info/) ## Revision Promptingimproves industrial LLM processes *Disclaimer: A human wrote this page\.* ## TL;DR *Industrial prompting*refers to automated processes that prompt LLMs repeatedly with the same instruction on different inputs\. Often, an industrial prompt needs to re\-process an input after it has been updated\. *Revision Prompting*is a technique that makes industrial prompting of updated inputs**faster, cheaper, and more consistent**\. To this end, Revision Prompting supplies the LLM with the original input, the original output, and the changes to the input, and instructs it to produce a patch that updates the original output accordingly\. ## The Problemthat Revision Prompting solves We prompt LLMs in two ways: Ad\-hoc promptingPrompts LLMs manually, with a custom instruction per call\.Examples- Asking a coding agent to implement a new feature\. - Asking a chatbot to draft an email\. Industrial promptingPrompts LLMs as part of an automated process, with the same instruction across calls\.Examples- Extracting structured information from invoices as part of an accounting pipeline\. - Translating documentation pages into other languages as part of a release process\. Industrial prompting typically processes someInputdata with anInstructionto produce someOutput\. Whenever theInputgets updated, industrial prompting naively re\-runs theInstructionon theUpdatedInputto produce theUpdatedOutput\. This approach has two downsides: Lack of consistencyLLMs are non\-deterministic, so theUpdatedOutputdiffers from the originalOutputbeyond what theUpdatedInputnecessitates\.Full processing time and token costsAlthough only parts of the input have changed, we produce theUpdatedOutputin full\. This is as expensive as the production of the originalOutput\.*Revision prompting*resolves both downsides of naive re\-runs by operating on the input and output**revisions**instead of the full input and output\. ## HowRevision Prompting works Assume you have processed someInputwith anInstructionto produce someOutputby prompting an LLM with Instruction:Input Now,Inputhas been updated, and you also want to process theUpdatedInput\. Revision Prompting processes theUpdatedInputby constructing theRevisionPromptas Instruction:InputproducedOutput\. The input got updated as follows:`diff\(Input, UpdatedInput\)`\. Please produce a patch to update the output\. The LLM responds to theRevisionPromptwith theOutputPatchthat we apply to theOutputto obtain theUpdatedOutput\. ExampleYou translate the product page of an e\-bike to German with the prompt InstructionTranslate to German:Input`The Vela 3 e\-bike has a range of 80 km\.Its battery recharges in three hours\.The frame is made from recycled aluminium\.Every Vela 3 includes a two\-year warranty\.` and the LLM produces Output`Das E\-Bike Vela 3 hat eine Reichweite von 80 km\.Sein Akku lädt in drei Stunden auf\.Der Rahmen besteht aus recyceltem Aluminium\.Jedes Vela 3 hat zwei Jahre Garantie\.` Later, a battery upgrade increases the range from 80 km to 100 km\. Instead of re\-translating the whole page, you prompt InstructionTranslate to German:Input`The Vela 3 e\-bike has a range of 80 km\.Its battery recharges in three hours\.The frame is made from recycled aluminium\.Every Vela 3 includes a two\-year warranty\.`producedOutput`Das E\-Bike Vela 3 hat eine Reichweite von 80 km\.Sein Akku lädt in drei Stunden auf\.Der Rahmen besteht aus recyceltem Aluminium\.Jedes Vela 3 hat zwei Jahre Garantie\.`The input got updated as follows:diff\(Input, UpdatedInput\)\- The Vela 3 e\-bike has a range of80km\.\+ The Vela 3 e\-bike has a range of100km\.Please produce a patch to update the output\. The LLM responds with OutputPatch\- Das E\-Bike Vela 3 hat eine Reichweite von80km\.\+ Das E\-Bike Vela 3 hat eine Reichweite von100km\. Applying theOutputPatchto the originalOutputproduces the updated translation\. TheOutputPatchcontains only two lines of text instead of a full re\-translation\. Unchanged content stays consistent with the original translation\. ## WhyRevision Prompting works ConsistencyBy supplying the LLM with the input revision`diff\(Input, UpdatedInput\)`, we ensure that theOutputPatchis limited to what the input changes necessitate\. Everything not touched by theOutputPatchremains identical to the originalOutput\. Therefore, theUpdatedOutputis**consistent**with the originalOutput\.Time & cost savingsRevision Prompting feeds the originalOutputback in as part of the prompt, so the LLM only generates the shortOutputPatch\. Most tokens thereby move from the output to the input\. Since processing time scales roughly with the length of the output, this eliminates most of the processing time\. It also converts most of the output token cost into much cheaper input token cost\. If the re\-run happens within a couple of minutes of the original run, prompt caching reduces part of the input token cost as well\.## Revision Prompting in practice Revision FormatsThe ideal formats for encoding`diff\(Input, UpdatedInput\)`and theOutputPatchdepend on theInstruction\. The[POSIX`diff`utility](https://pubs.opengroup.org/onlinepubs/9799919799/utilities/diff.html)is a useful generic format\. For JSON outputs, the[JSON Patch](https://jsonpatch.com/)format works well\.Expected SavingsThe time & cost reduction scales with the size of the input changes and the sensitivity of the output to changes in the input\. In our own industrial prompts, Revision Prompting reduces time by ~80%, and costs by ~65%\.