Introducing the Heretic Grimoire: The takedown-resilient, local-first backup system that keeps uncensored models available forever

Welcome to another episode of THE HERETIC SHOW, where authoritarian dreams are destroyed by unreasonably effective linear algebra! Let's start with an important announcement: ## Heretic now has an official website at https://heretic-project.org This website contains: * Links to all official resources associated with the Heretic project * A complete tutorial for using Heretic * Detailed installation instructions with multiple redundant installation sources * Searchable documentation for every configuration parameter There is no guarantee that platforms like GitHub and Hugging Face will continue to host Heretic resources in the future, so I recommend bookmarking this website as it will always point to wherever the individual project resources are currently located.   But now to the main event. As you may have noticed, hostility towards local LLMs is growing everywhere, and this is especially true for decensored models like those created by Heretic. Already the project has been targeted with a legal notice from Meta, and demonized in mainstream media publications. Unfortunately, the AI world remains dependent on a massive single point of failure for model hosting, which is very difficult to replace because LLMs are *huge*. What if that single point of failure actually fails one day, for one reason or another? What if, in order to obtain Heretic models, you can't simply visit Hugging Face anymore? What if tens of thousands of hours invested by the community to create those models simply vanish? This existential risk has been worrying me for some time, and after several months of cumulative work, I am happy to announce that we now have a solution: *Everyone simply downloads all Heretic models to their own system!* That way, if the original model is deleted, you still have a local copy. Easy, right? Now you're probably thinking that this is a silly joke. Well, here's the punchline: **Those models are just 9 kilobytes each,** so you can store thousands of them on your phone without even noticing. ## The Heretic Grimoire In Heretic 1.3, we introduced reproducible models. When uploading an abliterated model to Hugging Face with Heretic, you can now choose to include reproducibility information, which will be stored in the model repository in human-readable form. But there is also a machine-readable file named `reproduce.json` that contains all information needed to reproduce the model. That file is like a spell in a grimoire, allowing you to summon not a demonic entity, but the very same model it belongs to. It's the entire model in a 9 kb text file. **Heretic 1.4**, released today, contains comprehensive functionality for working with these files, a system I call the Heretic Grimoire. Here's how it works: First, make sure you actually have the latest Heretic version, which is required to use these features: pip install -U heretic-llm Now you can fetch all `reproduce.json` files from publicly available Heretic models on Hugging Face, and store them in a directory of your choice (in this case, `my_grimoire`): heretic --collect-reproducibles my_grimoire You now have a local backup of all reproducible Heretic models, properly catalogued. To update this collection, simply run the command again. It functions as an append-only backup, never deleting files even if the corresponding model no longer exists on Hugging Face. To restore one of those models, simply run heretic --reproduce path/to/reproduce.json Heretic will guide you through the process, checking your environment against the one that was used to create the model, and pointing out potentially problematic mismatches. The multi-hour computations that were required to make the original model do **not** have to be re-done, and the entire process typically takes around a minute. After you have exported the resulting model, Heretic will verify the hashes of the weight files against those stored in the reproduction manifest (they may or may not be identical, depending on how closely your system resembles the original one). That's it! While the Grimoire system is designed from the ground up as a local backup, you can also see a complete list of reproducible models, updated twice daily, on [**this beautiful app**](https://huggingface.co/spaces/heretic-org/Heretic-Grimoire) created by long-time Heretic contributor Vinay Umrethe, who also implemented the first part of the reproducibility system. Even today, this app already preserves no less than 10 models that have since been removed from Hugging Face, allowing them to be recreated at will. The 1.4 release also contains several other important improvements and bug fixes, which you can find in the [release notes](https://github.com/p-e-w/heretic/releases/tag/v1.4.0). Perhaps most notably, you can now choose to export a LoRA instead of the full model, which provides another path to cheap model storage, and opens interesting possibilities such as merging manually with non-standard weights.   ## Heretic releases on IPFS Over the past two months, the Heretic project has gradually embraced decentralized and federated infrastructure. We now have a Matrix space, redundant Git hosting, and every Heretic release is now available over IPFS, enabling decentralized retrieval of the release archives and their signatures. The CIDs are: | Filename | CID | | ------------------------------- | ------------------------------------------------------------- | | heretic-1.4.0.zip | `bafybeiaqxqjdtkkrqeamnkjudvxlnrj7mululk3ipiafcyfhp2i3chbnue` | | heretic-1.4.0.zip.sigstore.json | `bafkreidhxgotlfko23bajxbcoruljpt7wkuytew7fjuglotjpr3cm7bwi4` | | heretic-1.3.0.zip | `bafybeianhsrnlkxdf5btyvgsaahqkhurmrowkuk4ymddz37wcnxz7gjxoe` | | heretic-1.3.0.zip.sigstore.json | `bafkreiflkjpyazath4n4lhoi67rvgds4k3spcsqjloeby4uj2cs232s6ui` | | heretic-1.2.0.zip | `bafybeifxnfy6tkakofe5ktlmeayk6edhja6neuv37bldimiq76dncicqqa` | | heretic-1.2.0.zip.sigstore.json | `bafkreiaz64yklnigwrgq63ibt5udpaupe3blqposfjdzkcytdf2whrly6q` | | heretic-1.1.0.zip | `bafkreibf3anxagvlhuvlsbbix5apc2jf2azz76lhuh27dyuzvc6ptiseka` | | heretic-1.1.0.zip.sigstore.json | `bafkreiapgtrl6qyybalmswzfz7dm2a7a4svsjs2sg5svm2orua5druafty` | | heretic-1.0.1.zip | `bafkreiag3mlkc76bhwcudhm7osqxdhmvywmc4kncdbc5ajtnd7tih4ftem` | | heretic-1.0.1.zip.sigstore.json | `bafkreibmtnfu2mtri3jcpewod3b2xj25xlo6xo4gyp7t3jyw5ttwmwubae` | See https://heretic-project.org/security for how to verify signatures. And if you happen to run an IPFS node, please pin these files (they're just a few hundreds kilobytes each) to help keep them available for everyone! Cheers :)