Wordgard Release 0.1

# Wordgard Release 0.1 Source: [https://marijnhaverbeke.nl/blog/wordgard-0.1.html](https://marijnhaverbeke.nl/blog/wordgard-0.1.html) I am happy to announce that my latest project, which I've been talking about for years, is now out with a first release\. The project is called[Wordgard](https://wordgard.net/)\. It is a new iteration of a[ProseMirror](https://prosemirror.net/)\-style rich text editor system, integrating the things I've learned since stabilizing ProseMirror, nine years ago\. The architecture also takes a lot of inspiration from the version 6 redesign of the[CodeMirror](https://codemirror.net/)text editor\. Wordgard is \(once again\) a JavaScript library that uses the browser DOM to display its editor interface\. It is licensed under an MIT license\. The code is available on[my Forgejo server](https://code.haverbeke.berlin/wordgard/)\. It's a little concerning how I keep implementing new editors over and over \(by my count this is the 6th non\-trivial one\)\. But doing this somehow hasn't lost its charm yet\. It still feels like the designs get better every iteration\. I don't expect I'll find editor\-architecture nirvana before I retire, but I'm sure I'm getting closer to it\. ## Motivation I'm still proud of ProseMirror, and ProseMirror isn't going anywhere—it will continue to be maintained\. But there are parts of its design that make me wince every time I have to work with them, because at this point I know that I should have done them differently\. Instead of trying to change ProseMirror to incorporate these new insights, I have chosen to create a completely new system with a new name\. A ProseMirror 2\.0 with an incompatible interface would amount to the same but make it ambiguous what people mean when referring to ProseMirror\. Trying to graft stuff on in a backwards\-compatible way as an 1\.x version would produce a compromised win32\-style mess\. I'm not all that fond of the*ProseMirror*pun anymore either \(it's CodeMirror but for prose, get it?\) So: green field full rewrite\! You'll find a lot of ideas from ProseMirror in Wordgard, but the programming interface is built from scratch, without concern for compatibility\. Let's look at the parts of ProseMirror that I think I improved on\. ### Stop Doing Steps “Make sure you compensate for the document shift caused by the first step when adding a second\.” “To figure out what range of the document was replaced, you have to iterate through the sequence of steps in both directions, mapping positions in the new document forward and positions in the old document backward\.” “Yes, I'll have one replace\-around\-step please\.” — statements dreamed up by the utterly deranged\. ProseMirror change representation was designed by a person who was very much occupied with the problem of preserving semantic meaning for changes even if the changes were transformed, but who also didn't have a lot of experience with change formats\. Steps break down changes into atomic parts that each do a single clear thing\. A given editor update might involve any number of them, each defined to act on the document produced by the one before it\. They serve their purpose, but they are seriously awkward to work with\. Wordgard uses a much simpler but arguably more powerful system based on my experience with CodeMirror's change representation, which derives from the old “delta” format from ShareJS\. In CodeMirror, a change is a sequence of sections, each of which either preserves a part of the old document, or replaces it with a piece of new content\. So in a document of length 10, inserting an L at 4 is represented`\[keep 4\] \[replace 0 with "L"\] \[keep 6\]`, and deleting the first two characters would be`\[replace 2 with ""\] \[keep 8\]`\. Wordgard extends this with modification sections, which preserve the structure of a section, but add or remove marks to it \(which are things like emphasis, link style, or image alt text\)\. Making the word from 3 to 6 bold would be represented as`\[keep 3\] \[update 3 \+bold\] \[keep 4\]`\. Of course, unlike CodeMirror's plain text, rich text content isn't just a flat string\. Because Wordgard uses a token\-counting indexing system for document positions \(the same system ProseMirror uses\) the change format can address the document as a flat sequence of tokens \(node open and close tokens, and leaf tokens\), into which it splices new sequences of tokens\. These types of changes can easily be combined, so that a single transaction always has a single change associated with it, which is easy to inspect and reason about\. They also support a limited form of operational transformation, making it possible to merge a bunch of changes that are all described in terms of the start document\. That gives us an ergonomic way of describing transactions with multiple changes and makes it possible to implement collaborative editing and an undo history that supports undoing some changes but not others\. But the document is not*really*a flat sequence of tokens\. Those tokens only make sense if they combine to form a well\-formed tree\. If you delete a node closing token, for example, the tokens aren't balanced anymore, and you'll have created a change that cannot be applied\. Thus the code that creates change sets has to be able to do some things that weren't necessary in the plain\-text version—checking and correcting changes to make sure they produce a valid document structure\. This issue also comes up in the handling of operational transformations\. If you reinterpret a step so that it can be applied after another step, that transformation must also preserve the property of not making the document invalid\. And if you need applying transformed step A after B to yield the same document as applying transformed B after A \(which you do generally need, for this to be any use\), it gets more subtle\. What Wordgard's change model does is, when transforming changes, to derive a fix\-up change that corrects the result of the combined steps in such a way that it produces the same fix for A\-over\-B that it produces for B\-over\-A \(by being careful about what inputs the fix\-generation code gets\)\. If not fixed, both sequences would, due to the guaranteed provided by the transformation algorithm, produce the same possibly invalid document\. By composing them both with the same fix, they both produce the same valid document\. Most changes don't actually need a fix, but this approach ensures that those that do still converge\. ### Schema Composition Because ProseMirror document schemas specify relations between nodes in a rather direct way, setting them up generally needs to be done by hand\. Node and mark types live only within a given schema—you can share their definition object, or part of it, but there's no usable node identity between schemas\. Wordgard takes a different approach, making it so that node and mark types are their own independent thing, which may be part of different document schemas\. This makes those objects usable as typed, autocompletion\-supporting handles that stand in for node or mark types, and makes it easier to compose schemas by just throwing together the elements you need\. There are still situations where you need to change some relation between nodes or marks directly\. For that, a schema can override these relationships on existing elements\. The definition of the node or mark specifies its default content or target types, but a schema that wants to use the element differently can change those\. This makes it possible to do a lot more with the basic built\-in nodes, and thus makes it more viable to provide editing support extensions and system integration \(say, menu buttons\) directly for those nodes\. ProseMirror is plagued by the issue of schemas being so generic that it was hard to provide reusable functionality—code was either specific to a schema, or it just didn't know what any of the schema elements meant\. This works much better in Wordgard\. Another schema composition problem in ProseMirror was the way node attributes are defined\. Things like text alignment or alt text were defined directly in the target node, as a node attribute, strongly tying them to that node type\. This made it a chore to add something like alignment or text direction to your schema, because you'd have to add that attribute to every textblock node\. Generalizing marks to that they can be used for things like that has made adding such functionality in a modular way a lot easier\. The node types themselves don't even to know what marks are targeting them\. ### Content Constraints The ability to specify the allowed content for a given parent node using a regular expression is ProseMirror's signature feature\. Wordgard no longer supports this\. A node's content description can only constrain what type of children it supports, not their order\. There are a few different reasons for this\. The biggest one is that it is just too hard to write generic document\-manipulation code for ProseMirror\. If your code isn't written for a specific schema, it can assume almost nothing about what transformations are valid, and needs to check every single thing it does against the content constraints\. This gets so subtle and burdensome that I kept getting it wrong myself\. If the person who designed the system cannot use it, that's not a good sign\. Beyond that, I've become convinced that hard\-locking your document shape with these kinds of constraints will often be detrimental to the user experience\. Documents are edited through a series of small editing actions that add up to the intended shape\. If your editor won't allow users to make intermediate steps that leave the document in an odd shape, that will often frustrate them\. There are definitely situations where ProseMirror's content constraints worked really well\. But they often involved a lot of careful design and scripting, in order to make sure the editing experience was good\. Wordgard is going with a more relaxed system to encourage a less rigid approach to document shape\. For cases where you really do need to specify invariants beyond what the schema rules offer, it provides an abstraction called a “correction” which is a programmatic way to fix up document shapes that you don't want to allow\. These have the advantage that they are programs, and can thus correct the document in more intelligent and context\-aware way than what the content expression enforcement would do\. They also work for things that even ProseMirror's constraints couldn't express, such as making sure tables are rectangular\. ### Extension System Compared to what we ended up doing in CodeMirror 6, ProseMirror's extension system is rather crude\. You have plugins that can affect the system, and you can kind of influence plugin precedence by changing their order\. But because each plugin does a number of different things, you easily end up in a situation where you need a plugin to be low\-precedence for one of its hooks, but high\-precedence for another\. The CodeMirror system, based on[facets](https://marijnhaverbeke.nl/blog/facets.html), makes extensions a much more fine\-grained thing, and allows every extension value to set its own precedence category\. Facets are typed extension points, and can be defined by any code, not just the library itself\. So editor extensions can define their own extension points, which is useful surprisingly often\. Wordgard pretty much copies CodeMirror's system here, including its state update and reconfiguration mechanisms\. A configuration, in this system, is not an array of plugins, but a tree of extensions, each of which can do anything from defining an event handler to configuring the editor's attributes to adding a new piece of editor state\. A given feature implementation typically consist of a group of extensions that work together to produce the desired behavior\. You can, for the most part, just drop such extension bundles into your configuration without too much thought and have them work together well, because the primitives extensions build on have been defined in such a way that they compose cleanly\. ### Dependence on the Browser A lot of the issues people run into with ProseMirror are related to the way it delegates selection behavior to the browser's native implementation\. This approach isn't unreasonable\. Doing cursor motion properly, in bidirectional text and content styled in potentially weird ways, is hard\. The idea was to let the browser do it, see what it did, and update our own model of the selection to follow that\. Unfortunately, browsers did not do as good a job as hoped\. They'll refuse to move the cursor past some kinds of content, sometimes won't draw the cursor at all, other times draw it in the wrong place, and glitch out mouse selection drag gestures if you look at them funny\. So Wordgard bites the bullet and does almost all pointer and keyboard\-based selection itself\. This involved implementing bidirectional text handling \(which is useful to have for other purposes as well\), forming some kind of model of the way the content is laid out, and drawing the cursor ourselves\. The one area where this is unfortunately not possible is touch selection\. You can do an okay job reimplementing that, but doing so seems to irrevocably break the native context menu, which tends to be pretty hard to do without on phone and tablet systems\. So touch selection is native\. Fortunately, it tends to have less strange misbehavior than keyboard selection\. Browsers have come quite a way in consistent support for editing events \(notably`beforeinput`\) in the past 9 years\. Actual real\-world testing will have to show whether this is really tenable yet, but so far it looks like Wordgard can do without the trick that ProseMirror is built on, where it monitors the document DOM for changes and parses changed content to construct an actual document change\. Wordgard just handles`beforeinput`events for everything except composition text input\. That avoids the need for a whole class of messy workarounds\. ## Status Wordgard is a bit further along than my previous projects were at the time they were announced—its core interface supports almost everything I want it to support, and I've written a bunch of extensions to confirm that my design is practical\. The docs are still somewhat rough but things like the reference manual are complete and usable\. You can install it as`wordgard`from the npm registry, and read how to use it on the[website](https://wordgard.net/)\. That being said, it has been my experience that lots of issues won't come up until people are using a system for real work\. There's a bunch of things I still want to add to the system myself, and I hope that others will also start looking into it now that it's public\. Even though this isn't the first editor I've built, I'm sure further insights will require me to rethink parts of the public interface\. I'm releasing this first version as 0\.1 and will stay on 0\.x versions for a while \(likely at least a year\) to gather feedback, fix bugs, and help me clear up the rough corners of the system\. I am releasing the code under an MIT license, like I did with my earlier projects\. I seriously considered using a more restrictive licensing style, in order to have some more control over my work, but I realized that I'm mostly interested in this being widely used\. My working model has been one of abundance—providing enough value for the world at large that even if only a little of that value rolls back to me, it's a living\. I don't want to make it my business to create artificial scarcity, play gatekeeper, or enforce complicated licenses\. It's a shame that a permissive license will allow companies I don't like very much to use the software \(generally without even paying me\), but that's part of the package\. Speaking of companies I don't like—regardless of license, it is clear to me that the “AI” vultures will slurp up my code and all the hard\-won ideas it contains moments after I publish this\. I wish a pox upon them and their slop machines\. But I don't think there's a credible thing I can do about this\. For anyone to be able to run this system, the JavaScript code must be on the web\. For anyone to be able to figure out how to work with it, the docs must be public\. So, at least until courts decide that copyright actually does still mean something, this will be fed to all the big language models\. No language models were used in the creation of this software\. I design and build my systems by hand\. As a deviation from standard open\-source practice, I'm doing an experiment where I don't take pull requests for Wordgard\. I've realized that I find those the worst part of maintenance work\. Reviewing a big change and negotiating all the adjustments needed to make it fit my expectations is generally more work than implementing the change myself\. And now that the cost of generating code has gone down dramatically, the arrangement where people can dump code on me and I'm expected to review and maintain it \(or argue why I don't want to, which is also work\) has become even less attractive\.