CodeMirror 6: How to set up collaborative editing with clients and server?

I’m trying to build a collaborative wiki-like functionality for a web-app using CodeMirror 6. However, I have a bit of a hard time understanding how to use the collab extension.

I’ve got the editor set up on the client side, and passed in a collab extension created with collabExt = collab({startVersion: wikiPageVersionFromServer, clientId: userName}). I listen to editor events and try to retrieve changes to send to the server by adding an extension along the lines of:

listenerExtension = EditorView.updateListener.of(editorUpdate => {
         const updates = sendableUpdates(this.editorState)
         this.sendUpdatesToServer(updates)
})

And I pass in the wiki page text from the server and the extensions when creating the editor state: EditorState.create({doc: wikiPageText, extensions: [collabExt, listenerExtension, otherExtensions]}).

However, the sendableUpdates function seems to always return an empty array in the listener code above, even when editorUpdate contains the characters typed in the editor (verified by logging them).

In addition, there were some other parts of the collab API that I didn’t quite understand, despite trying to glance at the source. My questions are:

  • How and when is sendableUpdates supposed to be used?
  • The receiveUpdates(state: EditorState, updates: readonly Update[], ownUpdateCount: number) → Transaction function from the collab package takes an ownUpdateCount parameter, what should be passed in here? It’s not the document version number, is it? From looking at the code, it seems to be used to skip entries in the update array?
  • Is the following general understanding of using the collab API correct?:
    • The clients should listen to changes in their editors and use sendableUpdates to get a list of updates,
    • then send the updates to the server along with the version number retrieved with getSyncedVersion,
    • the server checks that the version number matches with its latest one, applies the updates to a Text object it keeps, and increases its version number by the number of updates,
    • then the server sends the updates to other clients (but not the client where they originated?), who use receiveUpdates to apply the list of updates to their editor states.
    • (In case the server had a higher version number, it rejects the update and tells the client, which applies the latest changes from the server with receiveUpdates and sends the old updates plus any new updates retrieved with sendableUpdates to the server along with it’s updated getSyncedVersion)

In general I found the CodeMirror 6 architecture very elegant and nice to work with, despite a bit of a learning curve. It would be very nice if using the collab extension was documented in more detail, either in the guide or in an example, as I believe it’s a great strength of the library (that’s why I ended up using CodeMirror 6 and not trying to implement OT on my own).

Pretty much as you showed, assuming this.editorState has been updated by the time that handler runs. Though you’ll probably want to include some logic in there to avoid spamming the server with concurrent requests containing the same updates on every transaction (since it’s likely that there is still a request pending when the next transaction happens).

You’ll probably want to track some kind of connection state (idle, pending updates, sending) to manage this.

The number of updates at the start of the updates array that originate from this client. The server will have to track the client id for each update so that it can provide this information.

Also the client where they originated. To be able to robustly deal with connection issues when sending steps, clients only ‘confirm’ their own steps when they receive them back from the server.

Thanks for the answers. I did figure out my problem - the EditorState passed to sendableUpdates was indeed not the current one.

Next question: Is there some built-in way to serialize and de-serialize a ChangeSet so that it can easily be sent between servers and clients?

Serializing and de-serializing it to JSON is not enough, as we need to instantiate the ChangeSet on the receiving end.

I did try to use a serialization library with a whitelist of classes that it is allowed to de-serialize and instantiate (I don’t want to allow the server to instantiate arbitrary classes with arbitrary data from untrusted clients, that way lies security issues), but that ran into the problem that ChangeSet uses Text objects internally, that store their data in TextLeaf objects, and they are not publicly visible, so I can’t register them as whitelisted with the serialization library (and there could probably be other internal classes than TextLeaf as well).

Oh, good point, I forgot to add JSON functionality to ChangeSet. This patch adds it.

Thanks, got it working now!

In the end I sidestepped the collab extension, and used a system similar to the split view example (except sending the ChangeSets over the network), using the map function on the ChangeSets as necessary and using the EditorView.dispatch() function to apply the change sets.

However, I ran into a problem with exceptions from the view when testing with simulated network delay and several clients:

Uncaught DOMException: Failed to execute ‘collapse’ on ‘Selection’: The offset 74 is larger than the node’s length (16).
at eval (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3085:28)
at DOMObserver.ignore (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4269:20)
at DocView.updateSelection (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3066:32)
at DocView.update (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:2945:18)
at EditorView.measure (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4783:30)
at EditorView.readMeasured (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4849:18)
at EditorView.posAtCoords (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4979:14)
at queryPos (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3963:20)
at Object.get (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3995:30)
at MouseSelection.select (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3793:36)

It seems to happen when doing a deletion at the same time as the document receives a deletion from the server (I was selecting a long text with the mouse and replacing the selection with a few letters, in rapid sequence on each client that had their network updates throttled to once every 5 seconds).

Am I just using the library wrong (perhaps I should use EditorView.update() instead of EditorView.dispatch() to apply changes?), or should I file a bug report for this?

If you rolled your own collab extension replacement, the problem is likely in your code.