CodeMirror 6: How to set up collaborative editing with clients and server?

1

I’m trying to build a collaborative wiki-like functionality for a web-app using CodeMirror 6. However, I have a bit of a hard time understanding how to use the collab extension.

I’ve got the editor set up on the client side, and passed in a collab extension created with collabExt = collab({startVersion: wikiPageVersionFromServer, clientId: userName}). I listen to editor events and try to retrieve changes to send to the server by adding an extension along the lines of:

listenerExtension = EditorView.updateListener.of(editorUpdate => {
         const updates = sendableUpdates(this.editorState)
         this.sendUpdatesToServer(updates)
})

And I pass in the wiki page text from the server and the extensions when creating the editor state: EditorState.create({doc: wikiPageText, extensions: [collabExt, listenerExtension, otherExtensions]}).

However, the sendableUpdates function seems to always return an empty array in the listener code above, even when editorUpdate contains the characters typed in the editor (verified by logging them).

In addition, there were some other parts of the collab API that I didn’t quite understand, despite trying to glance at the source. My questions are:

  • How and when is sendableUpdates supposed to be used?
  • The receiveUpdates(state: EditorState, updates: readonly Update[], ownUpdateCount: number) → Transaction function from the collab package takes an ownUpdateCount parameter, what should be passed in here? It’s not the document version number, is it? From looking at the code, it seems to be used to skip entries in the update array?
  • Is the following general understanding of using the collab API correct?:
    • The clients should listen to changes in their editors and use sendableUpdates to get a list of updates,
    • then send the updates to the server along with the version number retrieved with getSyncedVersion,
    • the server checks that the version number matches with its latest one, applies the updates to a Text object it keeps, and increases its version number by the number of updates,
    • then the server sends the updates to other clients (but not the client where they originated?), who use receiveUpdates to apply the list of updates to their editor states.
    • (In case the server had a higher version number, it rejects the update and tells the client, which applies the latest changes from the server with receiveUpdates and sends the old updates plus any new updates retrieved with sendableUpdates to the server along with it’s updated getSyncedVersion)

In general I found the CodeMirror 6 architecture very elegant and nice to work with, despite a bit of a learning curve. It would be very nice if using the collab extension was documented in more detail, either in the guide or in an example, as I believe it’s a great strength of the library (that’s why I ended up using CodeMirror 6 and not trying to implement OT on my own).

2

Pretty much as you showed, assuming this.editorState has been updated by the time that handler runs. Though you’ll probably want to include some logic in there to avoid spamming the server with concurrent requests containing the same updates on every transaction (since it’s likely that there is still a request pending when the next transaction happens).

You’ll probably want to track some kind of connection state (idle, pending updates, sending) to manage this.

The number of updates at the start of the updates array that originate from this client. The server will have to track the client id for each update so that it can provide this information.

Also the client where they originated. To be able to robustly deal with connection issues when sending steps, clients only ‘confirm’ their own steps when they receive them back from the server.

3

Thanks for the answers. I did figure out my problem - the EditorState passed to sendableUpdates was indeed not the current one.

Next question: Is there some built-in way to serialize and de-serialize a ChangeSet so that it can easily be sent between servers and clients?

Serializing and de-serializing it to JSON is not enough, as we need to instantiate the ChangeSet on the receiving end.

I did try to use a serialization library with a whitelist of classes that it is allowed to de-serialize and instantiate (I don’t want to allow the server to instantiate arbitrary classes with arbitrary data from untrusted clients, that way lies security issues), but that ran into the problem that ChangeSet uses Text objects internally, that store their data in TextLeaf objects, and they are not publicly visible, so I can’t register them as whitelisted with the serialization library (and there could probably be other internal classes than TextLeaf as well).

4

Oh, good point, I forgot to add JSON functionality to ChangeSet. This patch adds it.

1 Like
5

Thanks, got it working now!

In the end I sidestepped the collab extension, and used a system similar to the split view example (except sending the ChangeSets over the network), using the map function on the ChangeSets as necessary and using the EditorView.dispatch() function to apply the change sets.

However, I ran into a problem with exceptions from the view when testing with simulated network delay and several clients:

Uncaught DOMException: Failed to execute ‘collapse’ on ‘Selection’: The offset 74 is larger than the node’s length (16).
at eval (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3085:28)
at DOMObserver.ignore (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4269:20)
at DocView.updateSelection (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3066:32)
at DocView.update (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:2945:18)
at EditorView.measure (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4783:30)
at EditorView.readMeasured (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4849:18)
at EditorView.posAtCoords (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:4979:14)
at queryPos (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3963:20)
at Object.get (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3995:30)
at MouseSelection.select (webpack-internal:///./node_modules/@codemirror/next/view/dist/index.js:3793:36)

It seems to happen when doing a deletion at the same time as the document receives a deletion from the server (I was selecting a long text with the mouse and replacing the selection with a few letters, in rapid sequence on each client that had their network updates throttled to once every 5 seconds).

Am I just using the library wrong (perhaps I should use EditorView.update() instead of EditorView.dispatch() to apply changes?), or should I file a bug report for this?

6

If you rolled your own collab extension replacement, the problem is likely in your code.

8

(reposting to rephrase)
Hi, I’m trying to get this to work too, using the collab module,
overriding the view dispatch method and de/serializing the ChangeSet
(omitting ownUpdateCount since all received updates are external):

./client.js

import {serializechanges} from "./utils.js";
var view = new EditorView({
  state,
  dispatch:function(transaction) {
    view.update([transaction]);
    if (transaction.changes.empty) return;
    let updates = sendableUpdates(view.viewState.state);
      .map(serializechanges)
      .map((changes,index) => ({...updates[index],changes}));
    socket.emit("save",{updates})
  }
});
socket.on("save", function({updates}) {
  updates.map(serializechanges).forEach((changes,index) =>
    Object.assign(updates[index], {changes}));
  view.update([receiveUpdates(view.viewState.state,updates)]); 
}

./socket.js

socket.on("save", async function({updates}) {
  let {serializechanges} = await import("./utils.js");
  let received = updates.map(serializechanges)
    .map((changes,index)=>({...updates[index], changes}));
  socket.room.state.update([receiveUpdates(socket.room.state,received)]);
  socket.server.sockets.emit("save",{updates})
});

./utils.js

export var serializechanges = ({changes}) =>
   Array.isArray(changes) ? ChangeSet.fromJSON(changes) : changes.toJSON();

but the lines with update([receiveUpdates(...)])
leave the server-side state unchanged (as read by state.doc.toString()),
and the peer-side view throwing this to updates after a first successful one:
RangeError: Applying change set to a document with the wrong length
at ChangeSet.apply (codemirror_rollup.js:1936),
because sendableUpdates() keeps containing
every previous update -
besides repeating the changes on the same client
instead of “confirming” them (although I can avoid
this one by not emitting back there).
What am I missing to “apply” transactions to a standalone state object,
and for the client’s state to move along with the visually successful update?
Thanks for the help!

9

Sorry, that’s too much code for me to debug it for you. I already went ahead and formatted it, since in its original shape it was entirely unreadable, but it’s still confusing me.

We’ll need a more complete example for a collaborative setup in the docs at some point, but it may be a while until I get to that, so for now I can only recommend you read the existing guide thoroughly and maybe try using typescript to catch obvious errors like the way you’re not passing the third argument to receiveUpdates.

10

@marijn thanks for the awesome codemirror 6, I already love it.

I had a collaborative codemirror 5, and I guess it’s time to update now.
Please let me know if I can help somehow with the documentation.

Where do I insert coins? .) …

11

An example that shows collaborative editing is definitely something that I should prioritize, since it’s not trivial to set up. Will put it on my list!

3 Likes
12

woud be awesome to give a hint in this thread/place when doc and/or example could be looked in. Many thanks!

13

I think I have the collab module working. Here’s some (partial) example code and some refelections.

Context

I wanted to get CM6 talking to an existing OT system that has its own OT operations (based on those from ot.js). I have written a CodeMirrorOtAdapter class that does this using the collab module. I am not sure if this ‘tie into existing system’ use case was a focus for the module, but I imagine we won’t be the only ones who try to do this, so I hope it’s helpful to see this use case.

Parts of the adapter are specific to our (overleaf.com) set up, and I’ve mostly snipped those out here to focus on the bits that are more likely to be generally useful.

Disclaimer: this is still just a prototype — it passes basic tests, but much testing remains to be done.

Imports

I haven’t yet upgraded this prototype to CM 0.17; it is still on 0.16, but I don’t think there were any material changes to the collab module (just some imports that need changing). Here they are for 0.16.

import {
  EditorState,
  EditorView,
  basicSetup
} from '@codemirror/next/basic-setup'
import {
  collab,
  receiveUpdates,
  sendableUpdates
} from '@codemirror/next/collab'
import { history } from '@codemirror/next/history'

Sending and Receiving Updates

class CodeMirrorOtAdapter {
  constructor(otClient) {
    this.otClient = otClient // our existing OT implementation
    this.editorView = null // gets set later

    // The sendableUpdates contain all unconfirmed updates every time
    // the listener fires. This count lets us ignore the ones we've
    // already seen and sent to the server.
    this.sendableUpdateCount = 0

    this.updateListenerExtension = EditorView.updateListener.of(
      editorUpdate => {
        const listenerUpdates = sendableUpdates(editorUpdate.state)
        while (this.sendableUpdateCount < listenerUpdates.length) {
          // We have a new update!
          const update = listenerUpdates[this.sendableUpdateCount]
          ++this.sendableUpdateCount

          // Convert it from CM format to our own format...
          const textOperation = convertChangeSetToTextOperation(update.changes)
          // [snip] submit our operation to the server via this.otClient

          // Note: our otClient handles batching changes and may compose some
          // of them together so we don't spam the server with every keystroke
          // when the user is typing fast, but it also remembers the number
          // of uncomposed operations (updates) that we saw here so we can
          // find ownUpdateCount later.
        }
      }
    )

    // Note: our existing otClient already works out whether changes
    // are local or remote based on clientIds, which is similar to the
    // approach suggested by this module.
    this._handleLocalChange = this._handleLocalChange.bind(this)
    this._handleRemoteChange = this._handleRemoteChange.bind(this)
    this.otClient.onLocalChange(this._handleLocalChange)
    this.otClient.onRemoteChange(this._handleRemoteChange)
  }

  _handleLocalChange({ uncomposedOperationCount }) {
    const ownUpdateCount = uncomposedOperationCount
    this.editorView.dispatch(
      receiveUpdates(this.editorView.state, [], ownUpdateCount)
    )
    this.sendableUpdateCount -= ownUpdateCount
    if (this.sendableUpdateCount < 0) throw new Error('too many ownUpdates')
  }

  _handleRemoteChange({ change }) {
    const remoteUpdates = []
    for (const operation of change.getOperations()) {
      // [snip] various other conversion stuff
      remoteUpdates.push({
        changes: convertTextOperationToChangeSet(textOperation)
      })
    }
    this.editorView.dispatch(
      receiveUpdates(this.editorView.state, remoteUpdates, 0)
    )
  }

Editor Setup

The editor gets created in a React component.

function Editor({ pathname }) {
  // [snip] lots of setup stuff to get `initialContent`
  useEffect(() => {
    const adapter = new CodeMirrorOtAdapter(otClient)

    const editorState = EditorState.create({
      doc: initialContent,
      extensions: [
        basicSetup,
        history(),
        collab({
          // Note: otClient already has its own clientId tracking; setting it
          // here doesn't actually matter AFAICT, but it doesn't hurt.
          clientId: otClient.clientId
        }),
        adapter.updateListenerExtension
      ]
    })

    const editorView = new EditorView({
      state: editorState,
      parent: container.current
    })
    adapter.editorView = editorView

    return () => {
      editorView.destroy()
    }
  }, [ /* various stuff from context */ ])

  // [snip] actually rendering it
}

Reflections

Overall the collab module is very helpful. It’s awesome that undo just works, for example!

If the interface is still up for iteration, my feedback would be:

  • There are facets for version and clientId, but nothing in the module itself really depends on them. At least in our case, I think these more naturally sit outside of CodeMirror’s state, as part of the supporting system that also has to deal with websockets, websocket fallbacks, authentication and authorization, connection problems, etc…
  • I am not sure in how many situations the client will actually have an updates array to give the receiveUpdates method. An array of unconfirmed updates is certainly needed in order to handle concurrent updates, and CM maintains this internally in collabState.unconfirmed. The handling of the updates array in receiveUpdates seems to suggest that the client should keep its own copy of this data in sync with the internal unconfirmed updates (and add in the remote updates at the end). In my example, when local updates are confirmed by the authority, I pass an empty array for updates so the slice with ownUpdateCount is a no-op; this works but seems a bit confusing.

Getting this prototype working was a fun project and a good intro to the new CM! I should also apologise for my stubborn use of JavaScript instead of TypeScript. Hopefully I have not made any very bad blunders as a result.

1 Like
14

That’s a good point. Though receiveUpdates does roll version forward, I guess that might be straightforward enough to leave to external code as well.

Could it be that this just looked odd because you’re doing your own conversion on the updates? In a situation where you’re just exchanging updates, you’d just pass the sendableUpdates to the server and listen for new updates that it sends you, which you can pass directly to receiveUpdates. No need to keep an own copy. Or am I misunderstanding what you mean?

The idea is to not rely on the success of your confirm request to conclude that updates have gone through, since that might be a false negative (see “Two Generals Problem”), but always have the server send back any updates it received, and confirm precisely those updates with receiveUpdates that you get back.

15

OK, I think I see where you’re coming from now. It would indeed make more sense if our otClient were not already doing some of the same work upstream before I convert our operations to CM updates.

One more thought on this: I think it would be quite clean to split receiveUpdates into confirmUpdate, which would shift one from collabState.unconfirmed, and receiveUpdate, which would handle one remote update. Given that the caller has to loop through all the updates it receives from the authority to decide whether they are its own, it could fairly easily call confirmUpdate or receiveUpdate as the case might be for each one. One snag is that this approach would lose the composition step that receiveUpdates currently has, which I am guessing is a performance optimisation? (But I would guess n is small here.)

I’m afraid I’m still not sure exactly what you have in mind here, but I agree that handling unreliable connections is very important, and I’m sure that your example will make it clear.

(In particular, I can see how Two Generals applies here in a general sense, but there are several ways of setting up the protocol for the authority and client(s), and I think many of them break the symmetry that one ordinarily sees between the two generals in the Two Generals problem, so I am not quite sure how to look at it.)

16

just some comment coming from the world of pub/sub messaging…
a spilt into confirmUpdate and receiveUpdate sound a way to think about.
Thinks happens during connection is broken, maybe shouldn’t disappear, but could be send/re-ordered again when connection is established again.

17

Update: After some more testing, it turns out that the above approach is not sound, because our OT and CodeMirror’s OT resolve at least one class of conflicts differently.

Example: if we start with the string ab, and the client (say) deletes that text and the server inserts a c in the middle, the delete ‘wins’ in CM’s map:

const text = Text.of(['ab']) 
const c = ChangeSet.of({ from: 0, to: 2 }, 2)
const s = ChangeSet.of({ from: 1, insert: 'c' }, 2)
const cp = c.map(s, true)
const sp = s.map(c)
console.log(c.compose(sp).apply(text), s.compose(cp).apply(text))
// => TextLeaf { text: [ '' ], length: 0 } TextLeaf { text: [ '' ], length: 0 }

and the result for both client and server is an empty string. In our/ot.js OT, the insert ‘wins’, and the result for both client and server is c:

const text = 'ab'
const c = new TextOperation().remove(2)
const s = new TextOperation().retain(1).insert('c').retain(1)
const [cp, sp] = TextOperation.transform(c, s)
console.log(c.compose(sp).apply(text), s.compose(cp).apply(text))
// c c

So, I’m not sure there’s really anything to be done about this on the CodeMirror side, but here are a couple of ideas:

  • There may be an argument that the map operation should try to preserve the insert to avoid losing data, but there are usually compromises that must be made when defining these transform functions, and even if you changed this (at this relatively late stage…), there might well be other disagreements that I haven’t found yet that.

  • The collab extension could take a configurable map function that would default to the built-in map but allow other schemes? The actual mapping part of the collab extension is I think limited to these two lines, so this might not be crazy, but it does seem untidy. I think our implementation would have to convert the ChangeSet to our format (again), transform it, and convert it back to a ChangeSet, so it would also be fairly inefficient.

At the moment, it’s looking like we’ll unfortunately have to roll our own version of the collab extension to make this work with our OT (much more likely to be wrong!), or change our backend to support both our current TextOperations and the new ChangeSets (scary but still looking at this!).

18

I think if you’re not going to use the OT built into ChangeSet you will indeed have a hard time using the collab module, and setting up something separate is the way to go, yes.

19

There is now an example for collaborative editing on the website.

1 Like
20

awesome marijin,

many thanks for this!! This was the missing piece of info i (and several other) are looking for!

21

just an idea:
what do you think of “activating” some kind of code “highlighting” to show in the same example what’s CodeMirrow intend for (and cable) ?

Idea behind: i could imagine that this page will be shared to many parties, so it’s a good opportunity to show some more…