Troubleshooting article for CM6

Hey - I’m mostly suggesting this because I wanted to share my experience that I had with CM6 where I had significant, mysterious issues implementing it on my site. I figured this would be the sorta thing that would go into a troubleshooting section.

Anyways, the issue was that I was having bizarre errors where seemingly CodeMirror was treating various objects as, well, not what they were.

The exact expression that tipped me off was pretty much like this:

// pretend this TreeBuffer object was 
// found / instantiated somewhere earlier in the code
const treeBuffer: TreeBuffer; 
if (treeBuffer instanceof TreeBuffer ) { ... } // this is somehow false
else { ... } // code breaks here

// output: TreeBuffer

What was happening was that due to web bundling / dependency resolution, in my case with Snowpack, the externally loaded extension that I was creating was actually importing a wholly separate instance of CM6. That means that the treeBuffer object was not an instance of TreeBuffer, it was an instance of another TreeBuffer!

The way I fixed this, and I assume how you’d fix this in any other bundler, is that I forced Snowpack to deduplicate CodeMirror dependencies, e.g. CodeMirror itself and Lezer. (BTW: Snowpack, in its backend, uses Rollup to install modules, so really I told Rollup to deduplicate.)

The errors that I was getting did not make it immediately obvious whatsoever that this is what was happening, so I feel like an article documenting errors like this could be helpful. I’m not sure if any changes in how CodeMirror is authored would help here, so I’m not suggesting that.

Everything works pretty good now!

PS: Could we get a NPM package that basically re-exports all official CM6 languages? The reason I request this is that when you’re using CM6 for say, authoring Markdown that has embedded code blocks, you basically just want to import every language you can because it’s nice to have. When you do this however, your package.json has a giant @codemirror section filled with language dependencies. Having a package just kinda do this for me would be great, and if additional languages come out, it would automatically be made available whenever the dependency was updated in the project.

Sorta related: This is honestly really tempting me to hammer out a super-lightweight Prism highlighter for CM6, with the intended use-case being for things like comment boxes and really lightweight instances of CM6, with few extensions. I think I’d make it not even fake a syntax tree, it would just be a separate highlighting plugin. It doesn’t seem too difficult to do due to the range-shenanigans infrastructure set up in CM6, which is pretty cool.

1 Like

was actually importing a wholly separate instance of CM6.

We’ve had issues like that with ProseMirror as well—though they theoretically deduplicate packages when versions are compatible, several tools can still end up needlessly loading multiple instances of a package, and indeed, the failures caused by that are really painful to debug.

I’m not sure what to do about this, beyond removing all uses of instanceof (and probably some other constructs, like comparing with singletons) from the codebase, which seems impractical.

The @codemirror/language-data package exports a database of metadata about languages along with shims to dynamically load them (with dynamic import expressions). Bundling all of them statically seems wasteful (but it may be possible to configure a bundler to treat dynamic imports as if they are static, I’ve never tried).

This is definitely possible. Or just use the stream parsers in @codemirror/legacy-modes, which have the advantage of actually being incremental.

I absolutely wouldn’t change anything due to the failures of build tools. That’s why all I suggested was just to document this behavior somewhere. It’s a really easy fix anyways, at least with Rollup.

Well, this is basically what I wanted, and was already using - but I meant a NPM package that, as dependencies, included the official CM6 languages. That way you would not need to directly add all of those languages as dependencies so that language-data will work. They wouldn’t need to be bundled or anything, just made available to language-data or something akin to it. Languages would still be loaded async. It’s for cleaner package dependencies, basically. Maybe I’m just missing something?

EDIT: Okay, I wasn’t clear enough. I mean something exactly like language-data, with its list of LanguageDescription objects, except that it, as package dependencies, includes the languages it may attempt to dynamically import. Currently, importing a language from language-data requires you to have that language as a dependency somewhere, because language-data's dependencies don’t actually include the languages it may attempt to import.

Oh, the way I’d do it would be ‘incremental’ (more like heavily cached). The stream parser, I’ll be honest, is not my favorite. I prefer regex-madness highlighters, and the ‘hunting for state’ using a WeakMap has misbehaved on me, although that could just be my own fault. A benefit of porting Prism in a more direct fashion could be immediate compatibility with Prism themes, which is just a good incentive to use CM6 as it would support whatever Prism stuff you had before. I know Prism is used a lot with small editors.

An alternative would be to just spend some time making more Lezer grammars, I suppose. Although, I guess my hesitation with that is that if I make a proper grammar for a language I don’t want it to be mediocre because I only focused on getting pretty colors on screen.

Ah, right. language-data was supposed to have dependendencies on all of those, but it seems I messed up the script that generated the split-repitoritory package.json files. I’ll fix that later.

the ‘hunting for state’ using a WeakMap has misbehaved on me

This isn’t terribly mature code yet, so if you see it misbehave, please file a bug so that I can investigate.

1 Like

Just to add some perspective on the duplicated dependency issue: I’m running into a similar problem with third-party extensibility.

We develop a markdown editor app currently using CM5, and our app allows the end user to develop and run plugins as JS modules.

With CM5, it’s very easy to “drop in” an extension for CodeMirror. For example, if the user wants vim mode, they can just drop in vim.js.

I’d say it’s fairly hard for an end-user to interface with a CM6 in bundle, if that’s possible at all?

I think it’s a difficult design problem and it’s totally understandable that this might not be the target use-case for CM6.

Actually, a strength of CM6 IMO is that it can be massively reconfigured on the fly. Take a look at ReconfigurationSpec. You can dispatch a transaction that appends an extension to the current configuration. You can also reconfigure or replace existing extensions. I made a ‘Enable Spellcheck’ button using this method.

In terms of code duplication and extensions, you just have to be very careful and ensure that any extensions that users make import CM6 objects from the correct source. I’m sure you could do this even if CM6 were to be bundled - but idk exactly what that would entail.

Yes, thank you for the tip. I think I got a good grasp on how to provide configurability as the maker of an app. My situation is a bit different in that we provide a set of CM6 extensions out-of-the-box that’s properly bundled with the various pieces of CM6 into a single deduplicated rolled-up .js file, but we also want to provide a way for the users of the app to tap into the editor and augment it as they see fit.

This can be easily done in CM5 because there was a global CodeMirror object, as well as all the internals of the entire editor can be accessed from properties of the instance object.

In CM6, many things require a reference to module packages. For example, if one were to access the userEvent annotation of a transaction, the code would be:

import {Transaction} from '@codemirror/state';
// ...
const userEvent = transaction.annotation(Transaction.userEvent);

However, the plugin author would have to import the correct Transaction from the bundled js package inside the application, rather than a copy they pulled from their own npm/package.json (which would be a duplicate copy).

To further complicate the matter, it’s possible that a plugin author would want to use some functionality in CM6’s repo that isn’t already bundled with the app. For example, we may only provide @codemirror/lang-markdown but someone who wishes to add javascript highlighting would have to import @codemirror/lang-javascript.

This package depends on 6 other packages, some of which are bundled, some aren’t: @codemirror/autocomplete, @codemirror/highlight, @codemirror/language, @codemirror/lint, @codemirror/state, @codemirror/view.

Now we’re in the same situation as described by:

Yes, good point about this type of ad-hoc extension being much harder with a bundler. But indeed, I don’t consider it an important enough use case to offset the advantages of having a real module system.

Ok I think that’s fair.

If anyone else runs into this issue, my proposed workaround would be to maintain a list of CM6 packages as a first party app maker, and ask third party plugin authors to include that list of as ‘external’ packages in their bundler so they won’t include a duplicate copy. Then it’s just a matter of exporting them correctly via require. It’s still tedious to maintain such a list and painful to make backward-compatible changes like adding or removing CM6 packages, but at least it’s doable :stuck_out_tongue_closed_eyes: