CodeMirror 6: Minimal CM-5-style highlighting tokenizer

If my goal is to achieve minimal token-based highlighting in CodeMirror 6, what the best way to do it? Should I create a Lezer grammar where root is basically a stream of tokens, or is there a simpler way without using Lezer at all?

To give some context, I want to highlight a language where the grammar is too complicated to be worth a full parsing effort, and very basic (CM-5-like) token highlighting is more than sufficient.

You can use the stream-syntax package to write a highlighter in CM5-style, or do a simplified grammar as you describe. Both should work well. The Lezer approach will probably take less code and run a bit faster.

1 Like

Thanks, that makes sense. Found stream-syntax here, but not on npm?
https://github.com/codemirror/codemirror.next/tree/master/stream-syntax

Edit: nvm it’s part of @codemirror/next which makes sense.

Either way I’ll just do simple Lezer grammar, seems straightforward enough.