Static highlighting using CM v6

v6
1

I want to highlight some pieces of code for a documentation of a new poor-supported language, it is alif. I coded a custom simple mode with @codemirror/legacy-modes/mode/simple-mode, everything is fine till now. Now I want to statically highlight alif code, I thought about using highlight.js, but it would be much easier to create a centralized place of writing highlighting modes. Using CM as a static highlighter will make development faster, if I edit alif simple modes and everything is updated.

Here is a related article, but v5 not CM v6 (next).

2

To implement a runmode-like functionality with CM6, use the @codemirror/highlight package like so

import {highlightTree, defaultHighlightStyle} from "@codemirror/highlight"
export function runmode(textContent: string, language: Language, callback: (text: string, style: string, from: number, to: number) => void, options?: Record<string, any>) {
  const tree = language.parseString(textContent);
  let pos = 0;
  highlightTree(tree, defaultHighlightStyle.match, (from, to, classes) => {
    from > pos && callback(textContent.slice(pos, from), null, pos, from);
    callback(textContent.slice(from, to), classes, from, to);
    pos = to;
  });
  pos != tree.length && callback(textContent.slice(pos, tree.length), null, pos, tree.length);
}

The reason why we need to keep a position state is because unstyled tokens are not emitted.

Note that in my experience, although dynamically loading languages is easier in CM6 because its given as es modules, I’ve found the overall bundle size required to replicate the runmode functionality larger because @codemirror/highlight depends on @codemirror/view and @codemirror/state but 1. that may be tree-shaking misconfiguration 2. if you’re highlighting in server, bundle size might not be a problem.

5 Likes
3

parseString is no longer available since @codemirror/language@0.19.
How can I do it in the latest version? Thanks!

4

Call language.parser.parse(string) instead. I forgot to mention this in the release notes—added it.

1 Like
5

By using language.parser.parse(string) and running highlightTree() under Node.js I get Exception:

      Uncaught TypeError TypeError: rule.tags is not iterable
          at highlightRange (c:\Users\john\Desktop\codemirror-6\node_modules\@codemirror\highlight\dist\index.js:443:38)
          at highlightRange (c:\Users\john\Desktop\codemirror-6\node_modules\@codemirror\highlight\dist\index.js:495:22)
          at highlightTreeRange (c:\Users\john\Desktop\codemirror-6\node_modules\@codemirror\highlight\dist\index.js:504:13)
          at highlightTree (c:\Users\john\Desktop\codemirror-6\node_modules\@codemirror\highlight\dist\index.js:376:5)
          at <anonymous> (c:\Users\john\Desktop\codemirror-6\index.js:20:1)
          at run (internal/modules/esm/module_job:183:25)
          --- async function ---
          at runMainESM (internal/modules/run_main:46:31)
          at executeUserEntryPoint (internal/modules/run_main:76:5)
          at <anonymous> (internal/main/run_main_module:17:47)

// My index.js file:

import {javascript} from '@codemirror/lang-javascript'
import {highlightTree, defaultHighlightStyle} from '@codemirror/highlight'

const support = javascript();

let tree = support.language.parser.parse(`
  for (let i = 0; i < 10; ++i) {
      console.log('i: ' + i);
  }
`);

highlightTree(tree, defaultHighlightStyle.match, (from, to, token) => {
  console.log({from, to, token});
});

USED DEPENDENCIES:

1.  "@codemirror/highlight":        "^0.19.8"
2.  "@codemirror/lang-javascript":  "^6.1.4"
3.  "codemirror":                   "^6.0.1"

USED Node.js: v16.0.0

6

Any 0.19 version means you’re using a deprecated package. In this case, you’ll probably want to use @lezer/highlight, since @codemirror/highlight no longer exists in the stable version.

7

Thanks!

@lezer/highlight solved my problem.

8

I followed this thread to get syntax highlighting for a specific section. The syntax highlighting works since different span styles are applied but I can’t work out how to get the styles that I want applied.

Using the defaultHighlightStyle highlighter gives me two letter classes, using the classHighlighter highlighter gives me a better result but everything is prefaced by ‘tok’ and somethings don’t match what I would expect.

I expected to get classes like ‘cm-variable’, ‘cm-operator’, ‘cm-property’ etc. whereas even the ‘tok’ classes get ‘tok-variableName’ instead of ‘cm-variable’. Am I missing a built in highlighter that gives the classes I’m expecting? Or does this have to be written as a new highlighter?

9

Yes, if you want specific classes you’ll have to write a highlighter that assigns them.

10

Sorry, to clarify I was hoping a defaultHighlighter that highlights tokens in the same way mode.token used to exists.

If not, would there be an easy way to reverse engineer this?