How to create custom syntax highlighter using stream parser

I am using codemirror for custom use case where every line goes through a function and I get set of tokens corresponding to each word. I used CodeMirror.defineMode in codemirror 5 which worked perfectly for me. Now I am trying to upgrade to codemirror 6 but not sure how to convert my custom CM5 mode to CM6.
For reference my CM5 function looked like this -

CodeMirror.defineMode("hissab-mode", function () {
  return {
    token(stream) {
      for (const tkn of currTokens) {
        if (stream.match("#") || stream.match("//")) stream.skipToEnd();
        else if (stream.match(tkn.token)) return tkn.style;
      }
      stream.next();
      return null;
    },
  };
});

currTokens holds array of objects with info of all current tokens with corresponding style info.

PS - This is the live implementation if you are curious

1 Like

I seemed to have made some progress with this using the following code -

const state = EditorState.create({
      doc: "a ",
      extensions: [basicSetup, StreamLanguage.define(hissabLang),]
    });
const hissabLang = {} ;
hissabLang.token = function (stream) {
  for (const tkn of currTokens) {
    if (stream.match("#") || stream.match("//")) stream.skipToEnd();
    else if (stream.match(tkn.token)) return tkn.style;
  }
  stream.next();
  return null;
}

The hissabLang.token function now returns a tag name instead of a class name like in CM5.
How do I create a custom tag with style classes and attach it to my StreamLanguage ? A simple example with help a ton! thanks.

Stream languages can return strings much like CM5 modes did. Check out @codemirror/legacy-modes for examples.

(Also, please move that check for comments out of your loop. It’s making me nervous like the way it is.)

2 Likes

Thanks @marijn for the response.
I checked the examples in the legacy modes, but my code above is also doing the same thing. From my understanding, my hissabLang object has a token method which would return null or a string. How do I tell CM6 it is a legacy mode and should use the string as a class name? (as opposed to tagname)

Also thanks for pointing out the stupid mistake in comments check part of the code.

Do you need custom class names, as opposed to the ones used by standard modes? That’s not supported in stream-parser at the moment, it seems.

Thats fine. I just used codemirror tags and defined their styles in my own theme which worked. Thanks again for your help.