Highlighting/data-language not activate for Robomind language

v6
1

I am the developer of RobomindAcademy, a free site teaching Computational Thinking that has been using CodeMirror for more then 10 years (dank Marijn!). Currently i am preparing a major revamp of the Academy, which includes upgrading to CodeMirror 6.

  • I have created a Robomind Lezer parser including a set of tests (modelled after Marijn’s Javascript parser) that all pass.
  • The Robomind language support more then 25 natural languages which are ‘dialects’ in the Lezer parser.
  • A tree constructed with robomindLanguage.parser.parse(“repeat { forward }”) and examined using a cursor looks OK, including highlighting tokens.
  • I created an HTML page containing a simple editor which allows dynamically switching between Javascript mode and Robomind Mode. This page and the switching works OK, however …

When switching to Javascript mode, highlighting works, and INSPECTING the ‘.cm-content’ div shows a ‘data-language=javascript’ tag. When switching to the Robomind mode, no highlighting and no ‘data-language’ tag. When in Robomind mode, the Robomind Lezer parser is called (a few well-planned console.log statements in the Lezer parser confirms this) so the mode seem to be active. There are no warnings issued in the browser’s console at any time.

I have been at this for days now, and have stripped back the code to the essentials but no luck. In a debug session the LanguageSupport/LRLanguage/LRParser blobs for Javascript look like those for Robomind although ‘context’ in LRParser is undefined.

Can anybody suggest a next course of action? I suspect the CodeMirror6 runtime rejects parts of the Robomind language support but i can not find which and why. Is there a verbose-mode i can switch on? Any suggestions? Help!

2

Are you defining highlighting tags for your language?

What does your LRLanguage.define call look like?

3

FYI, i have used your lezer-javascript/lang-javascript as inspiration. I have combined them both in one module but am planning to seperate them today in lezer-robomind/lang-robomind. As extra info, the completions are working in the simple editor so something is accepted :wink:

My highlight.js looks like this (as i said, stripped back to its essentials):

import {styleTags, tags as t} from "@lezer/highlight"

export const roboHighlight = styleTags({
    REPEAT: t.controlKeyword,
    repeat: t.controlKeyword,
})

In my index.ts i have the LRLanguage.define as follows:

let parserWithMetadata = parser.configure({
    props: [
        indentNodeProp.add({
            Codeblock: delimitedIndent({closing: "}"}),
        }),
        foldNodeProp.add({
            "Codeblock": foldInside,
        })
    ]
})


export const robomindLanguage = LRLanguage.define({
    name: "robomind",
    parser: parserWithMetadata,
    languageData: {
        commentTokens: {line: "#"},
        closeBrackets: {brackets: ["(", "[", "{", "'", '"', "`"]},
        indentOnInput: /^\s*\{$/
    }
})

export function robomind(locale = "en") {
    const lang = robomindLanguage.configure({dialect: locale}, `robomind_${locale}`);
    const completions = buildCompletions(locale);

    return new LanguageSupport(lang, [
            lang.data.of({
                autocomplete: completeFromList(completions)
            })
        ]
    );
}

Thanks! … Jan

4

I am splitting the code in separate lezer-robomind and lang-robomind modules and looking at the equivalent lezer-javascript/lang-javascript i notice some rollup logic that was missing from the combined module. I will try to get the demo HTML running again and will report back.

5

I finished splitting up the code into lezer-robomind and lang-robomind. These modules use package.json and rollup.config.js that are almost identical to lezer-javascript/lang-javascript including the use of cm-buildhelper in the lang-javascript module.

The demonstration HTML runs again using the new modules but the symptoms remain the same. Highlighting/data-language when the javascript mode is active, no-hightlighting/data-language when robomind mode is active.

The robomind parser tests OK and i made the lang-robomind module as close to its javascript brother as i can, but still no luck.

6

One issue you might be running into is duplicated packages. If you’re developing the language as a separate package with its own node_modules dir and then linking it into another project, that won’t work because both will be loading a different instance of @codemirror/state and @codemirror/language, and instanceof checks will fail.

7

Marijn, you are my hero for today. I indeed have three packages (lezer-robomind, lang-robomind and raeditor) that each have their own nodes_modules. They are local and linked together using something like:

npm install --install-links=false file:../lezer-robomind

After your message i did the following:

  1. made sure all three packages use the same versions of things like @lezer/lr. There were differences!
  2. added the following line to rollup.config.js/plugins ‘resolve({dedupe: [“@codemirror/state”, “@codemirror/language”]})’
  3. removed existing node_modules and run npm install & npm dedupe

I do not know what of the three did the trick, but i now have highlighting and ‘data-language=“robomind_en”’ in robomind mode.

I only started using npm/rollup after reading your codemirror documentation a couple of months ago so i know just enough to be dangerous. Do you have suggestions how to improve my development setup? In any case, you are my hero and thanks for the great editor. I will drop you a note when the new Robomind Academy goes life.

8

npm can be a bit of a pain to work with in setups like this. The least messy solution is probably to set up a parent ‘workspace’ project (see these docs) that you check out the modules into, and install all them all together with a shared node_modules tree.