Highlighting nested Markdown, and within JavaScript template strings?

Sorry for the long post. In the first part, I show off a neat trick I can do with CodeMirror 6’s Markdown language mode, thanks to its flexibility. In the second part, I point out a way in which I wish CodeMirror 6’s JavaScript language mode was more flexible, and ask how difficult it would be to implement.

I’m making a tool that takes Markdown with code blocks, and it’s useful to document Markdown by showing an example in Markdown, so I configured CodeMirror 6 to highlight nested Markdown. This is ugly, but it works:

const codeLanguages = [
  LanguageDescription.of({
    name: "javascript",
    alias: ["js", "jsx"],
    async load() {
      const { jsxLanguage } = await import("@codemirror/lang-javascript");
      return new LanguageSupport(jsxLanguage);
    },
  }),
  LanguageDescription.of({
    name: "typescript",
    alias: ["ts", "tsx"],
    async load() {
      const { tsxLanguage } = await import("@codemirror/lang-javascript");
      return new LanguageSupport(tsxLanguage);
    },
  }),
  LanguageDescription.of({
    name: "css",
    async load() {
      const { cssLanguage } = await import("@codemirror/lang-css");
      return new LanguageSupport(cssLanguage);
    },
  }),
  LanguageDescription.of({
    name: "python",
    alias: ["py"],
    async load() {
      const { pythonLanguage } = await import("@codemirror/lang-python");
      return new LanguageSupport(pythonLanguage);
    },
  }),
  LanguageDescription.of({
    name: "json",
    async load() {
      const { jsonLanguage } = await import("@codemirror/lang-json");
      return new LanguageSupport(jsonLanguage);
    },
  }),
  LanguageDescription.of({
    name: "sql",
    async load() {
      const { sql, PostgreSQL } = await import("@codemirror/lang-sql");
      return sql({ dialect: PostgreSQL });
    },
  }),
  LanguageDescription.of({
    name: "html",
    alias: ["htm"],
    async load() {
      const { jsxLanguage } = await import("@codemirror/lang-javascript");
      const javascript = new LanguageSupport(jsxLanguage);
      const { cssLanguage } = await import("@codemirror/lang-css");
      const css = new LanguageSupport(cssLanguage);
      const { htmlLanguage } = await import("@codemirror/lang-html");

      return new LanguageSupport(htmlLanguage, [css, javascript]);
    },
  }),
];

const markdownLanguage = markdown({
  codeLanguages: [
    ...codeLanguages,
    LanguageDescription.of({
      name: "markdown",
      alias: ["md", "mkd"],
      async load() {
        return markdown({
          codeLanguages: [
            ...codeLanguages,
            LanguageDescription.of({
              name: "markdown",
              alias: ["md", "mkd"],
              async load() {
                return markdown({
                  codeLanguages: [
                    ...codeLanguages,
                    LanguageDescription.of({
                      name: "markdown",
                      alias: ["md", "mkd"],
                      async load() {
                        return markdown({
                          codeLanguages,
                        });
                      },
                    }),
                  ],
                });
              },
            }),
          ],
        });
      },
    }),
  ],
});

The result is that I can highlight something like this:

`````md
# chems

````js
console.log(3);
function wow() {
  console.log('wowDoge');
}
````

````md
# I can haz nested markdown

Lolz.

```js
console.log('epicWin');
```
````
`````

You can try it at https://notebook.resources.co/ and it highlights properly on the left side, with CodeMirror 6. Source is at GitHub - ResourcesCo/notebook

I’m curious if there is a better way to do this. CodeMirror 6 handles it quite well, though, so mostly I’d just like to point out this ability that the Markdown mode has!




On the JavaScript side, there is something that isn’t yet supported by CodeMirror 6, and I’m not sure if part of Lezer would need to be redesigned to support it. That is highlighting code inside tagged JavaScript strings. It might be complex, but there is this issue from the official playground for Lit, formerly lit-element, a hot new frontend JavaScript framework: Support HTML and CSS inside JS/TS by downgrading to CodeMirror 5 by aomarks · Pull Request #53 · PolymerLabs/playground-elements · GitHub

The basic idea is that if you had HTML like this:

<a class="big-red-car">

…and you needed to make the color configurable, you would add the color to the template string:

htm`<a class="big-${color}-car">`

…and it should highlight the HTML as one HTML document, and not as separate strings <a class="big- and -car">, because by itself the last one would highlight differently, in that " would begin the attribute rather than end it.

Besides embedding a language mode, like HTML above, or CSS or SQL which I’ve also seen, in a way that handles non-contiguous code, I think it might need some control over what the ${} gets treated as, for the purpose of highlighting the text outside it. I think the best default would either be one or more word characters or an empty string.

A good start would be to make it so CodeMirror 6 would highlight as well as playground-elements’ CodeMirror 5 editor. I’d like to attempt it, but I’m not sure where to begin.

This is one of the motivating use cases of the Lezer update that I’m currently working on. So the good news is that support is coming. The bad news is that it might take a while since this system is getting somewhat subtle and I want to make sure I don’t paint myself into a corner with a bad design.

1 Like

That’s good to hear. It seems that in the meantime it might be possible to use stream-parser to get that thing that’s missing for playground-elements at a big performance penalty.

Lezer 0.15 is out, and should make parsing mixed languages inside specific syntactic constructs a lot easier. See parseMixed.

2 Likes

Hi, thanks for the awesome work. Are you planning to make an example specifically for nested parsing?

At some point, yet. I’ve added an item to https://github.com/codemirror/website/issues/13

2 Likes