Delimiter disappears when parsing with nested grammars

I am trying to parse a custom language, Nota (grammar), with Javascript embedded inside it (grammar). For example, this screenshot shows a CM6 editor with a program (left), and the Lezer parse tree (right). In this screenshot, nesting is disabled, so only the top-level language (Nota) is being parsed. Notice that {width: 1px} has a Js token.

To enable nesting, I add wrap parameter to the parser’s configuration:

let wrap = parseMixed((node, _input) => {
  if (node.type.id == Js) {
    return { parser: js_language.parser };
  }
  return null;
});

Then this happens:

Notice that:

  • Javascript gets the correct syntax highlighting.
  • The right square bracket disappeared from the span of ArgCodeNamed.
  • The curly braces of {Content} are no longer considered matched.

Any idea where the missing square bracket is going? Am I using parseMixed incorrectly? What seems strange is that using parseMixed seems to be affecting the parse tree outside of the replaced sub-tree.

That was a bug in the mixed-language parser. This patch, included in @lezer/common 0.15.11, should fix it.

2 Likes

Thanks @marijn!

This fixed the issue with the square bracket going missing, although I’m still noticing an issue with bracket matching.

Screen Recording 2021-12-16 at 11.14.01 AM.mov

Consider the right-most curly brace. If I select a different delimiter than the matching left brace, then when I move my cursor back to the right-most, it is considered unmatched in the editor. But when I select the matching left brace, and THEN select the right brace, it is matched in the editor.

I cannot reproduce this issue if I disable mixed parsing.

(Also I can move this discussion to a Github Issue if you prefer!)