Trouble defining asymmetric inline parsers in `@lezer/markdown`

Lezer
1

Hi, I’m trying to extend markdown syntax in Lezer/CodeMirror for an app I’m working on.
This new syntax should be inline and of the form ![[text]], i.e. text between the opening delimiters ![[ and ]].

The two best pieces of relevant information I found are DelimiterType's section in the GitHub docs for lezer-markdown, and this discussion here in this forum.

My strategy is to define two new inline parsers, one for each delimiter, and use cx.addDelimiter() to register them when found (with the corresponding values for open and close, and using DelimiterType.resolve with the same value for both). The full (relevant) code:

export const embeddedRefTags = {
  embeddedRef: Tag.define(tags.link),
  embeddedRefStart: Tag.define(tags.processingInstruction),
  embeddedRefEnd: Tag.define(tags.processingInstruction),
}

const embeddedRef = "EmbeddedRef";
const embeddedRefStart = "EmbeddedRefStart";
const embeddedRefEnd = "EmbeddedRefEnd";

const embeddedRefStartDelim = { resolve: embeddedRef, mark: embeddedRef };
const embeddedRefEndDelim = { resolve: embeddedRef, mark: embeddedRef };

export const markdownEmbeddedRefs: MarkdownConfig = {
  defineNodes: [embeddedRef, embeddedRefStart, embeddedRefEnd],
  parseInline: [{
    before: "Image",
    name: embeddedRefStart,
    parse(cx, _, pos) {
      const openMark = '![[';
      if (cx.slice(pos, pos + openMark.length) !== openMark) {
        return -1;
      }

      return cx.addDelimiter(embeddedRefStartDelim, pos, pos + openMark.length, true, false);
    }
  }, {
    before: "LinkEnd",
    name: embeddedRefEnd,
    parse(cx, _, pos) {
      const closeMark = ']]';
      if (cx.slice(pos, pos + closeMark.length) !== closeMark) {
        return -1;
      }

      return cx.addDelimiter(embeddedRefEndDelim, pos, pos + closeMark.length, false, true);
    }
  }],
  props: [
    styleTags({
      [embeddedRefStart]: embeddedRefTags.embeddedRefStart,
      [embeddedRefEnd]: embeddedRefTags.embeddedRefEnd,
      [`${embeddedRef}/...`]: embeddedRefTags.embeddedRef,
    }),
  ],
}

this doesn’t seem to work; this extension above does seem to match the correct text, however iterating the syntax tree (with this extension registered) shows no signs of it. Here’s also a minimal reproduction.

What am I doing wrong?
If it’s relevant, changing both the open and close parameters of cx.addDelimiter() to true produces the expected behavior (they show up on the syntax tree, matching pairs of ![[ tokens and pairs of ]] tokens).

Originally I had a solution which worked but used dirty tricks (most notably using cx.parts heavily, in a similar fashion to some of the snippets in the thread mentioned above) and I’m now trying to do it properly.

Thanks!

2

The type object for the opening delimiter has to match the one for the closing delimiter—in your code, embeddedRefStartDelim and embeddedRefEndDelim are used and don’t match.

3

Ah, I see! I thought only the resolve fields of both delimiters need to match, not the delimiter objects themselves.

Then, If I understand, only a single delimiter object (say, EmbeddedRefDelim) should be used?

If so, both the start and end delimiters are marked by the same syntax node, e.g. EmbeddedRefMark. Is there a canonical way to mark them as distinct nodes, e.g. EmbeddedRefStart and EmbeddedRefEnd, as in the example above?

Also, in that case does defining EmbeddedRefStart and EmbeddedRefEnd as new nodes (like in my original example) serve any purpose rather than providing my InlineParsers with a name?

By the way, if anyone’s interested - I’ve modified the above reproduction accorordingly - working code.

Many thanks!

4

No, that doesn’t seem to be possible at the moment. If this is important here I guess I could add a feature for it.

And indeed, you don’t need two marker node types if you only use one.