How can I change or add nesting rules to an existing language?

v6
1

I see that the Svelte language definition configures nesting this way:


// the plugin for codemirror
export function svelte() {
  return new LanguageSupport(svelteLanguage, [
    javascript().support,
    css().support,
    autoCloseTags
  ])
}

export const svelteLanguage = LRLanguage.define({
  parser: svelteParser.configure({
    wrap: configureNesting(defaultNesting),
    // ...

const defaultNesting: NestedLang[] = [
  {
    tag: "script",
    attrs: attrs => attrs.type === "text/typescript" || attrs.lang === "ts",
    parser: typescriptLanguage.parser
  },
  // ...
]

But for my grammar / plugin I’m making, I’d like to extend JavaScript, and I see that the svelte equiv here is:

export function javascript(config: {jsx?: boolean, typescript?: boolean} = {}) {
  let lang = config.jsx ? (config.typescript ? tsxLanguage : jsxLanguage)
    : config.typescript ? typescriptLanguage : javascriptLanguage
  return new LanguageSupport(lang, [
    javascriptLanguage.data.of({
      autocomplete: ifNotIn(dontComplete, completeFromList(snippets.concat(keywords)))
    }),
    javascriptLanguage.data.of({
      autocomplete: localCompletionSource
    }),
    config.jsx ? autoCloseTags : [],
  ])
}

which, doesn’t appear to give me options to extend the parser / add wrap entries to it.

I’ve seen some mention of “overlay” in the forums, how does that work? is that what I want?

What I want to do is:

in JS/TS, when I encounter a <template> </template> block, change the parser to my custom thing.
And this can be inside a class body as well:

class Demo {
  <template>
    hello world (from a class)
  </template>
}
2

That isn’t possible—a compiled LR parser (which is what package like @lezer/javascript contain) is a self-contained thing that can’t be modified anymore to support new syntax. Doing something like this requires forking the JavaScript grammar and rewriting it to add your extension.

3

would you accept a PR to add a flag to support this?
This has already been done for treesitter, and is potentially coming soon to SWC’s parser

4

Is this some kind of standard proposal or widely supported syntax? I’ve never seen it.

5

There is a proposal here: Appendix: Optional Second Phase - Polaris Design Sketches

And all of the Ember/Glimmer users use this syntax.

I have two apps,
https://tutorial.glimdown.com which would benefit immensely from <template> support (which I’m trying to implement :tada:)
and https://limber.glimdown.com (iframed in the above interactive tutorial)

And a ton more background here: https://github.com/emberjs/rfcs/pull/779 (> 16k words to read here)
Other implementations:

In general, I’ve found that most JS parsers are in-extensible (Babel, SWC, tsc, etc), forcing things like JSX in to the core of the parser, kinda shutting everyone else out. not fun – would be much better of all language parsers were pluggable, esp since we’ve been dealing with embedded syntaxes for well over a decade now.

widely supported

part of improving support is to… add support :wink:

6

Sure, but I can’t afford to include every variant people use in my parser. As it stands, this doesn’t seem big enough to add to @lezer/javascript, sorry.

7

would it be worth while helping with tweaking the parsers to allow for extension or layering of some sort? so that way any language extension can be separate from what you maintain?

8

This would be technically very difficult and problematic, and is something I’ve decided during the design of Lezer to be out of scope for that system.

1 Like