can we pass a dynamic regex for grammar?

Lezer
1

Hi,

First of all, thanks for creating this cool parser.

I wonder is there any way that I can replace the rule of grammar in runtime?

For example:

@top { expression }

@tokens {
  expression { @asciiLetter }
}

and in runtime, I may need to change the expression as @digit for example.

Or is there any workaround?

Thanks

2

Not for normal tokens. But external tokenizers can be replaced at run-time via the tokenizers option and the configure parser method.

3

thanks for your reply. Lemme test those two

4

I tried to understand the things you mentioned, and then I checked the package, but it seems the ExternalTokenizer input only accept a data with number type. I am not sure how to pass a custom regex to it.

For example, I wanna highlight texts using the user input, I created a grammar

@top RegexText { regex }

@external propSource regexHighlighting from "./highlight"

@external tokens regexToken from "./tokens" { regex }

@detectDelim

For the token file,

import { ExternalTokenizer } from '@lezer/lr';

import { regex } from './regex.grammar.terms';

export const regexToken = new ExternalTokenizer((input, _stack) => {
  input.acceptToken(regex);
});

I stuck at this step