How to convert grammar into a JavaScript object ?

I want to convert grammar into a JavaScript object (similar to automata representation of it) which can be iterated and all possible next tokens (states) can be generated from the current position.

My use-case is to implement a generic autocompletion for my custom grammar. Idea is to get all next set of tokens possible from current position and using them to suggest all possible options. For this I need to get grammar in a JavaScript object and then use it to predict all set of states possible.

Any ideas/help on this would be helpful.

The parser in lezer-generator isn’t exported (and I don’t really want to export it, since that means it has to stay compatible) but there’s a lezer parser for lezer grammars that might be useful here.

(Edit: fix link)

Hi @marijn , I tried using lezer grammar to convert .grammar file into a tree but it seems tricky to parse grammar file notations and convert them into possible next states. For example in grammar one can use regex like (A+ | B* ). In these cases, it would have been better if with the help of parser (LRParser) I can get all possible set of next tokens reachable from current cursor position. I see that currently in LRParser there is a method nextStates but the information is encoded in numbers which is hard to understand. Can you please expose some function which can return next set of tokens from current position ? Any guidance here would be very helpful.

An LRParser is an optimized single-purpose data structure meant for parsing. I don’t think it’s a good idea to try to make that a general-purpose grammar description object.

Is there a utility to get such thing ? Given that internally parser does similar thing of producing next states (here tokens), it feels kind of reinventing the wheel to do the same thing.

No, the library doesn’t expose anything like that.