I want to convert grammar into a JavaScript object (similar to automata representation of it) which can be iterated and all possible next tokens (states) can be generated from the current position.
My use-case is to implement a generic autocompletion for my custom grammar. Idea is to get all next set of tokens possible from current position and using them to suggest all possible options. For this I need to get grammar in a JavaScript object and then use it to predict all set of states possible.
The parser in lezer-generator isn’t exported (and I don’t really want to export it, since that means it has to stay compatible) but there’s a lezer parser for lezer grammars that might be useful here.
Hi @marijn , I tried using lezer grammar to convert .grammar file into a tree but it seems tricky to parse grammar file notations and convert them into possible next states. For example in grammar one can use regex like (A+ | B* ). In these cases, it would have been better if with the help of parser (LRParser) I can get all possible set of next tokens reachable from current cursor position. I see that currently in LRParser there is a method nextStates but the information is encoded in numbers which is hard to understand. Can you please expose some function which can return next set of tokens from current position ? Any guidance here would be very helpful.
An LRParser is an optimized single-purpose data structure meant for parsing. I don’t think it’s a good idea to try to make that a general-purpose grammar description object.
Is there a utility to get such thing ? Given that internally parser does similar thing of producing next states (here tokens), it feels kind of reinventing the wheel to do the same thing.