Create autocomplete from grammar terms

Lezer
1

The grammar already contains the syntactic structure of the language. And when you create a parser from the grammar, a list of terms is also created. These terms appear in the syntax tree that the parser produces.

I was thinking, what if I could request the possible next terms that the parser is looking for at the position of the cursor (basically, what is shiftable/reducable)? This may give me both Expression and Variable (just a Variable is one of the possible Expression options), but then I would just only do something with Variable, namely return a Completion with the list of all currently defined variables.

Because I am currently very stuck on writing the autocomplete, since just walking the syntax tree which also contains error nodes, I find very difficult. And whenever I try to come up with something, there’s a lot of node.firstChild, node.nextSibling, state.sliceDoc(), error nodes in the way, subsequent non-terminals being a child of the error node (instead of sibling of error node), which all seems not very robust and error-prone.

2

In my experience, the parser state is not that useful for autocompletion. You’ll typically be completing identifiers, which are often a single token in the grammar. Lezer only outputs the tree, not the parser state at a given document position, so it’d be a pain to get access to that in the first place. Looking at surrounding syntax nodes to classify a given position’s syntactic role, and deriving completions from that, has worked well for me.

3

So what is the best way to learn how to use the surrounding syntax nodes? It is by looking at the HTML autocomplete example, because that one confused me.

@top Program { 
  (StateDefinition newline | newline)* StateDefinition? 
}

StateDefinition {
  StateName Assign "{" (key<"color"> Colon Quoted<ColorName | ColorHex>)? "}"
}

StateName { identifier }
Assign { "=" }
Colon { ":" }

Quoted<term> { quote1 term quote1 | quote2 term quote2 }
key<term>    { @extend[@name="Key"] <identifier, term> }

@tokens {
  identifier        { $[a-zA-Z_] $[a-zA-Z0-9_]* }

  ColorName         { $[a-zA-Z]+ }
  ColorHex          { "#" $[0-9a-fA-F]* }

  quote1            { "'" }
  quote2            { "\"" }

  space             { " " }
  newline           { "\r"? "\n" }
  Comment           { "//" ![\n]* }  
}

// All spaces and comments will be skipped by the lexer.
@skip { space | Comment }

Say I wanted to write autocomplete for this grammar. Where I have a list of allowed StateName, which the autocomplete could give when starting to type a StateName. And the autocomplete should also provide the color: '*cursor*' after you type the {. And the autocomplete could give a list of suggested ColorName such as ‘red’, ‘green’, ‘blue’.