Pass specializer context to parser?

Lezer
1

I have a situation where I’d like to be able to specialize identifiers more specifically. In my case, while the identifiers are dynamic, I always know what they are in advance, and the set of identifiers will remain immutable for the lifetime of the parser.

Is there a way to pass them to the parser? While one can externally specialize, it’s unclear to me how I could, for example, supply a Set of strings to the parser so that in turn it could be supplied to the external specialization function.

2

You can replace external specializers with new values via the specializers option (as passed to LRParser.configure). Import some kind of default specializer in your grammar, and then create a new specializer function that knows about the desired set of words and do something like

parser.configure({
  specializers: [{from: defaultValue, to: mySpecializer(myKeywords)}]
})
3

Thanks for the response @marijn. I’ve been trying that to no avail. The problem I’m running into is in this loop:

 if (config.specializers)
    copy.specializers = this.specializers.map(s => {
        let found = config.specializers.find(r => r.from == s);
        return found ? found.to : s;
    });

Stepping through that in the debugger, what I’m seeing is that defaultValue is never found and thus never replaced. It’s just a simple function intended to be replaced:

export const defaultValue = () => -1;

I think the issue may be that what’s in this.specializers by that time isn’t going to be matched by the equality test in that loop.

4

Perhaps a more full exposition of what I’m attempting might reveal some error on my part:

Grammar excerpt:

@external specialize { Identifier } defaultValue from './tokens' {
  ProfileName
}

Function to be replaced:

export const defaultValue = () => -1;

Attempting the replacement:

import {ProfileName}  from './apex.terms';
import {defaultValue} from './tokens';

export const language = names => LRLanguage.define({
  parser: parser.configure({
    specializers: [{
      from: defaultValue,
      to:   value => {
        console.log(value);
        return names.profiles.has(value) ? ProfileName : -1
      }
    }]
  })
});

Walking through in the debugger, things look to be set up correctly on entry to the map() loop, and it’s walking through the array of specializers defined in the grammar, including the one to be replaced, but the one to be replaced is never matched by the equality test.

5

As an alternate solution here, I’ve resorted to attaching the keywords to the parser object contained in the language returned from LRLanguage.define(), then accessing them from that parser that’s accessible from the Stack object that’s provided as the second argument to an external specializer. It’s a bit grimy, but it works.

6

On trying this out, I found that ParserConfig.specializers was broken. Upgrading to @lezer/generator 1.1.1 and @lezer/lr 1.2.1 should fix that.

1 Like
7

It does indeed fix it; thanks. Working properly now.