Lookahead from within terms

Lezer
1

Imagine a toy rule-set:

@tokens {
  Name { std.asciiLetter+ }
  Number { std.digit+ | "0x" $[0-9a-f]+ }
}

It defines a tokenizer that will parse “0x” as a Number followed by a Name whereas it is probably a malformed number.

Is there a way to add a non-capturing negative lookahead of ![0-9a-f] to the Number term without writing an external tokenizer?

Thanks!

2

No. Lexer tokens are a strictly regular language, and don’t do lookahead.

1 Like
3

Thank you for the quick reply, Marijn!

1 Like