Are there character sets for Unicode support?

Lezer
1

Hi, I’m trying to support Unicode variable names for my custom language, but I can’t seem to find a way to easily match all characters that seem “textual” (numbers + letters).

I basically want to do the equivalent of [\p{L}\p{N}] in regex.

Is this possible within Lezer currently?

Thank you

2

No, you’ll have to specify specific character code ranges. Lezer tokens are compiled to an automaton, so it can’t use JavaScript’s regexp support for character categories in its matching.