Python parser: `*` and `**` confusion

fcollonval · February 3, 2023, 3:08pm

This is related to that comment

The Python parser seems to consider * and ** as modifier as specified there:

lezer-parser/python/blob/eac78f1a6fe1a034739b81e56d99313dd46d9641/src/highlight.js#L4


      
          import {styleTags, tags as t} from "@lezer/highlight"
          
          
export const pythonHighlighting = styleTags({
            "async \"*\" \"**\" FormatConversion FormatSpec": t.modifier,
            "for while if elif else try except finally return raise break continue with pass assert await yield match case": t.controlKeyword,
            "in not and or is del": t.operatorKeyword,
            "from def class global nonlocal lambda": t.definitionKeyword,
            import: t.moduleKeyword,
            "with as print": t.keyword,
            Boolean: t.bool,
            None: t.null,
            VariableName: t.variableName,
            "CallExpression/VariableName": t.function(t.variableName),
            "FunctionDefinition/VariableName": t.function(t.definition(t.variableName)),

And never as arithmetic operator (but are ok as update operator) - although the grammar is defined there:

github.com

lezer-parser/python/blob/eac78f1a6fe1a034739b81e56d99313dd46d9641/src/python.grammar#L219-L220


      
          expression !times ArithOp{"*" | "@" | "/" | "%" | "//"} expression |
          expression !power ArithOp{"**"} expression

This is highlighted in the following snapshots in which modifier are bold green and operator purple

Is there a way to improve the parser to distinguish the two usages for * and **?

marijn · February 3, 2023, 3:26pm

That was unintentional. This patch should help.

fcollonval · February 3, 2023, 3:36pm

You are amazing