Understanding lezer precedence

1

Hello,
I have this grammar:

@skip { spaces | newLine }

@top Program { Expr }

@precedence {
    join @left
}

Expr {
  JoinExpr
}

JoinExpr {
    BasicExpr |
    JoinExpr !join "," Expr
}

BoundedExpr {
    "{" Expr "}"
}

BasicExpr {
    String |
    BoundedExpr
}

@tokens {
  String { '"' (!["\\] | "\\" _)* '"' }
  spaces[@export] { $[\u0009 \u000b\u00a0\u1680\u2000-\u200a\u202f\u205f\u3000\ufeff]+ }
  newLine[@export] { $[\r\n\u2028\u2029] }
}

It works fine, for example this code {"a", "b"} will produce this parse tree:

Program:
└╴Expr:
  └╴JoinExpr:
    └╴BasicExpr:
      └╴BoundedExpr:
        └╴Expr:
          └╴JoinExpr:
            ├╴JoinExpr:
            │ └╴BasicExpr:
            │   └╴String: "a"
            └╴Expr:
              └╴JoinExpr:
                └╴BasicExpr:
                  └╴String: "b"

Now I want to support the case with trailing comma before the closing brace like {"a", "b" ,}
How can I achieve that? I tried the following but not working:

@skip { spaces | newLine }

@top Program { Expr }

@precedence {
    trailing,
    join @left
}

Expr {
  JoinExpr
}

JoinExpr {
    BasicExpr |
    JoinExpr !join "," Expr
}

BoundedExpr {
    "{" Expr !trailing ","? "}"
}

BasicExpr {
    String |
    BoundedExpr
}

@tokens {
  String { '"' (!["\\] | "\\" _)* '"' }
  spaces[@export] { $[\u0009 \u000b\u00a0\u1680\u2000-\u200a\u202f\u205f\u3000\ufeff]+ }
  newLine[@export] { $[\r\n\u2028\u2029] }
}

Any thoughts?

2

If I refactor this to a more typical Lezer grammar representation (without the meaningless wrapper nodes—if there’s no comma I don’t think the node should be a JoinExp), and remove the precedence before the comma, this seems to work without problem

@skip { spaces | newLine }

@top Program { expr }

@precedence {
    join @left
}

expr {
    String |
    BoundedExpr |
    JoinExpr { expr !join "," expr }
}

BoundedExpr {
    "{" expr ","? "}"
}

@tokens {
  String { '"' (!["\\] | "\\" _)* '"' }
  spaces[@export] { $[\u0009 \u000b\u00a0\u1680\u2000-\u200a\u202f\u205f\u3000\ufeff]+ }
  newLine[@export] { $[\r\n\u2028\u2029] }
}
3

The problem is that Expr rule has another path

Expr { JoinExpr | SomethingElse }
4

Why is that a problem?

5

Ah! yeah sorry it’s not Expr problem, it’s that there is another rule that is using BasicExpr because the comma is prevented to show there except within a BoundedExpr

6

When converting these kinds of classical context-free grammars, that use deep stacks of rules to express precedence, to Lezer, it’s often better to just collapse the stack into a single rule and use precedence markers to control precedence.

7

Yeah makes a lot of sense, alright I’ll do my best to optimize it. Thanks for help.