Lezer - GLR split not working

Lezer
1

Hi! I am working on the Julia grammar: GitHub - JuliaPluto/lezer-julia: Julia parser for Lezer :slight_smile:

Problem

I am struggling to get an ambiguity to work. There are two syntax patterns that I want to support:

# a matrix
[
  1 2
  3 4
]

and

# a list comprehension, spanning multiple lines
[
  sqrt(x)
  for x in 1:10
]

The first works. The second one does not parse. It starts to parse this as a Matrix, and then errors on the unexpected for token.

My .grammar code looks like:

array {
  ( ComprehensionExpression { '[' Generator<expr-idx> ']' }
  | MatrixExpression { '[' MatrixRow (_t MatrixRow)* _t? ']' }

  // you can ignore this part
  | VectorExpression { '[' sep<',', (!vec (expr-idx | binding-idx<'Assignment'>))> ']' }
  )
}

It seems like the first path (ComprehensionExpression) is not explored. When I remove the MatrixExpression path, then it does parse the second example correctly.

My questions is: I believed that if there are two possible paths (joined with |), and only one parses correctly, then that one will be used. Why is this not the case here?

Solving with ambiguity markers

I tried to solve this with ~ and dynamic precedence. This gives me:

array {
  ( ComprehensionExpression[@dynamicPrecedence=8] { '[' ~ambiiiiiguous expr-idx !simple GenFor (GenFor | GenFilter)* ']' }
  | MatrixExpression { '[' ~ambiiiiiguous MatrixRow (_t MatrixRow)* _t? ']' }

  // you can ignore this part
  | VectorExpression { '[' sep<',', (!vec (expr-idx | binding-idx<'Assignment'>))> ']' }
  )
}

This does not change the behaviour.

To reproduce:

Checkout GitHub - JuliaPluto/lezer-julia: Julia parser for Lezer . Add the second example somewhere in the tests to see it fail.

Remove this line and test again, to see it now correctly parse the list comprehension.

Thanks for taking a look!

2

By default, if there are two paths that lead to incompatible parses for the same tokens, you get a static error until you break the tie with a ! precedence or explicitly allow ambiguity with a ~ marker. You appear to be doing the former in this grammar, which will cause the higher-precedence interpretation to block the other one entirely. In this case, I suspect the !mat at the start of MatrixRow is the point where the comprehension is prevented from matching. Both allow an expression, but MatrixRow has a precedence that makes it override the other.

1 Like