no backtracking after some convincing point

Lezer
1

wonderful.
I am just setting out to parse what I have been compiling with regexes (here and here) until now.
It’s a simple javascript with extra stuff, like Sunpit and io (not that io)

IOing is like xpath, it creates and interacts with structure.

Sunpit is like a heading.
It may contain words and|or start with a number:

S 3 innards
    ...code

It may contain an IO expression:

S o yeses/because/blon_itn
    ...code

In which case I want that to be IOing, though Title can consume anything on a line because:

    nonnl { ![\n] }

If not given the choice (removing SunpitHead from Sunpit) it will match IOing fine.

My question is… How can it be convinced IOing is the way to go when it is up to "S o ", so I can get some kind of error inside an IOing syntax node, etc

Not sure if this is ambiguity or precedence, as the lezer guide mentions. Ignore the @dynamicPrecedence which does nothing.

Something like “(*COMMIT)” from perldoc perlre, which “when backtracked into on failure it causes the match to fail outright”.


@tokens {
    IOness { "i" | "o" }
    Sunpitness { "S" }
    Title { nonnl+ }
    Name { (@asciiLetter | "_")+ (@asciiLetter | "_" | @digit)* }
    Number { @digit+ }
    nonnl { ![\n] }
    nl { "\n" }
    Sigil { "@" | "#" }
    space  { @whitespace }
    @precedence {Number, Title, Sunpitness, IOness, SunpitHead, Name}
    @precedence {nl, space}
}

@top Program { Line* }
Line { inline nl }

inline { Comment | expression | space* }

Comment { "#" nonnl+ }

expression { Name | Number | IOing | Sunpit }




IOing[@dynamicPrecedence=10] { IOness ' ' IOpath }

IOpath { Leg ("/" Leg)* }
Leg { Sigil? Name }



Sunpit { Sunpitness " " (IOing | SunpitHead) }
SunpitHead { Number | Number Title | Title }

I’m not too confident in my abilities here yet.
My new-design-for-software system has lately been converting syntax into cytoscape.
Seems like there is some serious art to make there, eg being able to follow names off into other piles of code, as you do with code discovery whenever you forget how things work etc.

Thanks for making open source software. Here’s a picture of the above.

lezer choosing Title over IOing
lezer choosing Title over IOing1920×1080 256 KB

2

Lezer is not a backtracking parser. I must say I don’t understand your use case well enough to say anything concrete, but if you have syntax where two things match, and you’re using GLR parsing to accept both of them, dynamic precedence is indeed how you’d disambiguate this. But since your grammar contains to ~ GLR markers or @extend tokens, that dynamic precedence will do nothing.

1 Like