Know which part of syntaxtree was changed


Is there a way to know, with help of Lezers incremental parsing, what parts of the tree have remained unchanged?

I’m analysing the tree to infer variable usage, but it gets pretty heavy to run that on the whole tree every keystroke.

Is anything like that possible?

1 Like

It is possible, but not implemented. It’s also not entirely obvious to define. Did a node whose content is the same as in the old tree but who got a different parent node unchanged? Depending on what your answer to that is, and whether a conservative estimate works for you, you may be able to use a WeakMap to associate Tree instances with an ‘already handled’ flag, and use that to avoid re-processing subtrees that you’ve already seen.

Awesome, this works for some cases (when there is an .tree on the cursor).
Is it okay to do the same caching on cursor.buffer and cursor.index together? It seems to work (apart from having to correct the positions), but I feel like there might be a reason you didn’t document these :stuck_out_tongue:

Are you sure you’re doing something that’s sufficiently expensive that caching helps there? Buffers will be small.

Yeah, seems so… maybe my grammar isn’t good at creating trees? For my 500 lines test case I get just one tree… Not sure what about my code is so slow that it gets speed up this much by caching…

Maybe I should just make it debounced (so the editor can render and my extra parsing will only run after some time)

Tree structure should be largely independent of grammar (though having a repeat operator somewhere around the top level constructs might help).

I had a WeakMap wrapper that allowed one to associate stuff with every node (by cursor or SyntaxNode) as part of an experiment a while back. Maybe I should add that to the @lezer/common interface.