An example of an incremental parse?

Lezer
1

I’m trying to get my head around incremental parsing… is there an example of how the parsing API, including TreeFragments and applyChanges etc. should be used somewhere?

I don’t need a hand crafted example, just some existing reference code that I can look at… that’d help immensely!

Thanks!

2

It’s used in @codemirror/language/src/language.ts, but there’s a lot of other stuff going on there making it hard to isolate. Roughly, you keep a set of tree fragments that can be reused for further parses. After a parse, you use TreeFragment.addTree to add your tree to it, and on a document change, you use TreeFragment.applyChanges to remove changed ranges and move tree pieces that the change puts on a different offset.

let tree = parser.parse(myDoc)
let fragments = TreeFragment.addTree(tree)
// Doc changed, range 100-200 removed
fragments = TreeFragment.applyChanges(fragments, [
  {fromA: 100, toA: 200, fromB: 100, toB: 100}
])
let newTree = parser.parse(newDoc, fragments)
// Add new tree to fragments
fragments = TreeFragment.addTree(newTree, fragments)
3

Ahhh thanks! I think I was barking up the wrong tree, then… I was looking at startParse, mainly because in the docs it says:

Start a parse, returning a partial parse object. fragments can be passed in to make the parse incremental.

When would I use startParse, then as opposed to the flow you described?

Also, playing with startParse, I don’t understand how the ranges argument should be used? E.g. if I provide the from and to character indices of the doc’s changes, then the resulting tree doesn’t include them. If I provide 0...(newDoc.length) as the range, though, then the resulting tree does.

4

When you don’t want the parser to run through in one go, but want to control how far it goes or abort it after a given amount of time.

If you pass it, only those ranges of the input are included in the parse. I’m not sure what you mean by ‘doesn’t include them’.