How to make sure code is parsed into tokens?

v6
1

I am trying to build interactions where the user can request some code to be added to the editor programmatically - I take following steps

  • insertCode
  • addFocus
  • setCursor

insertCode function is

cm.dispatch({
    changes: { from, insert: text },
  })

Problem is when adding code programmatically - sometimes the CM tokenzier (or parser or whatever module is responsible for different colors for different words basically) fails to work - so all code is rendered using a single color. In this case, if the user clicks somewhere in the editor - the parser works again.

I am trying to figure out if there is something I can do to make sure the any newly added code is correctly tokenized instantly. Any help would be appreciated thank you.

2

That would be a bug. Do you have an example that can reliably reproduce this?

3

Unfortunately not yet. It happens sometimes but not always. I am thinking may be it has to do with editor not having focus when code is added to it. So I am trying to experiment with the following order

  • addFocus
  • insertCode
  • setCursor

for the parser to work, does the editor need to have focus and the cursor shown? every time the user clicks inside the editor - the parser corrects itself cleanly.

Will post here if I can reproduce this reliably.

4

If the parser has been running for 3 full seconds (in the past 30 seconds) and the editor is unfocused, it’ll start backing off. But unless you have a giant document, a really inefficient parser, or an extremely slow computer, it seems like that limit should not be hit so easily.

5

Thanks. I was able to resolve this in Vue 3 using nextTick function. It turns out if I wait for Vue to update the DOM and then add focus to the editor and set cursor programmatically - it parses all tokens correctly. May be adding focus is enough, setting cursor may not be required. But I was elated to have it work - so haven’t tested that.