Hi,
I’m currently working on a custom mode for a custom language. That works nicely so far. Since I’d like to do some more work than just tokenizing once the whole text is processed, my question is: How do i know all the text has been processed? E.g. is there some way to check if the stream which is accessible in the mode’s token() function is at “end of file” instead of just “end of line”? Or something similar? Can I define a “done” callback which is then executed by CodeMirror when the tokenizing is done?
I could include some “content is a valid word of the language” flag in the tokenizer’s state or perfom some action whenever the content read so far is valid, but I’d never know in advance if there is garbage content in the lines to follow - in which case no further action should be performed.
Can this currently be done without extending CodeMirror itself?
This kind of thing is not what modes are designed for – they intentionally have a limited view on the text, so that they can be cheaply and reliably re-ran on pieces of the code as it is updated.
For more complicated analysis, I recommend you write external code that listens for changes, fetches text from the editor, and runs the analysis at its own pace.
Thanks for your input. I thought I had replied right away, but I must have missed the proper Button since the text I wrote was still in the answer window.
Anyway, in the mean time I created a SingleLineLexer which works on a CodeMirror.StringStream (thanks for exposing this type so it is reusable!). Secondly, I added a MultiLineLexer which splits some text into lines and feeds them to the SingleLineLexer used internally. Always when the current line’s stream returns true for .eol(), the next stream is created for the appropriate line.
To be able to reuse the SingleLineLexer for the mode definition I’ll again wrap the lexer, add a bit of state handling and additional styling I didn’t want to include in the lexer and pass a factory function for the lexer to the mode definition using the passed “config” object.
At first I went with using the “parserConfig” object, but my extra properties were missing when CodeMirror called getMode or something like that. Could be my bad since I didn’t waste much time on this and just used “config”. Doesn’t feel very clean to me right now but I can live with it.