Equivalent of getStateAfter in CM6

anisus · December 30, 2021, 2:08pm

What is the equivalent of calling cm.getStateAfter(null, true) when migrating from CodeMirror 5 to CodeMirror 6, using StreamParser?

Thanks for a great library!

marijn · December 30, 2021, 5:53pm

This doesn’t exist anymore, since parsers work differently now, and the stream parser doesn’t export a feature like this. What were you using it for?

anisus · December 30, 2021, 7:14pm

I’ve been using CodeMirror 5 for command line parsing, which I understand is quite different from the intended use. But it has been working quite well, with the exception of lack of spell check (which is why I try to migrate).

Example:

codemirror_console

My custom mode parses the command to both highlighting it with color, but also to interpret and store the parsed parts in the state object. Later (when pressing Enter), I retrieve the state with getStateAfter and use it to execute the command.

In the screenshot example above, the state would, at the end of the last token (which is “Creator of the realm”) contain something similiar to:

{
    cmd: "setTag",
    tagName: "creator",
    tagAttr: "desc",
    desc: "Creator of the realm"
}

A different use case, I know

In CM6, I have seen that the state does exist inside the syntax tree:

syntaxTree(editorState).resolve(editorState.doc.length)).node.children[0].props[11]

Now… that line of code is clearly not very useful (see the “arbitrary” 11), but at least it is there. But how to get it in a correct way?

anisus · December 30, 2021, 10:18pm

I think I’ve gotten it to work by replicating the internal findState function in @codemirror/stream-parser:

github.com

codemirror/stream-parser/blob/ab2d41364b671a8edf2693372cbff269d8d8b228/src/stream-parser.ts#L114

    
      
                if (end == pos) break
                statePos = line.to + 1
              }
              let {text} = cx.state.doc.lineAt(pos)
              return this.streamParser.indent(state, /^\s*(.*)/.exec(text)![1], cx)
            }
          
          
  get allowsNesting() { return false }
          }
          
          
function findState<State>(
            lang: StreamLanguage<State>, tree: Tree, off: number, startPos: number, before: number
          ): {state: State, pos: number} | null {
            let state = off >= startPos && off + tree.length <= before && tree.prop(lang.stateAfter)
            if (state) return {state: lang.streamParser.copyState(state), pos: off + tree.length}
            for (let i = tree.children.length - 1; i >= 0; i--) {
              let child = tree.children[i], pos = off + tree.positions[i]
              let found = child instanceof Tree && pos < before && findState(lang, child, pos, startPos, before)
              if (found) return found
            }
            return null

Hopefully I haven’t misunderstood anything important

marijn · December 31, 2021, 11:42am

The recommended way to do this in CM6 would be to switch to a Lezer-based parser, and see what nodes exist in the parse tree (see syntaxTree) of the line you’re interested in. You could also look at the tree emitted by the stream parser, though that’ll be flat and somewhat harder to interpret.