InjectGaps in lang-markdown crashing when used with mixed language parsing

1

Hi,
I’m having an issue with crashes occurring in edge cases when using lang-markdown and mixed language parsing together. For context, I have a template language TemplateString such as

@top TemplateString { (Expression | StringContent)* }


@local tokens {
    ExpressionStart[closedBy=ExpressionEnd]{ "{{" }
    @else StringContent
}

@precedence {
    Expression
    StringContent
}

@skip {} {
    Expression {
        ExpressionStart (JavaScriptText* | "") ExpressionEnd
    }
}

@local tokens {
  ExpressionEnd[openedBy=ExpressionStart] { "}}" }
  @else JavaScriptText
}

Using this grammar, I have a parser that parses external markdown and javascript within the expressions.
My parser looks like -

const mixedMarkdownParser = templateStringParser.configure({
  wrap: parseMixed((node) => {
    if (node.type.isTop) {
      return {
        parser: markdownLanguage.language.parser,
        overlay: (node) => {
          return node.name === 'StringContent'
        },
      }
    } else if (node.name === 'JavaScriptText') {
      return { parser: javascriptLanguage.parser }
    } else {
      return null
    }
  }),
})

When I wrap this in a language support and load this into an editor, It works mostly. However, with some edge cases, this error is thrown -

Uncaught TypeError: Cannot read properties of undefined (reading 'from')
    at movePastNext (index.js:914:1)
    at injectGaps (index.js:922:1)
    at BlockContext.addGaps (index.js:887:1)
    at BlockContext.finish (index.js:884:1)
    at BlockContext.advance (index.js:684:1)
    at MixedParse.advance (index.js:1471:1)
    at MixedParse.advance (index.js:1487:1)
    at eval (index.js:339:1)
    at ParseContext.withContext (index.js:374:1)
    at ParseContext.work (index.js:327:1)
    at LanguageState.apply (index.js:513:1)
    at StateField.update [as updateF] (index.js:533:1)
    at Object.update (index.js:1798:1)
    at EditorState.eval [as computeSlot] (index.js:2630:1)
    at ensureAddr (index.js:2034:1)
    at new EditorState (index.js:2567:1)
    at EditorState.applyTransaction (index.js:2630:1)
    at get state [as state] (index.js:2281:1)
    at EditorView.update (index.js:6469:1)
    at Object.eval [as current] (makeCodeEditor.tsx:145:1)
    at EditorView.eval [as _dispatch] (useCallbackRef.ts:15:1)
    at EditorView.dispatch (index.js:6450:1)
    at applyDOMChange (index.js:5813:1)
    at DOMObserver.flush (index.js:6229:1)
    at MutationObserver.eval (index.js:5935:1)

From injectGaps in the markdown parser.
It’s possible to hit this with multiple different inputs, and I’m not completely sure what triggers it, but I’ve been able to reproduce it consistently with the input

{{}}
b
{{}}

If you copy paste this into the editor, place your cursor at the start of the editor, and press space, you consistently get said issue.

2

Well, that was a mess to track down. Thanks for the reproduction case! This patch should help.

3

Thank you! This fixed it!

4

Hi Marjin,
The first patch fixed a lot of the edge cases, but I’ve been able to reproduce the bug again.
I use this as the input to replicate it -

<h3>${{fun()
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     }}</h3>

To replicate it, I then put my cursor at the end of this text, and I hold enter, at two newlines, the cursor starts jumping and I get the inject gaps issue.

5

Oh, indeed, that was caused by a somewhat different manifestation of the same underlying issue. This patch tries to solve that in a more fundamental way.

6

Ah thank you - would you be able to release another version so I can test it out?
Tried linking my package.json to the git commit but I believe the code in the github is not built yet, so it errors out on my end.

7

I’ve tagged 1.0.4

8

Hi Marjin,
Thanks! However, I’m still able to replicate the bug - I’m using this input here

<h3>${{numbro((offer1_sub.value            ==2&&offer1_bill.value=='quarterly')?2*offer1_contract_sum.value:(off       er1_su}}</h3>
foo

bar

And at the end of the input I hold enter, and the injectGaps issue raises up again

9

Oh no! The combination of incremental parsing and input gaps appears to be very poorly tested. You input exposed another problem—attached patch (1.0.5) should help with this.

10

Thank you! This seems to have fixed all the issues!