Block comment for StreamLanguage

v6
1

Hey!

I have defined a StreamLanguage and I would like it to match a multi-line block comment in the following format:
/* my comment another line */
The token stream is single line, so I would like to match the first part of the comment and hold a value somewhere indicating that the comment has started and check for a match. Is there a straight-forward way to do this?

My current code matches only single line:

import { StreamLanguage } from '@codemirror/language'

const lang = StreamLanguage.define({
	token(stream, state) {
		if (stream.eatSpace()) return null
		if (stream.match(/\/\*.*?(\*\/|$)/)) return 'blockComment'
		stream.next()
		return 'literal'
	},
})
2

You may find the following article useful on how to use a regular expression to find JavaScript comments. I hope it helps.

3

Thank you for the resource!
But the issue is that the stream only contains one line, but not the whole text, so I am wondering, if there a way to pass some metadata while parsing next line.

4

You have to keep a value in your tokenizer state that indicates whether the stream is currently in a comment, and base further tokenization on that.

5

Cool! Thank you for the tip!
Here is the code for someone getting into the same problem:

import { StreamLanguage } from '@codemirror/language'

const lang = StreamLanguage.define({
	startState() {
		return {
			blockCommentActive: false
		}
	},
	token(stream, state) {
		if (stream.eatSpace()) return null

		// multiline comment matching
		if (stream.match(/^\/\*.*\*\//)) return 'blockComment'
		if (stream.match(/^\/\*.*$/)) {
			state.blockCommentActive = true
			return 'blockComment'
		}
		if (state.blockCommentActive) {
			if (stream.match(/^(.*?)\*\//)) {
				state.blockCommentActive = false
				return 'blockComment'
			}
			stream.match(/^.*/)
			return 'blockComment'
		}
		stream.next()
		return 'literal'
	}
})