External specialize token injection at runtime

v6
1

Hi,

I am new in codemirror/lezer.
I am trying to write a new language.

In syntax.grammer file:

@external specialize {identifier} fields from "./tokens" {
	Metric[@name=Metric],
	Dimension[@name=Dimension]
}

In tokens.ts

const metrics = ["quantity", "cost" ];
const dimensions = ["name", "date"]

export function fields(name:string) {
	name = name.toLowerCase();
    if (metrics.includes(name)){
		return Metric;
	}
	if (dimensions.includes(name)){
		return Dimension;
	}
	return -1;
}

My question is: how to send to language module metrics and dimensions variables at runtime, especially access metrics and dimensions from tokens.ts file (metrics and dimensions depend on context).
My main script main.js is outside of language module:

window.view = new EditorView({
    doc: 'cost',
    extensions: [
        basicSetup,
        mylangScript()
    ],
    parent: document.querySelector("#editor")
});

Thanks a lot!

PS: I’ve just discovered codeMirror / Lezer. It’s magic!

2

You can use the specializers option to LRParser.configure to replace a given specializer (say, identifier) with another one.

1 Like
3

Related: Pass specializer context to parser?

1 Like
4

Thanks a lot. It works. Final solution:

main.js

const metrics = ["quantity", "cost" ];
const dimensions = ["name", "date"]

window.view = new EditorView({
    doc: 'cost',
    extensions: [
        basicSetup,
        myScript(metrics, dimensions)
    ],
    parent: document.querySelector("#editor")
});

index.ts from myScript language module (related part only)

import {initializeUserTokens, defaultField, userField} from "./tokens"

const parserWithMetadata = parser.configure({
	specializers: [{ from: defaultField, to: userField }]
})

export const myScriptLanguage = LRLanguage.define({
  parser: parserWithMetadata
})

export function myScript(metrics, dimensions) {	
	initializeUserTokens(metrics, dimensions);	
  return new LanguageSupport(myScriptLanguage, [])
}

tokens.ts (related part only)

const userTokens = {
	metrics: [],
	dimensions: []
};
export function initializeUserTokens(metrics, dimensions){
	userTokens.metrics = metrics || [];
	userTokens.dimensions = dimensions || [];
}
export const defaultField = () => -1;
export function userField(name:string) {
	name = name.toLowerCase();
	
	if (userTokens.metrics.includes(name)){
		return Metric;
	}
	if (userTokens.dimensions.includes(name)){
		return Dimension;
	}
	return -1;
}

And finally myScript.grammar

@external specialize {identifier} defaultField from "./tokens" {
	Metric[@name=Metric],
	Dimension[@name=Dimension]
}

Thanks again !