Custom language ast and dynamic variables linting

v6
1

Greetings. Thanks for your great work on codemirrors (I used previous version).

I need implement an editor supporting custom template langauge. It’s an in-house templating solution (don’t ask, not my decision). Examples of the strings are Mama mila ramu {what} and {Who} got {where} on {when}.

I want to have syntax highlighting, autocompletion of the variables (sets are dynamic, will change depending on context in which editor is used) and linting of the variables usage (check variable usage against list of known ones).

I can get autocompletion wihout introduction of own grammar (it’s in the source code). But for the advanced features I need AST. For those I’ve created grammar and tests (using your repo with langauge example). I get tests (they fail and success properly). But when I introduce language to the editor, I’m not getting any ast-related highlighting. Also I don’t get proper AST (I see only 1 node at all times) when I do myLangyage.parser.parse.

Please help with following questions:

What’s wrong with my grammar, and/or approach? Are there better ways to get things I want done?

Repo with the code GitHub - podgorniy/atl: atl

To run tests npm run watch-test.

To run example of usage of wrapper-editor have npm run watch-build and then npm run demo and open http://localhost:1234

2

I think you need a style; try

import { LanguageSupport, defaultHighlightStyle, syntaxHighlighting, } from "@codemirror/language";
// then in extensions
syntaxHighlighting(defaultHighlightStyle, { fallback: true }),

or try something custom with HighlightStyle.define

1 Like
3

Neither tags.operator not tags.variableName have a custom color in the default theme. Maybe that’s the issue?

If your tests pass, it seems like you are getting the proper nodes?

4

True. This was missing. I got some highlight working after adding this extension and “playing” with highlight definitions.

Yes. I was confused that I did not see layout changes inside the editor and did not see ast in the result of parser.parse call. But in both cases my assumptions were wrong: editor’s DOM updates only under some circumstances (not every style definition will make it happen), and ast needs special walker to see it.

For now my blockers are resolved. I’ll get back in case I’ll need help for variable name validation and linting. I’ll keep the repo for historical reasons for those who will want to do the same.

5

I managed to achieve listed goals. I keep the repository GitHub - podgorniy/atl: atl in read-only mode for those who will have same set of goals and for historical reasons. In total it contians

  1. Custom template language grammar with tests.
  2. Highlighting of the variable usage.
  3. Highlighting of the invalid variables (compares agains a list of available ones).
  4. Autocompletion of the variables based on the list of provided ones.
  5. Highlighting of the syntax errors according to grammar.
  6. Class-wrapper around Codemirror@6, and demo of its usage.

Thanks again @marijn for your work and attention.

PS. You might want to export Severity type so people like could construct linting suggestion objects without issues. I had to use any to overcome the issue, link to the code I’m talking about atl/src/main.ts at main · podgorniy/atl · GitHub