# Template style language

**URL:** https://discuss.codemirror.net/t/template-style-language/5558
**Category:** Lezer
**Created:** [January 16, 2023, 8:15pm UTC](https://discuss.codemirror.net/t/template-style-language/5558 "2023-01-16T20:15:02Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![rossm6](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/rossm6/32/3633_2.png) [@rossm6](https://discuss.codemirror.net/u/rossm6)
#### Post date: [January 16, 2023, 8:15pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/1 "2023-01-16T20:15:02Z")

</div>

Ultimately I want a user’s input into the codemirror editor to be a string which concatenates consectutive parts where a part is either a template or not and where the template will contain a javascript expression to be evaluated.

For example (if x is number 2),

“abc {{ x + 2 }} zep”

Would evaluate to “abc 4 zep”.

As you probably gathered “{{” and “}}” are used to determine the javascript expression.

From what I’ve researched I will need to use the “mixed parser”.

However I’m struggling to jump over the first hurdle of building the outside parser.

Based on “lang-json” I’ve got something close but need a little help pointing me in the right direction or telling me what I’m aiming for isn’t even a go-er.

This is the grammar.

```auto
@top Templates { value* }

value { Object | String | whitespace }

Object { "{{" Stuff* "}}" }

String {
  string
}

Stuff {
  string | whitespace
}

@tokens {

  string { '"' char* '"' }
  char { $[\u{20}\u{21}\u{23}-\u{5b}\u{5d}-\u{10ffff}] | "\\" esc }
  esc { $["\\\/bfnrt] | "u" hex hex hex hex }
  hex { $[0-9a-fA-F] }

  whitespace { $[\n\r\t] }

  "{{" "}}"
}

list<item> { item ("," item)* }

@detectDelim

```

And these tests are passing.

```auto
# Basic

{{"foo"}}

==>

Templates(Object(Stuff))

# Multiple Templates

{{"foo"}}{{"foo"}}

==>

Templates(Object(Stuff), Object(Stuff))

# Spaced out multiple templates

{{"foo"}} {{"foo"}}

==>

Templates(Object(Stuff), Object(Stuff))

# Strings outside templates

"abc" {{"foo"}} "mno" {{"foo"}} "xwz"

==>

Templates(String, Object(Stuff), String, Object(Stuff), String)

```

All I need is to change it so plain text, instead of strings, are permitted inside and outside the templates.

However if I make this slighty change

```auto
Stuff {
  char* | whitespace
}

```

I’m getting a conflict which needs “reducing”.

```auto
shift/reduce conflict between
  Object -> "{{" · "}}"
and
  Stuff -> 
With input:
  "{{" · "}}" …
Shared origin: value -> · Object
  via Object -> "{{" · Stuff+ "}}"
    via Stuff+ -> · Stuff
      Stuff -> ·

```

---

<div class="post-metadata">

### Author: ![marijn](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/marijn/32/136_2.png) [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [January 16, 2023, 8:38pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/2 "2023-01-16T20:38:33Z")

</div>

Have you seen the [Mixed-Language parsing example](https://codemirror.net/examples/mixed-language/)?

---

<div class="post-metadata">

### Author: ![rossm6](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/rossm6/32/3633_2.png) [@rossm6](https://discuss.codemirror.net/u/rossm6)
#### Post date: [January 16, 2023, 9:00pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/3 "2023-01-16T21:00:03Z")

</div>

Thanks for the quick response.

I’ve had a look again at the example. It seems though that this takes for granted an outside parser already and only explains how to then mix it with another. An example of a templating language like what I’m trying to achieve would be helpful. Mine though is perhaps a tad trickier than many because “{{” could be interpreted as plain text or the beginning of a template. On that note I did just try adding in a “precedence” list inside the tokens and that’s now got something working. Whether it’s “right” though I’m not sure. I guess so long as I have a token which contains each template I should be able to get started with now mixing this in with the javascript parser.

Ta

---

<div class="post-metadata">

### Author: ![marijn](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/marijn/32/136_2.png) [@marijn](https://discuss.codemirror.net/u/marijn)
#### Post date: [January 16, 2023, 9:38pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/4 "2023-01-16T21:38:13Z")

</div>

The example [links](https://codemirror.net/examples/mixed-language/twig.grammar.txt) the grammar it uses, which might be useful.

---

<div class="post-metadata">

### Author: ![yswang0927](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/yswang0927/32/3584_2.png) [@yswang0927](https://discuss.codemirror.net/u/yswang0927)
#### Post date: [August 2, 2023, 7:18am UTC](https://discuss.codemirror.net/t/template-style-language/5558/5 "2023-08-02T07:18:18Z")

</div>

Hi rossm6,

do you resolve it? I need to parse like `This is a {{ new Date() }} content with js expression` also.  
Can you help me?

---

<div class="post-metadata">

### Author: ![josh](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/josh/32/1108_2.png) [@josh](https://discuss.codemirror.net/u/josh)
#### Post date: [August 12, 2023, 8:14pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/6 "2023-08-12T20:14:48Z")

</div>

I’m also curious in what your final approach was. My goal was to be able to allow arbitrary text within double braces `{{ }}` within a JSON file as we run the content within those braces as an expression which creates a resulting valid JSON file.

I tried following the [Mixed Language](https://codemirror.net/examples/mixed-language/) example and swapped out HTML → JSON languages within the outer mixed parser (wrapper), but if I load a regular JSON file without a double braced expression `{{ }}`, I get an error message like:

```auto
index.js:1823 CodeMirror plugin crashed: TypeError: Cannot read properties of undefined (reading 'some')
    at hasChild (index.js:1007:26)
    at TreeNode.nextChild (index.js:552:91)
    at TreeCursor.enterChild (index.js:803:42)
    at TreeCursor.firstChild (index.js:813:32)
    at HighlightBuilder.highlightRange (index.js:336:25)
    at HighlightBuilder.highlightRange (index.js:329:26)
    at highlightTree (index.js:267:13)
    at TreeHighlighter.buildDeco (index.js:1672:13)
    at new TreeHighlighter (index.js:1654:33)
    at ViewPlugin.create (index.js:1877:42)

```

I originally used the raw twig.grammar example (with the highlights commented out) and tried simplifying the grammar to just the double braces, but I can’t figure out where I’m going wrong or what I should be looking at for troubleshooting.

> **Simplified \`expression.grammar\`**
>
> ```plaintext
> // Very crude grammar for splitting expressions from regular text
> 
> @top Template { (directive | Text)* }
> 
> directive {
> Expression
> }
> 
> @skip {space} {
> Expression { "{{" ExpressionContent "}}" }
> }
> 
> @tokens {
> Text { ![{] Text? | "{" (@eof | ![%{] Text?) }
> ExpressionContent { ![%}] ExpressionContent? | $[%}] (@eof | ![}] ExpressionContent?) }
> space { @whitespace+ }
> @precedence { space ExpressionContent }
> "{{" "}}"
> }
> 
> ```

> **Simplified \`expressionLanguage.js\`**
>
> ```javascript
> import {parser as expressionParser} from "./expression-parser.js"
> import {jsonLanguage as originalJsonLanguage} from "@codemirror/lang-json"
> import {parseMixed} from "@lezer/common"
> import {LRLanguage} from "@codemirror/language"
> 
> const mixedJsonParser = expressionParser.configure({
> wrap: parseMixed(node => {
> return node.type.isTop ? {
> parser: originalJsonLanguage.parser,
> overlay: node => node.type.name == "Text"
> } : null
> })
> })
> 
> export const expressionLanguage = LRLanguage.define({parser: expressionParser});
> export const jsonLanguage = LRLanguage.define({parser: mixedJsonParser})
> 
> ```

> **Snippet from CM Editor Setup**
>
> ```javascript
> import {expressionLanguage, jsonLanguage as customJsonLanguage } from "./codeMirror/expressionLanguage.js"
> 
> generateEditorExtensions(){
> //the languageSupport is set dynamically based on the selected language
> //we were previously using basic HTML+JS or JS language based on the need
> //we started by layering basic jsonLanguage and are now trying to replace that 
> //with a mixed customJsonLanguage that handles the {{ expressions }} 
> let languageSupport = new LanguageSupport(customJsonLanguage);
> return [
> //keymap, basicSetup,
> languageSupport,
> //oneDark, tabCustomizations, etc
> ]
> }
> 
> //...
> 
> this.editor = new EditorView({
> state: EditorState.create({
> doc: this.value,
> extensions: this.generateEditorExtensions(),
> }),
> parent: this.$refs.editor,
> })
> 
> ```

If I use the `expressionLanguage` export directly, things load fine, but if I use the mixed language `jsonLanguage` export, I get the error mentioned above.

**Edit:** If I use the ‘expressionLanguage’ and it’s associated highlighting _directly_, it _seems_ to be highlighting each part correctly:

![image](https://discuss.codemirror.net/uploads/default/original/2X/3/3ddf7fa790820f45ecee2db01e28eae4478813bd.png)

And the `tree` item in `hasTree()` that seems to be missing the expected `children` property is a TreeBuffer with a `buffer` and `set` (`<NodeSet>`) property.

---

<div class="post-metadata">

### Author: ![rossm6](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/rossm6/32/3633_2.png) [@rossm6](https://discuss.codemirror.net/u/rossm6)
#### Post date: [August 13, 2023, 5:03pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/7 "2023-08-13T17:03:42Z")

</div>

Hi both

Sorry for the delay responding.

I tricked myself into thinking I’d solved this when I claimed this earlier in the thread. To be honest I’ve the feeling the problem I had can’t be solved with the lezer parser at all and so the other way of supporting a language would need to be explored. “Stream parsing” I think is the other way (from memory), which the docs explain is a bit more involved. Things have changed for me though, so I won’t be needing this till later.

If either of you do work out how to solve the problem I mentioned, please do write it up here.

Thanks.

---

<div class="post-metadata">

### Author: ![josh](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/josh/32/1108_2.png) [@josh](https://discuss.codemirror.net/u/josh)
#### Post date: [August 14, 2023, 7:39pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/8 "2023-08-14T19:39:46Z")

</div>

**TLDR** : It ended up being a dependency conflict with my local project.

As part of the troubleshooting process, I put together a minimal reproduction and was surprised to find that it was working there.

> **[CodeMirror Custom Expression Language Tests - StackBlitz](https://stackblitz.com/edit/vitejs-vite-j7kj5u)**
>
> Test stack for CodeMirror custom expression language

> 🕶 The StackBlitz above could be a helpful sandbox if you want to play around with your own grammar. It uses Vite to dynamically rebuild the project when content changes and has a special hook to watch the `.grammar` file and rebuild it if you edit it.

I was really scratching my head as the sample code and package.json dependency definitions all came from my local local project. I started to wonder if it was potentially some other dependency or some part of my local build process that was contributing to the issue

…then I recalled one of the challenges I ran into when I moved from CM5 to CM6 where some old dependencies were cached in node\_modules and were unexpectedly getting re-used.

So I cleared out my node\_modules and `package-lock.json` and performed a fresh install of all the dependencies and lo-and-behold, it started working! 😊 😃

---

<div class="post-metadata">

### Author: ![rossm6](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/rossm6/32/3633_2.png) [@rossm6](https://discuss.codemirror.net/u/rossm6)
#### Post date: [August 14, 2023, 10:03pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/9 "2023-08-14T22:03:49Z")

</div>

Interesting. Thanks for sharing.

I haven’t had chance to check myself but I wonder if it will work mixed with a javascript parser where it understands “{{{}}}” to be an empty object literal.

That’s the thing which made me think it wouldn’t be possible without a stream parser.

I’ll look into it later and report back for anybody else following this thread.

---

<div class="post-metadata">

### Author: ![josh](https://discuss.codemirror.net/user_avatar/discuss.codemirror.net/josh/32/1108_2.png) [@josh](https://discuss.codemirror.net/u/josh)
#### Post date: [August 14, 2023, 10:13pm UTC](https://discuss.codemirror.net/t/template-style-language/5558/10 "2023-08-14T22:13:47Z")

</div>

> [@rossm6](#):
>
> I haven’t had chance to check myself but I wonder if it will work mixed with a javascript parser where it understands “{{{}}}” to be an empty object literal.

No, this grammar wouldn’t cover that, but it’s not a case I needed to worry about for my use-case. Best of luck!
