Help resolving dependency conflicts between Lezer and CodeMirror

colin0325 · July 4, 2025, 3:21pm

I am using v6 bundled with Vite. I am trying to create an editor with syntax highlighting for a custom language using Lezer. To test, I first created a Javascript editor, which worked just fine after a bit to tinkering. However, when I build my custom language parser using lezer-generator I am getting the classic Unrecognized extension value in extension set ((type) => {let result = match(type); return result === void 0 ? null : [this, result];}). This sometimes happens because multiple instances of @codemirror/state are loaded, breaking instanceof checks. error. I suspect that somehow @codemirror/state is embedded inside my parser, but I’m not familiar enough to say for sure. I really just need to know how to reconcile all of the Lezer dependencies with all of the CodeMirror dependencies.

main.js

import { EditorView, basicSetup } from "codemirror";
import { EditorState } from "@codemirror/state";

import { LRLanguage, LanguageSupport } from "@codemirror/language";
import { styleTags, tags } from "@lezer/highlight";

import { parser } from "./lang.js"; // generated parser

const myHighlighting = styleTags({
  "=": tags.operator,
  "+": tags.operator,
  "{": tags.brace,
  "}": tags.brace,
  "[": tags.squareBracket,
  "]": tags.squareBracket,
  ":": tags.punctuation,
});

const myLang = LRLanguage.define({ parser });

function myLangSupport() {
  return new LanguageSupport(myLang, [myHighlighting]);  // error here
}

const state = EditorState.create({
  doc: "console.log('Hello world');",
  extensions: [
    basicSetup,
    myLangSupport()
  ]
});

const view = new EditorView({
  state,
  parent: document.querySelector("#editor")
});

package.json

{
  "name": "edu",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview"
  },
  "devDependencies": {
    "vite": "^7.0.0"
  },
  "dependencies": {
    "codemirror": "^6.0.0",
    "@codemirror/language": "^6.11.2",
    "@codemirror/state": "6.5.2",
    "@codemirror/theme-one-dark": "^6.1.3",
    "@codemirror/view": "^6.38.0",
    "@lezer/common": "^1.2.3",
    "@lezer/lr": "^1.4.2",
    "@lezer/generator": "^1.8.0"
  },
  "overrides": {
    "@codemirror/state": "6.5.2"
  },
  "optimizeDeps": {
    "exclude": [
      "codemirror",
      "@codemirror/commands",
      "@codemirror/language",
      "@codemirror/state",
      "@codemirror/view",
      "@lezer/highlight",
      "@lezer/generator"
    ]
  }
}

marijn · July 4, 2025, 3:39pm

You can use npm ls -a to see what’s installed and why. There should never be duplicate versions of any @lezer or @codemirror packages. Also, you cannot just symlink in your custom language package, if it has its own locally installed dependencies.

colin0325 · July 4, 2025, 3:47pm

Here’s npm ls -a. The top level @codemirror/state getting overridden is interesting, although it seems like it’s still using 6.5.2 everywhere else. I don’t see anything that isn’t deduped, but maybe I missed something.

Not sure what’s wrong with the local language package, I built it in the same npm environment as the CodeMirror code. Am I supposed to build it elsewhere and then move it? That seems unnecessary.

myproject@0.0.0 /path/to/myproject
├─┬ @codemirror/language@6.11.2
│ ├── @codemirror/state@6.5.2 deduped
│ ├── @codemirror/view@6.38.0 deduped
│ ├── @lezer/common@1.2.3 deduped
│ ├─┬ @lezer/highlight@1.2.1
│ │ └── @lezer/common@1.2.3 deduped
│ ├── @lezer/lr@1.4.2 deduped
│ └── style-mod@4.1.2
├─┬ @codemirror/state@6.5.2 overridden
│ └── @marijn/find-cluster-break@1.0.2
├─┬ @codemirror/theme-one-dark@6.1.3
│ ├── @codemirror/language@6.11.2 deduped
│ ├── @codemirror/state@6.5.2 deduped
│ ├── @codemirror/view@6.38.0 deduped
│ └── @lezer/highlight@1.2.1 deduped
├─┬ @codemirror/view@6.38.0
│ ├── @codemirror/state@6.5.2 deduped
│ ├── crelt@1.0.6
│ ├── style-mod@4.1.2 deduped
│ └── w3c-keyname@2.2.8
├── @lezer/common@1.2.3
├─┬ @lezer/generator@1.8.0
│ ├── @lezer/common@1.2.3 deduped
│ └── @lezer/lr@1.4.2 deduped
├─┬ @lezer/lr@1.4.2
│ └── @lezer/common@1.2.3 deduped
├─┬ codemirror@6.0.2
│ ├─┬ @codemirror/autocomplete@6.18.6
│ │ ├── @codemirror/language@6.11.2 deduped
│ │ ├── @codemirror/state@6.5.2 deduped
│ │ ├── @codemirror/view@6.38.0 deduped
│ │ └── @lezer/common@1.2.3 deduped
│ ├─┬ @codemirror/commands@6.8.1
│ │ ├── @codemirror/language@6.11.2 deduped
│ │ ├── @codemirror/state@6.5.2 deduped
│ │ ├── @codemirror/view@6.38.0 deduped
│ │ └── @lezer/common@1.2.3 deduped
│ ├── @codemirror/language@6.11.2 deduped
│ ├─┬ @codemirror/lint@6.8.5
│ │ ├── @codemirror/state@6.5.2 deduped
│ │ ├── @codemirror/view@6.38.0 deduped
│ │ └── crelt@1.0.6 deduped
│ ├─┬ @codemirror/search@6.5.11
│ │ ├── @codemirror/state@6.5.2 deduped
│ │ ├── @codemirror/view@6.38.0 deduped
│ │ └── crelt@1.0.6 deduped
│ ├── @codemirror/state@6.5.2 deduped
│ └── @codemirror/view@6.38.0 deduped
└─┬ vite@7.0.2
  ├── UNMET OPTIONAL DEPENDENCY @types/node@^20.19.0 || >=22.12.0
  ├─┬ esbuild@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/aix-ppc64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/android-arm@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/android-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/android-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/darwin-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/darwin-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/freebsd-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/freebsd-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-arm@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-ia32@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-loong64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-mips64el@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-ppc64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-riscv64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/linux-s390x@0.25.5
  │ ├── @esbuild/linux-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/netbsd-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/netbsd-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/openbsd-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/openbsd-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/sunos-x64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/win32-arm64@0.25.5
  │ ├── UNMET OPTIONAL DEPENDENCY @esbuild/win32-ia32@0.25.5
  │ └── UNMET OPTIONAL DEPENDENCY @esbuild/win32-x64@0.25.5
  ├─┬ fdir@6.4.6
  │ └── picomatch@4.0.2 deduped
  ├── UNMET OPTIONAL DEPENDENCY fsevents@~2.3.3
  ├── UNMET OPTIONAL DEPENDENCY jiti@>=1.21.0
  ├── UNMET OPTIONAL DEPENDENCY less@^4.0.0
  ├── UNMET OPTIONAL DEPENDENCY lightningcss@^1.21.0
  ├── picomatch@4.0.2
  ├─┬ postcss@8.5.6
  │ ├── nanoid@3.3.11
  │ ├── picocolors@1.1.1
  │ └── source-map-js@1.2.1
  ├─┬ rollup@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-android-arm-eabi@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-android-arm64@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-darwin-arm64@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-darwin-x64@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-freebsd-arm64@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-freebsd-x64@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-arm-gnueabihf@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-arm-musleabihf@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-arm64-gnu@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-arm64-musl@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-loongarch64-gnu@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-powerpc64le-gnu@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-riscv64-gnu@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-riscv64-musl@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-s390x-gnu@4.44.2
  │ ├── @rollup/rollup-linux-x64-gnu@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-linux-x64-musl@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-win32-arm64-msvc@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-win32-ia32-msvc@4.44.2
  │ ├── UNMET OPTIONAL DEPENDENCY @rollup/rollup-win32-x64-msvc@4.44.2
  │ ├── @types/estree@1.0.8
  │ └── UNMET OPTIONAL DEPENDENCY fsevents@~2.3.2
  ├── UNMET OPTIONAL DEPENDENCY sass-embedded@^1.70.0
  ├── UNMET OPTIONAL DEPENDENCY sass@^1.70.0
  ├── UNMET OPTIONAL DEPENDENCY stylus@>=0.54.8
  ├── UNMET OPTIONAL DEPENDENCY sugarss@^5.0.0
  ├── UNMET OPTIONAL DEPENDENCY terser@^5.16.0
  ├─┬ tinyglobby@0.2.14
  │ ├── fdir@6.4.6 deduped
  │ └── picomatch@4.0.2 deduped
  ├── UNMET OPTIONAL DEPENDENCY tsx@^4.8.1
  └── UNMET OPTIONAL DEPENDENCY yaml@^2.4.2

marijn · July 4, 2025, 3:56pm

Oh, the error message is coming because you’re passing a non-extension (myHighlighting) as an extension. You’ll want to use the return value of styleTags to configure your parser, not treat it like an editor extension. Using TypeScript can help a lot with this kind of thing, since it’ll point out the type errors.

colin0325 · July 4, 2025, 4:10pm

Yep, that was it. Thank you

For future me:

const myHighlighting = styleTags({
  "=": tags.operator,
  "+": tags.operator,
  "{": tags.brace,
  "}": tags.brace,
  "[": tags.squareBracket,
  "]": tags.squareBracket,
  ":": tags.punctuation,
});

const myLang = LRLanguage.define({
  parser: parser.configure({ props: [myHighlighting] })
});


function myLangSupport() {
  return new LanguageSupport(myLang);
}

const state = EditorState.create({
  doc: "console.log('Hello world');",
  extensions: [
    basicSetup,
    myLangSupport()
  ]
});

colin0325 · July 4, 2025, 4:39pm

Well, I spoke too soon. The error went away, but it syntax highlighting isn’t working. I’ve tried both with and without syntaxHighlighting in the extensions to no avail. I thought that might help since I don’t know exactly which tokens basicSetup is highlighting, but either way I get nothing.

const myHighlightStyle = HighlightStyle.define([
  {tag: tags.operator, color: "#f00"},
  {tag: tags.brace, color: "#0f0"},
  {tag: tags.squareBracket, color: "#00f"}
])

const myHighlighting = styleTags({
  "=": tags.operator,
  "+": tags.operator,
  "{": tags.brace,
  "}": tags.brace,
  "[": tags.squareBracket,
  "]": tags.squareBracket,
});

const myLang = LRLanguage.define({
  parser: parser.configure({ props: [myHighlighting] })
});


function myLangSupport() {
  return new LanguageSupport(myLang);
}

const state = EditorState.create({
  doc: "{ } [ ] + =",
  extensions: [
    basicSetup,
    // syntaxHighlighting(myHighlightStyle),
    myLangSupport()
  ]
});

marijn · July 4, 2025, 4:57pm

You’re only assigning style tags that, in the default highlighting style, don’t get a color. Maybe that’s related.

colin0325 · July 4, 2025, 5:02pm

I thought myHighlightStyle would have fixed that. Either way, if I assign one of the tokens to tags.number it still doesn’t take a color.

marijn · July 4, 2025, 7:57pm

Right, with the highlighter they should have colors. Did you check the parser output to see whether that actually parses the code the way you intend, and has nodes of those types in it?

colin0325 · July 5, 2025, 12:19am

Using parser.parse it seems that I only ever get one child node (starting at position 0), no matter what string I try to parse. This has to be a problem with the parser, right?

colin0325 · July 5, 2025, 11:08pm

I just tried again with a super basic grammar, in case I was setting something up wrong. Even with this grammar I’m still getting only one token.


statement {
  Polynomial
}

Polynomial {
  Term (("+" | "-") Term)*
}

Term {
  NumberLiteral (("*" | "/") NumberLiteral)*
}

@tokens {
  NumberLiteral { $[0-9]+ }
  "+"
  "-"
  "*"
  "/"
  space         { $[ \t\r\n]+ }
}

@skip { space }

marijn · July 6, 2025, 6:27am

How are you checking this? If I compile that grammar (I had to add a @top Program { statement+ } rule to make it compile), and parse the string "1+1", calling toString on the tree gives me Program(Polynomial(Term(NumberLiteral),"+",Term(NumberLiteral))), which looks right.

colin0325 · July 6, 2025, 2:33pm

Ah, copy-paste error; the @top Program { statement+ } was already there. Coming back to it a day later, everything seems to be working. I’m thinking that maybe the JavaScript parser was old compared to the Lezer grammar file. Unfortunately I’ve been tinkering so I don’t really have any of the older files to test. Oh well, either way it seems to be functional now. Thanks for all the support.