Hello, I am currently trying to enrich our code editor experience in our browser application by using LSP to try to autocomplete what the user would like to type. Has this been done before with Codemirror 6, and if not, are there any alternatives to this method?
Also curious if maybe Codemirror 6 has some form of autocomplete for Javascript and maybe I just missed it. Codemirror 5 seemed to have this functionality through the use of addons detailed below, but there doesn’t seem to be anything similar in CodeMirror 6.
https://codemirror.net/demo/complete.html
Been stuck on this for a few days, so any help or tips would be greatly appreciated. Thanks!
3 Likes
Supposedly it is possible to run the TypeScript server in the browser, but I haven’t actually tried this. There’s a blog post from someone’s who’s managed to connect a language server to CM6 here, but it doesn’t go into details on running the server.
There’s no JS completion in the lang-javascript package yet. There are some vague plans for integrating scope tracking (for highlighting and completion) into the language module, but I haven’t had time to seriously work on that yet.
Hey, I’ve been able to get this working pretty well (autocomplete, type error lints etc.), so I’m more than happy to help. One of my goals was to build an editor that worked without a backend running the LSP server, so all of this happens in the browser only.
Unfortunately my code isn’t open source (yet), so I cannot point you to a repository, but here’s the overall gist of what I did:
- When you talk about the Typescript LSP, I’m assuming you’re talking about
tsserver, which ships with Typescript and is the “brains” behind the compiler, so to speak. - Don’t try to run
tsserveron your own. While I did manage to do it inside a WebWorker, it’s far too tedious. The Typescript folks maintain @typescript/vfs - npm, which is exactly what you want (and is fully typed!). I would also recommend fetching core TS types from the Typescript CDN usingcreateDefaultMapFromCDN. (I personally had my own CDN setup, but the public TS one works just fine). - It is also worth it to understand
tsservercommands. Their typings are a great place to do that: TypeScript/protocol.ts at main · microsoft/TypeScript · GitHub - Once you get Typescript working, wiring it up to CodeMirror is a matter of using the
ts.languageServerinstance in the correct places. - For linting (showing up type errors as squiggly lines) you’ll want to use the
linterextension from@codemirror/lint. Inside the lint source, I make a call out tots.languageService.getSemanticDiagnostics - For autocomplete, I also found the blog post Marijn linked, but it didn’t prove very useful for my case. Instead, inside the
overridefunction of theautocompleteextension, I make a call out tots.languageService.getCompletionsAtPosition. This is very inefficient though, as the call will happen on every keystroke, debounced (it’s not really noticable for small lists becausetsserveris built to be fast), so I’m still trying to figure out a better way.tsserveralso has something called CompletionEntryDetails, which returns more “details” about an autocomplete item. They’re meant to be used to render details about the currently highlighted autocomplete item, but I’m yet sure how to get this information from CodeMirror, so I also fetchCompletionEntryDetailsfor all autocomplete items usingts.languageService.getCompletionEntryDetails. This makes it even more expensive, so be aware. - You can also use the
hoverTooltipextension from@codemirror/tooltip. In here, I make a call out tots.languageService.getQuickInfoAtPosition. - One last thing to remember is that you need to keep
tsserver's view of the currently open “file” (tsserver deals in terms of files) up-to-date with what you see. To do this, I override thedispatchfunction of the EditorView, and make a call out tots.updateFilefrom there, AFTER updating the view. make sure you debounce this to avoid UI freezes.
I’m still actively working on this TS + CodeMirror powered editor, so some things are far from perfect (and other features are missing), but overall it works pretty well!
A motivating factor for me was looking at Monaco Editor that the TS folks use on the TS Playground (typescriptlang.org/play/). That does not talk to a backend running an LSP server either, so everything they do in there is possible in the browser. Good luck!
10 Likes
Hey @madebysid, thanks for the help, this is fantastic! Had a quick question for you, since I am a still new to language servers, can this approach handle custom classes that the user has created in file, or that we have (i.e. a custom browser class that has a suite of functionality)? I feel this approach would really fit our use case, but I am curious what extra functionality I will need to work on adding to get this where I want it. Thanks again!
I’m not quite sure I follow, but are you talking about injecting custom types? If so, then yeah that’s possible too (I do the same). You can call the createFile method on the tsserver instance to do exactly that!
You’re supposed to give this custom types file a path, and usually you would follow the same conventions you would if you wanted to load custom types in a real Typescript project: so for example if you want to fake an import from a module called abcd, you would call createFile(“/node_modules/abcd/index.d.ts”, “export type X = string”)
Then in your main script/project, you can write “import { X } from ‘abcd’”, and TSServer will see no issue with it. You can do the same for your custom class.
1 Like
Hmmm, I see. I also had a few more questions regarding this topic. First off, how exactly are you replacing the normal autocomplete function with env.languageService.getCompletionsAtPosition()? I have tried just doing override: [env.languageService.getCompletionsAtPosition()] but I am just not sure what to put for fileName, position, and options inside of getCompletionsAtPosition()?
Second, could you explain what you meant by the 8th bullet in the original reply, which is talking about ts.updateFile? I am still just a bit confused on what you mean by overriding the dispatch file? Thanks again for all the help @madebysid, it is very much appreciated!
Did you look at https://www.npmjs.com/package/@typescript/vfs? Its README has full instructions on how to get document highlights for a file.
They create a file called index.ts in the virtual file system with some content, then initialize a virtualTypeScriptEnvironment with that file, that’s all you have to do as well.
Then, instead of getting document highlights, you can call getCompletionsAtPosition (also listed in the README). I would recommend playing around with @typescript/vfs outside CodeMirror just to get the hang of it first.
how exactly are you replacing the normal autocomplete function
You cannot just override: [env.languageService.getCompletionsAtPosition()], because the completion source’s signature won’t match up this way…
This is what my autocompletion config roughly looks like:
autocompletion({
override: [(ctx) => {
const { pos } = ctx;
// tsserver is initialized using @typescript/vfs
const completions = tsserver.languageService.getCompletionsAtPosition(
"index.ts",
pos,
{}
);
if (!completions) {
log("Unable to get completions", { pos });
return null;
}
return completeFromList(
completions.entries.map(c => ({
type: c.kind,
label: c.name,
}))
)(ctx);
}]
})
could you explain what you meant by the 8th bullet in the original reply, which is talking about
ts.updateFile?
Since you initialize tsserver with some file contents, you have to realize that tsserver has its own representation / view of the “file” you’re working with. When you change/type things in your editor, you have to tell tsserver about these changes as well so that the next time you ask something from it (like autocompletions, or type errors), it gives you the correct response. You have to keep your editor’s view in sync with tsserver’s view of the file. That’s what the updateFile method on the tsserver instance does.
In my case, I override the dispatch function in my new EditorView params this way:
const view = new EditorView({
// ...snip
dispatch: transaction => {
// Update view first
this.view.update([transaction]);
// Then tell tsserver about new file. Debounce this, I'm not doing that here for sake of clarity
if (transaction.docChanged) {
const content = doc.sliceString(0);
tsserver.updateFile("index.ts", content);
}
},
})
1 Like
Thanks for all this @madebysid ! I had been working through the documentation throughout the day, but I think my issues stem from a lack of understanding of Codemirror 6 functionality, so gonna spend some time today making sure I understand that. I may have more questions later, but thank you so much for your help thus far! I’ve been stuck on doing something like this for 2 weeks, and this help really steered me in the right direction. Cheers!
Hey @madebysid, first off just wanted to give a few thanks, I was able to get the code completion working at a base level and it is fantastic! Had a couple questions if you are still around on this thread. Firstly, working on incorporating hoverTooltip() and was curious where you declare it, since you need the CompletionContext to get the cursor’s position? Would you need to create an object during the AutoComplete functionality that uses ctx there? If so, I have not gotten it to successfully work when using languageService.getQuickInfoAtPosition and the example seen here CodeMirror Tooltip Example.
Another question is regarding a classes available functionality and displaying that on dropdown. An example would be, if I am typing “console.” in the code editor, I was items like “console.log” to populate in the dropdown, but it is not doing that. Did you run into any issues with this? Again, thank you so much for the help thus far, and I hope to hear from you
since you need the CompletionContext to get the cursor’s position
It’s true that you need the cursor’s position to be able to call getQuickInfoAtPosition, but you don’t necessarily need the CompletionContext to get the cursor position. I use the hoverTooltip extension from @codemirror/tooltip (just like the example you linked), and the callback that you give that extension gets access to the cursor position (In fact the example shows you exactly how). My extension looks like this roughly:
import { hoverTooltip } from '@codemirror/tooltip'
// ..snip
extensions: [
hoverTooltip((view, pos) => {
const quickInfo = tsserver.languageService.getQuickInfoAtPosition(
"index.ts",
pos
);
})
]
As a side note, cursors are just empty selections, so you can easily get them from an EditorState instance even if you don’t have access to the position directly. The first example on the page you linked also shows you how:
state.selection.ranges
.filter(range => range.empty)
.map(range => {
// range.head or range.anchor will be equal, and will both be the cursor position
})
I was items like “console.log” to populate in the dropdown, but it is not doing that
When you type console., does the log autocompletion not show up at all or does it just show up somewhere lower down the list? I personally didn’t see any issues, but I did have to provide a boost parameter to some items based on a completion’s sortText (you get this property for every item inside tsserver.languageService.getCompletionsAtPosition().entries) so that more relevant things show up on the top of the autocomplete list. IIRC the word console. should be enough for the log item to be the top match though (or at least somewhere near the top).
1 Like
To answer the question towards the end, I am just not getting any items at all when typing console., but I think it is something wrong with how I update the tsserver file. Trying to get debouncing to work with it and then I am hoping that solves that issue, I just have never used debouncing in JS specifically. I will try out that for hoverTooltip, thanks once again!
FWIW, I’ve realized that you should be throttling instead of debouncing. You want to keep tsserver’s view of the file as up-to-date as possible, so debouncing just unnecessarily delays the file update. (Lodash has methods for both of them that make this easy)
You don’t HAVE to denounce/throttle BTW, if you’re still debugging, just update the file on tsserver on every keystroke and see if that helps. You can also get tsserver’s current view of the file using the getSourceFile method on the tsserver instance.
I did implement throttling and it makes a huge difference 
. I did have a few more questions. Firstly, for the linter, the Diagnostic[] that is returned is incompatible with what is expected by linter(), so I was curious how exactly you translated one data type to the other? Another question I had was regarding the hover tooltip. Currently whenever I try to return the Tooltip object like in the handy Tooltip Example, it is still erroring, so did you also have to manipulate the results of getQuickInfoAtPosition() in a particular way. I have been working on both for about a day, so I figured it would be worth to ask for clarity. Thanks again for responding to these!!
@madebysid To give more context, it seems like “pos” from the following phrase:
hoverTooltip((view, pos, side) => {
is only a boolean, and we cannot pull out to or from using pos. I have tried looking through view to see if it had anything of use, but I have come up short so far. Not really sure what the issue is here, but figured I would give more context.
how exactly you translated one data type to the other?
Well you can .map from the TSServer version to CodeMirror’s version, here’s a excerpt from my codebase, ignore things that are not relevant:
import { displayPartsToString } from "typescript";
// ..snip
extensions: [
autocompletion({
activateOnTyping: true,
maxRenderedOptions: 30,
override: [
async (ctx: CompletionContext): Promise<CompletionResult | null> => {
const { state, pos } = ctx;
const ts = state.field(tsStateField);
try {
const completions = (await ts.lang()).getCompletionsAtPosition(
ts.entrypoint,
pos,
{}
);
if (!completions) {
log("Unable to get completions", { pos });
return null;
}
return completeFromList(
completions.entries.map((c, i) => ({
type: c.kind,
label: c.name,
// TODO:: populate details and info
boost: 1 / Number(c.sortText),
}))
)(ctx);
} catch (e) {
log("Unable to get completions", { pos, error: e });
return null;
}
},
],
}),
];
And here’s my hover tooltip:
import { displayPartsToString } from "typescript";
// ..snip
extensions: [
hoverTooltip(
async ({ state }: EditorView, pos: number): Promise<Tooltip | null> => {
const ts = state.field(tsStateField);
const quickInfo = (await ts.lang()).getQuickInfoAtPosition(
ts.entrypoint,
pos
);
if (!quickInfo) {
return null;
}
return {
pos,
create() {
const dom = document.createElement("div");
dom.setAttribute("class", "cm-quickinfo-tooltip");
dom.textContent =
displayPartsToString(quickInfo.displayParts) +
(quickInfo.documentation?.length
? "\n" + displayPartsToString(quickInfo.documentation)
: "");
return {
dom,
};
},
};
},
{
hideOnChange: true,
}
),
];
it seems like “pos” from the following phrase:
hoverTooltip((view, pos, side) => {
is only a boolean
That doesn’t sound right, I use the pos from the hoverTooltip (as I’ve shown above), are you sure you don’t have a typo somewhere?
I am a bit confused, what exactly is tsStateField? I am trying to find something with the same data type, but have since been unsuccessful. I tried the code and just using a test string as the dom.textContent, but it still is not populating, which is strange. The function is being run, since my console statement is hit every time I hover, but the item is not correctly being created.
@madebysid At the current state, it seems as though hoverTooltip just never gets run, as I have added some print statements and even those have not run. My initial issue was fixed with an update to all Codemirror packages. Its especially odd, since all of my other extensions run no problem. I am still not sure what tsStateField is, but even just using a test string, it still does not behave correctly.
what exactly is tsStateField?
Ah that’s my bad for including it, it’s just the instance of tsserver that I store as a state field so I can access it everywhere. I’m not sure what your setup looks like, so just see it as the tsserver instance from the @typescript/vfs NPM package.
I’m not sure how to help you debug your code, I suppose you’ll just have to read the reference / docs. The hoverTooltip example I showed you uses the hoverTooltip import from the @codemirror/tooltip package. If you do it step-by-step, one thing at a time, you should be able to at least get the hover tooltips working without tsserver first, then add tsserver calls. It seems in your case that the tooltips themselves aren’t working (which I never really saw), so you should debug that first.
@madebysid Out of curiosity, what version of “@codemirror/tooltip” are you using? I updated to the newest version of each dependency, and I am curious if maybe that’s the issue. Once again, thanks for all your help!
I’m on @codemirror/tooltip@0.18.4 right now