Editor driven line wrapping

Heya, I’m from Replit and I’ve been working on an extension for adding indentation preserving line wrapping to our editor. The WIP extension is technically functional, but there are just a lot of fundamental problems I don’t think I can resolve through an extension.

Example (cherry-picked) of our extension in action:

Basically, me and other people at Replit have come up with… maybe 5 or more methods to add this feature and all of them have serious problems. They depend on CSS hacks, widget/mark decorations which require particular DOM layouts, etc. I’ve completely lost confidence in the idea that an extension that does this reliably can be made, as every time we’ve tried an approach we find some “gotcha” with it, such as browser bugs or unexpected interactions with other extensions.

Even if a basic, reliable hanging indent extension is possible, it would still place control of where line breaks are in control of the browser. And with that you only get two choices (basically), which is breaking on every character, or breaking on what the browser considers a “word”. I think most users would rather lines not break in the middle of words, but often the browser makes terrible choices with word based line breaks.

Some examples:

image

image

These examples are somewhat contrived, as in many cases bad line wrapping can just be blamed on the editor being too narrow to even use. What I’m trying to get at is that there is a complete lack of control with line wrapping. If I wanted to make it so that line wrapping treated JS accessors (like the dot in foo.bar) as a word break point, I think I’d have to make mark decorations which displayed them as inline-block elements, which seems a little absurd.

Decorations in general mess with line wrapping, as the browser will treat the edge of an inline-block (and others) as a word break point, which causes some problems. An example scenario would be placing a checkmark widget before a word, with the word being a label. Unless you wrapped both the checkmark and the label word in another decoration, the browser might choose to line wrap directly after the checkmark, which would be confusing.

Indentation preserving line wrapping is one of our most requested features, especially since we had it at one point. Even if we add it back somehow with an extension, I feel that “intelligent” wrapping will then become one of our next most requested features, and that’s another can of worms.

With that all in mind, I ask that you consider making CodeMirror handle line wrapping itself though whatever means, and to make it at least mildly extensible. I know that this is substantial request but I think we’re totally out of options here.

Doing line wrapping properly, taking into account the actual width of the glyphs, bidirectional text, and character types, is far from trivial, and as you probably expected, I’m not very attracted to the idea of increasing the scope of the library to include this.

What went wrong with an approach to hanging indent that builds up line decorations adding text indent?

So there are a lot of approaches we tried, and the current one uses padding-left on the line element, and then wraps the beginning whitespace in a decoration and uses negative margin on that element to shift the first row of a line. This is basically a modification of the technique where you use padding-left and text-indent together.

The issue with the padding-left and text-indent technique is that it has inconsistent behavior with tabs and between browsers. I have no idea why, but if you use tabs instead of spaces, negative margins/indents start to act in a stepwise… pattern? It feels inexplicable to me. It’s not consistent between browsers either.

You can get around this somewhat by doing what my current technique does, which is wrapping the beginning whitespace in an element so that you kind of “isolate” the problematic tab characters, and then you shift that element to offset the first line. However, because this is kind of an abuse of the whole decoration system, it breaks when other extensions (such as a linter) wrap lines or at least a part of the beginning whitespace in a mark decoration.

One of the worst issues with this system is that everything is based off of the document - it has no knowledge of widgets, other mark decorations, etc. As an example, we have our AI programming assistant Ghostwriter, which shows “ghost text” when it’s suggesting something. This text is not part of the document, so it can introduce suggestions which then play badly with our hanging indentation extension. I have no idea how this could be reasonably fixed - I feel like I would have to resort to reading the DOM.

In general, the pattern of “it has bad interactions with other stuff but works great in isolation” is what’s so discouraging about adding this feature. I have no idea how to write good tests for it, because it’s so heavily dependent on layout, CSS, and what other extensions do. It’s possible I just “haven’t found the right solution”, but at this point I don’t trust that I have the knowledge needed to write an extension which can handle the growing list of edge cases.

Also, yeah, I know how large of a request I am making. I legitimately would be happy with even a basic line wrapping system that has limitations - such as monospace only fonts - but I know that’s just me being naive. I really do think it would be an amazing feature, especially if it was extensible. We’re working on making our mobile experience really good and having tight control over line wrapping would be invaluable. It’s up to you though, this is ultimately not a “killer feature” or something.

Oh, I wasn’t aware of the issues around tab and text-indent. Will investigate that a bit more.

How would custom wrapping solve this problem, though? You might be able to get halfway there by setting the editor to not wrap, and inserting <br>s as widgets. That’d cause some confusion in the editor over whether it is wrapped, but it would be a small thing to add a facet that explicitly tells the editor that (right now it checks the white-space style).

Which problem in particular?

I’m down for whatever approach that works and won’t badly interact with other extensions. Ideally it should enable at least implementing hanging indentation, otherwise I don’t see the point. Getting away from CSS “tricks” and actually using the DOM layout would be my preference, simply because it seems way less fragile. You’d know better than me, though.

That’s what I meant with my question—how does custom wrapping help with hanging indentation? You are proposing to build it into the editor core?

I wasn’t able to get the problems with text-indent and tabs to show up. Could you describe those in more detail?

Oh, okay, yeah basically I am. I mean hanging indentation is the most important aspect of this, but also some control over where the line breaks are is important to me as well. If there is a good way to do this outside of the editor core, I don’t know it.

The problems don’t show up on WebKit, but I’m pretty sure they show up on Chrome and Firefox. Try out this Try CodeMirror example, and play with the text-indent property in the theme extension. If you change the text-indent value up or down, the line indented with spaces will shift back and forth as you’d expect, but the line indented with tabs will act strangely - -1ch, -2ch, -3ch, and -4ch all have the exact same affect, but 0ch and -5ch are different. It’s probably based on the tab size.

Huh, it seems like they are trying to align the tab stop to the position where it would be if the text-indent wasn’t there. That is indeed very annoying. We’d really need browsers to support text-align: ... hanging but I guess that is going to take a while.

But no, I really don’t want to implement hanging indent (and/or line wrapping) in the core library, sorry.

Alright, thank you for your time.

This is weird right? I found that it was fixed for me by adding marginLeft: -1px to a decoration on the tabs/spaces… very weird. (I’m currently using text-indent: -Xch with margin-left: +Xch)

I found that to prevent breaking lines you can add a (high precedence!!! because you’ll want it to not be broken up) decoration to the thing you want to not break, and add white-space: nowrap, but that will make that item never line wrap :frowning: