Nested Highlighting Elements (CM6)

[edbaafi]

Hello!

I have a question regarding nested highlighting elements in CM6.

I have read the docs and see that one can use various “path” endings (/, !, *, or …) to control how nested StyleTags are dealt with (e.g. whether a variable name inside a parameter list gets the class, the parameter list gets the class, or both).

However I believe that there is an important case that is missed by the (/, !, *, or …) syntax and the current implementation: there doesn’t seem to be a way to tell the system that we want a separate DOM element for the outer rule (in this case parameter list).

Say we add the following to the StyleTags definition for the @codemirror/lang-javascript extension and have a HighlightStyle which maps tags.custom_parameters to class="cmt-parameters":

"ArgList/...": tags.custom_parameters,
 "TypeParamList/...":  tags.custom_parameters,
 "ParamList/...": tags.custom_parameters,

This results in all the matches inside the parameter list (e.g. variable name and parentheses) to be wrapped in individual DOM elements with two classes (e.g. class="cmt-parameters cmt-variableName”) rather than for them to have a single cmt-variableName class and to then wrap the whole parameter list in another element with class="cmt-parameters”.

While this may be desired for some use cases, in others it is not. A simple example is adding a subtle background color change when hovering over a parameter list. As is with CSS alone (without a JS hack), we can only change the background on hover of a section of the parameter list.

It seems the Decoration system can deal with this case, as the Underline example here will add such a wrapper element around the parameter list’s contents, but this option just isn’t exposed in the StyleTags definition syntax:

Also I know wrapper DOM elements created by the Decoration system will never be in/on more than one line but this is already built into the API and handled automatically.

With all that said, assuming I haven’t missed an undocumented way of already doing this, would you be open to a PR with an option to allow nested decoration elements rather than adding multiple classes to all the logically nested elements?

Something like <…> to indicate a separate element is created for both the enclosing and nested matches:

 "ArgList/<…>": tags.custom_parameters,
 "TypeParamList/<…>":  tags.custom_parameters,
 "ParamList//<…>": tags.custom_parameters,

Thanks for your consideration on this or any pointers to how this can already be done with the highlight extension!

-Ed

[marijn]

Even if the highlight extension were to support this, the hover trick wouldn’t be reliable (if another extension adds a lower-precedence decoration in the parameter list, that’d break it in two, and it would not be able to span across lines due to basic DOM structure constraints). So my position is that hover effects like that have to be explicitly created by arranging for a new decoration to be added on hover, and doing it through plain CSS is just not workable.

[edbaafi]

Thanks for the quick reply!

Unfortunately the hover example was just one example I thought would help explain the need for wrapper elements. In the end it’s the same need to create wrapper elements, it’s just we need a way to do this on every update not only on hover. So if we change your advice from:

arranging for a new decoration to be added on hover

to:

arranging for a new decoration to be added/maintained on every editor update

What is the official way to accomplish this? It would seem that we should be able to do this through an extension added at the end of the array passed to EditorState.create such that this extension will apply its wrappers after all the nodes it wraps are defined.

I have gotten something like this working except I have resorted to a hack where the extension’s update(value,transaction) (extension is implemented as a StateField) toggles between returning a RangeSet with the desired decorations (to create the wrapper elements) and an empty RangeSet. If we don’t alternately return the empty RangeSet, the original highlight decorations will break up the wrapper elements so they are no longer wrapping their desired children. But if update returns an empty RangeSet, this seems to “flush” things and scheduling a call to editorViewInstance.dispatch() on next tick then triggers the update that actually returns the RangeSet with the desired decorations.

Seems to work, but there must be a better way!

Perhaps I’m not explaining the need well but since “any attempt to add attributes or change the structure of nodes will usually just lead to the editor immediately resetting the content back to what it used to be” we need a way to work with the decoration system to create any desired wrapper elements whether these wrappers are created on hover or on every editor update. Of course, our expectations on where we can place wrapper elements is constrained by the “no wrapper may span multiple lines” caveat.

Thanks for your help on this!
Best,
Ed

[marijn]

Could you provide a simplified example of the code where the empty rangeset hack is necessary? That sounds very odd.

[edbaafi]

Thanks! I found a related issue that’s either a bug or my understanding of what decorations/markings should do is wrong so let’s start with that.

This example takes the code test(1); and tries to create an outer wrapper around the parameter list including the parentheses (1)and an inner wrapper around the parameter: the number literal 1

If you use the first (uncommented) extensions array, you get the desired results:

<div class="cm-activeLine cm-line">test<span wrapper="outer">(<span wrapper="inner"><span class="ͼc">1</span></span>)</span>;</div>

If you comment that extension array and uncomment one of the others, the outer wrapper gets broken up and even becomes a child of inner:

<div class="cm-activeLine cm-line">test<span wrapper="outer">(</span><span wrapper="inner"><span wrapper="outer"><span class="ͼc">1</span></span></span><span wrapper="outer">)</span>;</div>

I think this shows that the order that the markings are applied leads to varied results and would explain why “flushing” the previously applied markings was needed to get consistent/expected results from the highlighter. What I would expect is that the marking system only breaks up a node if it is required based on all requested markings (i.e. an overlap between marking requests that cannot be handled by simply wrapping one in the other). This would require markings to be applied after all extensions/plugins have finalized their marking requests but I haven’t learned the codebase enough to track down if this is what you do.

Here’s the example code:

import { EditorState, basicSetup } from "@codemirror/basic-setup";
import { javascript } from "@codemirror/lang-javascript";
import { EditorView, Decoration } from "@codemirror/view";
import { StateField } from "@codemirror/state";
import { RangeSetBuilder } from "@codemirror/rangeset";

const code = "test(1);";
const outerMark = Decoration.mark({ attributes: { wrapper: "outer" } });
const innerMark = Decoration.mark({ attributes: { wrapper: "inner" } });
const innerArgs = [5, 6, innerMark];
const outerArgs = [4, 7, outerMark];

const update = (state, rangeBuilderArgsArray) => {
  let rangeBuilder = new RangeSetBuilder();
  if (state.doc.toString().startsWith(code)) {
    for (let rangeBuilderArgs of rangeBuilderArgsArray) {
      rangeBuilder.add(...rangeBuilderArgs);
    }
  }
  return rangeBuilder.finish();
};

const innerField = StateField.define({
  create(state) {
    return update(state, [innerArgs]);
  },
  update(parameters, transaction) {
    return update(transaction.state, [innerArgs]);
  },
  provide: (f) => EditorView.decorations.from(f),
});

const outerField = StateField.define({
  create(state) {
    return update(state, [outerArgs]);
  },
  update(parameters, transaction) {
    return update(transaction.state, [outerArgs]);
  },
  provide: (f) => EditorView.decorations.from(f),
});

const innerOuterField = StateField.define({
  create(state) {
    return update(state, [outerArgs, innerArgs]);
  },
  update(parameters, transaction) {
    return update(transaction.state, [outerArgs, innerArgs]);
  },
  provide: (f) => EditorView.decorations.from(f),
});

new EditorView({
  state: EditorState.create({
    doc: code,

    //this one works
    extensions: [ javascript(),basicSetup, innerField, outerField]

    //these ones don't
    //extensions: [ javascript(),basicSetup,innerOuterField]
    //extensions: [javascript(), basicSetup, outerField, innerField],
  }),
  parent: document.body,
});

[edbaafi]

Also I just realized that the expectation that nodes are only broken up if absolutely necessary (based on information from all extensions), doesn’t address which node wraps which node when two decoration’s from and to are the same. In this case I think whichever was added first being wrapped by one added later would make sense. i.e. if two markings describe the same range, an earlier extension/plugin’s marking would be wrapped by a later one’s. And if added in the same extension, a marking added in an earlier call to RangeSetBuilder’s add method would get wrapped by one added in a later call.

[marijn]

@edbaafi

I think this shows that the order that the markings are applied leads to varied results

Yes, that is by design. The ordering of extensions determines their precedence, and nesting happens when a lower-precedence mark exists inside a higher-precedence once.

@edbaafi

This would require markings to be applied after all extensions/plugins have finalized their marking requests but I haven’t learned the codebase enough to track down if this is what you do.

Range sets are just values, so there is no ‘marking requests’, there’s just an ordered array of range sets as provided by the active extensions.