Proposal: Coarser Module Structure

v6
1

The ‘core’ library (not including the language packages) currently consists of 24 different packages. Setups usually don’t need all of them, but you’ll still easily end up with almost 20 of them, plus language support, for a typical editor project.

I really like the way this allows for tiny, separate codebases, makes sure I design package interfaces that actually allow 3rd party code to implement similar extensions, and produces very focused changelogs and releases. And if I at some point decide that, for example, the tooltip interface should work completely differently, I could release a new major version of @codemirror/tooltip without impacting the other packages.

Also, a structure like this encourages an ecosystem of 3rd party extensions, by giving them a similar shape to the ‘official’ packages. This is important to me, since I have no taste or energy to maintain every possible language and plugin myself as part of the core library.

But for client developers, the experience of setting up an editor and having to chase down 20 packages and figure out what to import from where is … not great. The long, flat list of package names in the reference guide’s table of contents is a pain even for me myself to navigate.

On top of that, npm often duplicates packages when you upgrade stuff and you have multiple things that depend on them, causing constant confusion.

So, as part of the things to take care of before locking down the API for a stable release, I have been thinking about package structure again. People have asked if the project can become a single package/repository, but I really don’t like that—it suggests this is a monolithic library, which is very much isn’t, and ties updates to all the various extensions together in big blobs of unrelated changes. Language packages should definitely be separate (having heaps of—often poor quality—contributed language modes in the main distribution almost burned me out on CodeMirror 5), so installing CodeMirror is never going to be a case of just linking in a single script tag or referencing some CDN script.

Given all that, what I’m currently leaning towards is to move a bunch of the smaller packages into other packages, to get a structure that is still modular, but not annoyingly modular. The interface as a whole of course doesn’t get smaller, but needing to install and import from fewer different packages should reduce cognitive load for developers somewhat.

The list below shows the design I am now looking at, with the remaining (8) core packages as top level items, and the packages that are being merged into them as sub-items.

  • state
    • text
    • rangeset
    • collab
  • view
    • tooltip
    • panel
    • gutter
    • rectangular-selection
    • matchbrackets
  • language
    • stream-parser
    • fold
  • autocomplete
    • closebrackets
  • commands
    • comment
    • history
  • language-data
  • search
  • lint

The exports from merged packages would remain the same, they’d just move to a different package, making the overall effort needed for porting your code over to this relatively small.

Before I pull the switch on this I’d be interested in feedback, so if you have any thoughts or questions, please reply here.

1 Like
2

Generally I like the idea of having less packages.

You could have a look at how preact splits up itself into chunks, while remaining a monorepository.

The basic idea is to have sub-directories ship with sub-libraries.

If you follow that line you could have all the benefits of single packages + a mono-repository infrastructure that just works.

For consumers they would import codemirror/state and get the latest and greatest from it. I personally favor that approach.

3

I don’t want a monorepo. With the interface stable, commits that have to span multiple packages should be really rare. As for distributing a single package, I think I outlined pretty well why I don’t favor that in the original post.

4

With y-codemirror.next I’d like to replace the default rangeset, history, and collab implementations with data structures that work on the Yjs CRDT. I feel these modules shouldn’t be part of the state & commands package which are used by everyone.

Another approach would be to keep the existing package structure and publish a single meta package that would install all “default” packages as dependencies and expose them using subpath exports. It would look like this:

// importing only a single module
import * as state from `codemirror/state`

// import several modules in a single call
import { state, view } from `codemirror`

Importing several modules in a single call works if your main export looks like this:

import * as state from `@codemirror/state`
import * as view from `@codemirror/view`
...

export state
export view
...

I used this approach in my lib0 package and module bundlers (rollup & webpack) are still able to eliminate unused functions. If this sounds interesting to you I’m happy to verify that this works with other popular bundlers as well.

That would make people happy that only want to import a single package and it would allow others to repackage CodeMirror to only use what they really need.

5

Coarser packages (plus maybe a meta package, an idea I really dig) would definitely make end user quality of life / ease of adoption simpler, IMO.

2 Likes
6

I’ll take a few steps back and think about things that are probably quite obvious to you :slight_smile: So, there’s two main groups of users of these interfaces – extensions and deployments. If I remember correctly you tend to consider these lines to be rather blurred, given that people often write custom code for their deployments (now hopefully in the form of extensions), but it helps me to look at the question from these two perspectives.

Extensions probably mostly don’t want to depend on view if they don’t have to (although you probably wouldn’t split an extension in view and non-view parts) and have somewhat fine-grained dependencies so that they can ignore unrelated major version changes. In general, an extension has to consider two environments: The source code tree with declared dependencies, and the editor instance it runs in. Optional dependencies are usually only optional on the editor instance level, but will still pull in the code (btw, what is your current solution for making sure that objects extensions receive are from the version they declared compatibility with or even instanceof their dependency?)

For deployments, I can come up with four tiers of required-ness (I probably miscategorized a few):

  1. always: state, text, rangeset, view, panel
  2. almost always: gutter, rectangular-selection, matchbrackets, language, stream-parser, commands, history, search
  3. often: tooltip, fold, autocomplete, closebrackets, comment, language-data, lint
  4. sometimes: collab

From a deployment point of view, some of these packages look like optional, language-specific features (lint, comment, closebrackets, fold), others replicate expected native behavior (search, history, maybe rectangular-selection and matchbrackets). collab is a highly specific feature with non-trivial integration cost. These are all feature-packages: Enabled and correctly configured, they provide some feature. Everything else (state, text, rangeset, view, tooltip, panel, gutter, language, stream-parser, commands, , autocomplete) you might need for the editor or some extension to work but it doesn’t by itself communicate a useful feature.

Ok, so, what do I think. First, collab is really specific and I would really not want to have a major version bump on state because of a breaking change in collab. I think it should stay separate, unless you are really sure there won’t be a breaking change :slight_smile: commands, history and search are part of the language-independent editor core your new state and view packages represent. comment, closebrackets, language, stream-parser, fold and language-data are all language-dependent. lint is another separate thing with considerable integration requirements. Put together: consider putting commands, history and search »closer« to the core; move comment to language; move collab out of state; move autocomplete to view and closebrackets to language.

7

I assume a tree-shaking bundler in this design (and already in the current setup, where for example view contains a bunch of utilities that not everyone will use). So even if you don’t need them, there should be no harm in having them in modules that you use parts of.

Well, except for RangeSet. I don’t think you’ll be able to replace that, since it is a direct dependency of a range of packages, and not something the library allows you to swap out.

I’ve had rather poor experiences with tooling support for subpath exports in the past, and don’t really want to gamble on those yet.

8

Thanks for reponding!

That makes sense. Since most setups won’t need this at all, and those that do will need to tightly integrate with it anyway, I guess there’s little cost to it being its own package.

Comment only uses the generic languageDataAt mechanism defined in @codemirror/state, though.

What would that look like?

Autocomplete is a large package, though, so it feels like it deserves its own package scope.

9

:slight_smile:

But it will almost always be used together with languages.

I don’t know. It’s a mixed bag. Maybe a @codemirror/core that depends on @codemirror/view and @codemirror/state and directly includes commands, history, search and autocomplete.

10

Another thing that I think would help with initial client setup is allowing the basic-setup package to pass configuration through to child packages, similar to tiptap’s starter-kit.

Right now, if you need any non-default configuration, it requires abandoning the single-dependency/single-import and then tracking down 20 tiny packages again.

11

You can just import the extension you want to configure, and add a configured version of it to your state, so I don’t think this is much of a problem.

As for omitting some of the extensions, yes, that will require you to build up your own configuration, but given that you wouldn’t be able to tree-shake the extra code out of your bundle if extensions could be omitted dynamically, and that there’ll be significantly less than 20 packages to add as dependencies in the future, I think that’s acceptable.

12

First of all, I’m surprised you consider things like tooltip to be “core.” I thought core to be only state and view. Semantics.

I appreciate you querying the community. We definitely have the issue at Replit with multiple package versions being installed, then we have to surgically fix yarn.lock. That said, I personally don’t mind it.

The only grouping I see that is clear-cut to me is text being part of state. In fact, I almost always import Text directly from the state package. Many state APIs deal with Text and not some interface Text that you can specify and pass along as the editor’s document data structure, but explicitly the class from CodeMirror.

I think the status quo is a little inconsistent. For example, why is gutter separate but drawSelection, dropCursor, placeholder…etc bundled with view. What you’re suggesting right now is at least consistent.

Regarding the reference guide, I wish the nav was based on top-level exports rather than the packages themselves. Ultimately, I don’t really care what package something is in.

Finally, this is mainly about the ergonomics of maintainers and consumers. A meta-package like @codemirror/kitchensink that re-exports everything can be automatically updated and published on every sub-package bump. Most consumers will prefer the kitchen sink, those who don’t can use the fragments. You, as the creator, seem to like the many packages. This way, we get the best of both worlds.

13

But then wouldn’t it contain hundreds and hundreds of items and become completely impossible to navigate?

That might indeed help. Do you know of any other library that is doing something like this? Would you expect the re-exports to be subpaths or * as x exports from a single module?