ADR: CommonJS and ESM decision

[kjugi]

As discussed in our last meeting #320

Summarizing the decision around ESM builds.

[UlisesGascon]

Great work doing the consolidation @kjugi! Thanks ❤️

I added some suggestions regarding tone and context, feel free to discus/accept/reject them ;)

[bjohansebas]

LGTM, With the changes proposed by Ulises

[kjugi]

Thank you @UlisesGascon for your great suggestions! It elevates this document to a new level. It's more mature now ✌️
@ljharb Thank you too for your comments and attention to detail! 🥇

[wesleytodd]

I am good with this as it is more of a "tracking document", but it may end up being something end-users look at and I am not sure it fully satisfies the questions they might have. We should keep an eye out for feedback on if we should make a user facing statement on this at some point, maybe as a blog post geared more toward end users.

[UlisesGascon]

maybe as a blog post geared more toward end users

100% that is something that we can do. Who wants to take leadership? :)

[samuelstroschein]

Chipping in from an "I want to use express dependencies but need ESM" perspective, see pillarjs/path-to-regexp#347.

If the maintenance effort of libraries like path-to-regexp to distribute CJS and ESM does not lead to additional overhead (@blakeembrey in #323 (comment)), why should the desire for express js to consume dependencies as CJS block dependencies from emitting ESM at their will?

Adopting path-to-regexp is a tough choice for me now which leads to:

• consistently pushing for ESM OR
• fork/not adopt path-to-regexp

I assume that experience will be universal across express js dependencies. Again, all under the presumption that distributing ESM in dependencies yields no additional overhead.

[wesleytodd]

I assume that experience will be universal across express js dependencies.

This was a main point of our discussion. It is not clear to me that it should be universal. The one thing we all agreed on is that we MUST ship CJS, but there seems to be no reason to limit packages which want to also ship ESM from doing so. That was the main feedback on this doc from our meeting yesterday IIUC.

[ljharb]

There are inherent costs and dangers in shipping a dual-format package - it exacerbates the likelihood of duplicated (and thus out of sync) state; it's unlikely both formats are being tested (whether they're hand-written, increasing the likelihood of deviations, or transpiled, increasing the likelihood of transformation bugs); it increases the package size (which seems to be important to the same folks that want native ESM).

Also, node can import CJS, and every build tool can translate it to ESM just fine if needed, so I'm still not clear on the value.

[samuelstroschein]

@ljharb hope to share some light on the necessity of ESM.

Ofc you can make the decision to ignore the (minor) group that needs ESM. That's completely OK. Everything is a trade-off.

, so I'm still not clear on the value.

I can't adopt path-to-regexp if it's not ESM because tree-shaking doesn't work pillarjs/path-to-regexp#347 and import errors occur pillarjs/path-to-regexp#346.

it increases the package size (which seems to be important to the same folks that want native ESM).

ESM people care about tree-shaking, not the package size.

[wesleytodd]

Yeah, I agree that I continue to be unclear on the value. That said, we did discuss on the call yesterday that we should expect them to be tested if we are shipping dual mode packages.

[ljharb]

@samuelstroschein right - i'm saying that CJS treeshakes the same as ESM, and if there's a specific tool you're using that doesn't support it, i'd encourage you to advocate they do so.

[wesleytodd]

If someone comes with proof that there is some technical barrier to efficiently using CJS in a real world use case, I would absolutely love to help, but that is not what has happened in any of these threads (see my question in the one linked about treeshaking).

EDIT: I realized this might be unclear in this comment so want to point out that I shared my opinion above that we should not stand against packages shipping dual mode if the folks maintaining those want to.

[bjohansebas]

Btw, the development version of Multer (expressjs/multer#399) is only for ESM, which means that in that development version, it should be converted to CommonJS.

[slagiewka]

For going ESM-only, the elephant in the room is TypeScript. Users wanting to run require(esm) (because their output from tsc is CJS) will need to either set --module nodenext when on Typescript 5.8 OR wait for 5.9 to use --module node20.

More on that here: microsoft/TypeScript#60761

If this doc would consider next major releases that target Node 20+ the summary would be simple: choosing ESM over CJS is a matter of preference. Publishing dual package makes no sense.
However, it can also be a matter of vision. If the express ecosystem wants to be considered modern, it would use modern standards as long as it allows the stability it's renown for.

[ctcpip]

This proposed ADR carves out the question of environment support, so that this ADR can focus on CJS vs ESM rather than runtime/env support.

[ctcpip]

blocking for now until we resolve all issues, and, ideally, settle on the language for a related ADR, which has, in a sense, a broader scope, and should allow us to clean up some of the language here

[dynst]

A switch actually requires more than just dropping support for old Node.js versions.

This block of code in the View constructor is currently dynamically, and synchronously, calling require(arbitrarystring), but in ESM, that kind of arbitrary import would have to be part of an async function, because it would have to use await import(...) syntax instead.

https://github.com/expressjs/express/blob/54af593b739ea44674e4a445efa15b8024f093da/lib/view.js#L75-L88

[jorenbroekema]

There's a fair amount of use cases for ESM builds of libraries such as path-to-regexp which do not follow what seems to be a primary premise of this ADR, which is that the tools are designed for NodeJS use cases.

Perhaps that's true also for path-to-regexp, as it was probably created with a NodeJS environment in mind, but nothing in that package is specifically NodeJS, it's quite agnostic to its environment.
Fortunately there is a browser API (URLPattern) inspired by it for browser usage nowadays, and with 90% global usage browser support as of writing this as well as a polyfill available.

That said, in browser-land, there's a lot of packages still making use of path-to-regexp, usually v6 and below because those versions still published as dual ESM/CJS. There are vulnerabilities (CVE-400 comes to mind) that can only be resolved by migrating to v8 which is CJS-only.

"Why not just use CJS, just bundle..."

I know that in most ecosystems, having a CJS-to-ESM conversion happening in your local dev and build tools is common (Vite being the most popular at this time), that said, there are plenty of ecosystems where the focus is on "browser-compatible out of the box". For example, CDNs will often mirror NPM registry and only handle bare import specifier resolution (or not, and expect you to deal with it via importmap, JSPM is an example of such a CDN), and nothing else, meaning that CJS-only packages are not usable from such CDNs. Especially in corporate environments where microfrontends and heavy CDN usage is quite common, this poses a significant challenge. This is just one example, I just happen to be quite active in frontend environments where a CJS-to-ESM conversion isn't a given, or is actively avoided for toolchain-minimalism or performance-related reasons.

Do what you will with that perspective... just wanted to make sure this is expressed and heard, and with some luck, considered in the decision making around this.

[ctcpip]

Locking so we don't lose conversation in zombie PRs. Please create a new issue or PR for further discussion/iteration if needed.