ESM Modules not supported

[tungnnt309]

What version of pkg are you using?

5.11.1

What version of Node.js are you using?

16.19.1

What operating system are you using?

macOS

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node16-macos-x64

Describe the Bug

This issue is mentioned in the original package, but the package is archived, then I found a solution in issue's comment:
vercel#1936

"pkg": {
   "scripts": [
     "node_modules/axios/dist/node/*",
   	[...]
   ]

But it still does not work

Expected Behavior

Compile app without axios (lastest version) error

To Reproduce

A Project that include the lastest version of axios (e.g: v1.6.4)

[robertsLando]

What's the error you see on console when you run the app? I have axios too in my project and using that solution works for me

[manast]

In my case I get these errors to begin with:

> Warning Non-javascript file is specified in 'scripts'.
  Pkg will probably fail to parse. Specify *.js in glob.
  /Users/manuelastudillo/Dev/myapp/node_modules/axios/dist/node/axios.cjs.map
> Warning Babel parse has failed: Missing semicolon. (1:10)

then it continues with the same errors as before:

Warning Failed to make bytecode node18-x64 for file /snapshot/myapp/node_modules/axios/dist/node/axios.cjs.map
> Warning Failed to make bytecode node18-x64 for file /snapshot/myapp/node_modules/axios/index.js
> Warning Failed to make bytecode node18-x64 for file /snapshot/myapp/node_modules/axios/lib/axios.js

and so on...

[robertsLando]

I suggest to try pre-building your application using es-build and then pkg against the single file bundle. Otherwise if you are able to spot the error feel free to submit a PR

[robertsLando]

Let's use this issue for all the others. Seems that starting from nodejs 18.19.0 and for all nodejs 20+ versions some modules that support both esm and cjs are not bundled correctly. This is IMO due to resolve package that is not able to correctly parse package.json exports fields (see issue).

Let's use this issue as a tracker for such issues instead of having multiple open ones.

The error will look like:

Error [ERR_REQUIRE_ESM]: require() of ES Module C:\snapshot\my-yao-pkg\node_modules\@apollo\server\dist\cjs\index.js from C:\snapshot\my-yao-pkg\main.cjs not supported.
index.js is treated as an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which declares all .js files in that package scope as ES modules.
Instead either rename index.js to end in .cjs, change the requiring code to use dynamic import() which is available in all CommonJS modules, or change "type": "module" to "type": "commonjs" in C:\snapshot\my-yao-pkg\node_modules\@apollo\server\package.json to treat all .js files as CommonJS (using .mjs for all ES modules instead).

    at Module.require (pkg/prelude/bootstrap.js:1851:31)
    at Object.<anonymous> (C:\snapshot\my-yao-pkg\main.cjs)
    at Module._compile (pkg/prelude/bootstrap.js:1930:22) {
  code: 'ERR_REQUIRE_ESM'
}

Alternative solutions

Below some solutions I tried and other possible working ones

[recommended] Use node --experimental-require-module option

In most cases adding --options experimental-require-module to pkg command line will solve the issue. This will not work if the imported module uses top level awaits

Using esbuild

Actually a possible solution is to use esbuild to pre-compile the bundle in cjs and then use pkg on the output bundle file. I have a working example of this in this repo: https://github.com/zwave-js/zwave-js-ui/blob/master/esbuild.js

Note that this solution doesn't only solve this issue but also produces very small binaries. That example also handles .node native addons, them require a little monkey patch on the file that is described in the link above. Feel free to add some more context to this.

Using bpkg

Other possible solution is to use bpkg tool instead of esbuild. I still never tried it so if someone does please give me some feedbacks. It should handle node native addons out of the box better then esbuild as it encodes them in base64 into the bundle.

Use Nodejs Single executable

Docs: https://nodejs.org/api/single-executable-applications.html.

I belive a lot in this feature and I think it will be the future of packaging nodejs applications. Anyway it's still experimental, if you would like to give it a try please add your feedback here.

[manast]

The only approach that finally worked for me was to use esbuild, but it was still a huge undertaking, as my project is quite large an complex...

[robertsLando]

@manast tried bpkg ? It should work too and should be easier. If I make it work like I want I could consider adding it to pkg behind an option

[robertsLando]

This may fix all those issues: nodejs/node#51977 (comment)

Ref: https://joyeecheung.github.io/blog/2024/03/18/require-esm-in-node-js/