Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

next-simple chat - ReferenceError: require is not defined in ES module scope, you can use import instead #383

Open
michellekaplan7 opened this issue Apr 25, 2024 · 10 comments

Comments

@michellekaplan7
Copy link

Error when running npm run dev of:

ReferenceError: require is not defined in ES module scope, you can use import instead This file is being treated as an ES module because it has a '.js' file extension and '/Users/michellekaplan/Desktop/practice/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.

@DiegoCao
Copy link
Member

DiegoCao commented Apr 27, 2024

Looking into this!

@michellekaplan7
Copy link
Author

Thank you @DiegoCao! Are you able to replicate this problem? Thanks for your help!

@DiegoCao
Copy link
Member

I've replicated the problem, will try to fix it

@michellekaplan7
Copy link
Author

I've replicated the problem, will try to fix it

@DiegoCao - Any updates on this? Thanks!

@ryanatkn
Copy link

ryanatkn commented May 4, 2024

I was able to work around this by disabling SSR. For SvelteKit this meant adding export const ssr = false; to src/routes/+layout.ts.

@Neet-Nestor
Copy link
Contributor

I've met the same problem. This issues need to be further investigated.

@tqchen
Copy link
Contributor

tqchen commented May 9, 2024

make a high-level remark here. The emscripten generate module have some conditional blocks for nodejs (this is default behavior of emscripten) and it does not work well with some of the bundlers.

if (NODE_JS) {
   require(nxxxx)
} else {
   this is what is useful in ES
}

One way to address this is to write a python script that remove the block in if (NODE_JS), which should be fine as long as we are not running in nodejs.

if (NODE_JS) {. 
    throw Error("This module does not yet work on nodejs env")
} else {
   job as usual
}

@louis030195
Copy link

i just changed the next.config.js, problem solved:

/** @type {import('next').NextConfig} */
const nextConfig = {
    reactStrictMode: true,


    webpack: (config, { isServer }) => {
        // Fixes npm packages that depend on `fs` module
        if (!isServer) {
            config.resolve.fallback = {
                ...config.resolve.fallback, // if you miss it, all the other options in fallback, specified
                // by next.js will be dropped. Doesn't make much sense, but how it is
                fs: false, // the solution
                module: false,
                perf_hooks: false,
            };
        }

        return config
    },
};

export default nextConfig;

@Neet-Nestor
Copy link
Contributor

i just changed the next.config.js, problem solved:

/** @type {import('next').NextConfig} */
const nextConfig = {
    reactStrictMode: true,


    webpack: (config, { isServer }) => {
        // Fixes npm packages that depend on `fs` module
        if (!isServer) {
            config.resolve.fallback = {
                ...config.resolve.fallback, // if you miss it, all the other options in fallback, specified
                // by next.js will be dropped. Doesn't make much sense, but how it is
                fs: false, // the solution
                module: false,
                perf_hooks: false,
            };
        }

        return config
    },
};

export default nextConfig;

This still doesn't work for me. In my case the export will throw error on npm run dev.

(node:43180) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/neet/code/web-llm/examples/next-simple-chat/next.config.js:22
export default nextConfig;
^^^^^^

SyntaxError: Unexpected token 'export'
    at wrapSafe (node:internal/modules/cjs/loader:1389:18)
    at Module._compile (node:internal/modules/cjs/loader:1425:20)
    at Module._extensions..js (node:internal/modules/cjs/loader:1564:10)
    at Module.load (node:internal/modules/cjs/loader:1287:32)
    at Module._load (node:internal/modules/cjs/loader:1103:12)
    at cjsLoader (node:internal/modules/esm/translators:318:15)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:258:7)
    at ModuleJob.run (node:internal/modules/esm/module_job:262:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:474:24)
    at async loadConfig (/Users/neet/code/web-llm/examples/next-simple-chat/node_modules/next/dist/server/config.js:678:36)

And even if I add "type": "module" to package.json, the following error will still occur:

 ⨯ ReferenceError: require is not defined in ES module scope, you can use import instead
This file is being treated as an ES module because it has a '.js' file extension and '/Users/neet/code/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.
    at file:///Users/neet/code/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/lib/index.js:4855:74
    at file:///Users/neet/code/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/lib/index.js:4867:5
    at file:///Users/neet/code/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/lib/index.js:4794:3
    at file:///Users/neet/code/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/lib/index.js:4795:3
    at file:///Users/neet/code/web-llm/examples/next-simple-chat/node_modules/@mlc-ai/web-llm/lib/index.js:5013:2
    at ModuleJob.run (node:internal/modules/esm/module_job:262:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:474:24) {
  page: '/'
}

I believe this is a bundler issue, but somehow this issue only happens to this example but not my another Next.js project. Very weird.

tqchen pushed a commit that referenced this issue May 14, 2024
### Overview
This PR post-processes web-llm's `index.js` by replacing all `new
(require('u' + 'rl').URL)('file:' + __filename).href` with
`"MLC_DUMMY_PATH"`:

```javascript
var _scriptDir = 
(typeof document === 'undefined' && typeof location === 'undefined' ? 
  "MLC_DUMMY_PATH": 
  typeof document === 'undefined' ? 
    location.href : 
    (document.currentScript && document.currentScript.src || new URL('index.js', document.baseURI).href)
);
```
which previously would raise error as shown in
#383 and other issues.

Other occurrences of `"MLC_DUMMY_PATH"` are only in branches of `if
(ENVIRONMENT_IS_NODE)` in runtime, which we do not consider / support as
of now.

We use `"MLC_DUMMY_PATH"` instead of `null` because we do not expect the
value to be used at all. If that is not the case, it would be easier to
debug with `"MLC_DUMMY_PATH"`.

### Details
When building projects that use web-llm with `next` (e.g.
`examples/next-simple-chat`), the **compile time** would complain about
the call for `require()`; runtime does not run into it because
`document` is not `undefined` when evaluating `_scriptDir`. Other
examples, like `examples/chrome-extension`, do not have this issue
because they build with `parcel`, which would fix it for us with
`@parcel/resolver-default`:

![image](https://github.com/mlc-ai/web-llm/assets/53290280/0b9df99a-f80e-4fed-8c19-88deb8aabfbd)

This PR's fix does not affect correctness because, by inspecting
`index.js`, `_scriptDir` is used to populate `scriptDirectory`, which is
used in the function `locateFile()`, which currently is only used for
`wasmBinaryFile` (but `isDataURI(wasmBinaryFile)` would never evaluate
to `false`):

```javascript
function locateFile(path) {
    if (Module["locateFile"]) { return Module["locateFile"](path, scriptDirectory) }
    return scriptDirectory + path
}

if (!isDataURI(wasmBinaryFile)) {
    wasmBinaryFile = locateFile(wasmBinaryFile);
}
```

We also do not remove other `require()` in `index.js` as of now, as from
the current understanding, they would not cause issues -- but we can
come back later when they do.

One observation that is not yet explainable is that, if we set
`"@mlc-ai/web-llm": "^0.2.35",` in
`examples/next-simple-chat/package.json`,
#383 would be observed. However,
if we use `"@mlc-ai/web-llm": "../..",`, no issue is observed -- we are
able to use `require()` in compile time.
@CharlieFRuan
Copy link
Contributor

This error should be addressed in npm 0.2.36. For details please see #397.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants