Errors in node_modules/@crawlee/http/internals/http-crawler.d.ts 140 errors in the same file

[jawspeak]

Which package is this bug report for? If unsure which one to select, leave blank

None

Issue description

This seems related to #2279

Running tsc I get errors including:

...

node_modules/@crawlee/http/internals/http-crawler.d.ts:419:25 - error TS1109: Expression expected.

419         methodRewriting?: boolean | undefined;
                            ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:421:28 - error TS1109: Expression expected.

421         dnsLookupIpVersion?: import("got-scraping", { with: { "resolution-mode": "import" } }).DnsLookupIpVersion;
                               ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:423:19 - error TS1109: Expression expected.

423         parseJson?: import("got-scraping", { with: { "resolution-mode": "import" } }).ParseJsonFunction | undefined;
                      ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:425:23 - error TS1109: Expression expected.

425         stringifyJson?: import("got-scraping", { with: { "resolution-mode": "import" } }).StringifyJsonFunction | undefined;
                          ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:426:22 - error TS1109: Expression expected.

426         localAddress?: string | undefined;
                         ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:427:16 - error TS1109: Expression expected.

427         method?: Method | undefined;
                   ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:429:26 - error TS1109: Expression expected.

429         createConnection?: import("got-scraping", { with: { "resolution-mode": "import" } }).CreateConnectionFunction | undefined;
                             ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:431:22 - error TS1109: Expression expected.

431         cacheOptions?: import("got-scraping", { with: { "resolution-mode": "import" } }).CacheOptions | undefined;
                         ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:433:15 - error TS1109: Expression expected.

433         https?: import("got-scraping", { with: { "resolution-mode": "import" } }).HttpsOptions | undefined;
                  ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:434:18 - error TS1109: Expression expected.

434         encoding?: BufferEncoding | undefined;
                     ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:435:25 - error TS1109: Expression expected.

435         resolveBodyOnly?: boolean | undefined;
                            ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:436:18 - error TS1109: Expression expected.

436         isStream?: boolean | undefined;
                     ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:438:22 - error TS1109: Expression expected.

438         responseType?: import("got-scraping", { with: { "resolution-mode": "import" } }).ResponseType | undefined;
                         ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:440:20 - error TS1109: Expression expected.

440         pagination?: import("got-scraping", { with: { "resolution-mode": "import" } }).PaginationOptions<unknown, unknown> | undefined;
                       ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:441:17 - error TS1109: Expression expected.

441         setHost?: boolean | undefined;
                    ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:442:23 - error TS1109: Expression expected.

442         maxHeaderSize?: number | undefined;
                          ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:443:27 - error TS1109: Expression expected.

443         enableUnixSockets?: boolean | undefined;
                              ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:444:5 - error TS1128: Declaration or statement expected.

444     } & {
        ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:444:7 - error TS1109: Expression expected.

444     } & {
          ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:446:98 - error TS1109: Expression expected.

446         hooks?: Partial<import("got-scraping", { with: { "resolution-mode": "import" } }).Hooks> | undefined;
                                                                                                     ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:446:109 - error TS1005: ',' expected.

446         hooks?: Partial<import("got-scraping", { with: { "resolution-mode": "import" } }).Hooks> | undefined;
                                                                                                                ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:448:105 - error TS1109: Expression expected.

448         retry?: Partial<import("got-scraping", { with: { "resolution-mode": "import" } }).RetryOptions> | undefined;
                                                                                                            ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:448:116 - error TS1005: ',' expected.

448         retry?: Partial<import("got-scraping", { with: { "resolution-mode": "import" } }).RetryOptions> | undefined;
                                                                                                                       ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:451:23 - error TS1005: ',' expected.

451         isStream: true;
                          ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:453:5 - error TS1128: Declaration or statement expected.

453     protected _encodeResponse(request: Request, response: IncomingMessage, encoding: BufferEncoding): {
        ~~~~~~~~~

node_modules/@crawlee/http/internals/http-crawler.d.ts:453:38 - error TS1005: ',' expected.

453     protected _encodeResponse(request: Request, response: IncomingMessage, encoding: BufferEncoding): {
                                         ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:453:57 - error TS1005: ',' expected.

453     protected _encodeResponse(request: Request, response: IncomingMessage, encoding: BufferEncoding): {
                                                            ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:453:84 - error TS1005: ',' expected.

453     protected _encodeResponse(request: Request, response: IncomingMessage, encoding: BufferEncoding): {
                                                                                       ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:453:101 - error TS1005: ';' expected.

453     protected _encodeResponse(request: Request, response: IncomingMessage, encoding: BufferEncoding): {
                                                                                                        ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:460:5 - error TS1128: Declaration or statement expected.

460     protected _extendSupportedMimeTypes(additionalMimeTypes: (string | RequestLike | ResponseLike)[]): void;
        ~~~~~~~~~

node_modules/@crawlee/http/internals/http-crawler.d.ts:460:60 - error TS1005: ',' expected.

460     protected _extendSupportedMimeTypes(additionalMimeTypes: (string | RequestLike | ResponseLike)[]): void;
                                                               ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:460:100 - error TS1011: An element access expression should take an argument.

460     protected _extendSupportedMimeTypes(additionalMimeTypes: (string | RequestLike | ResponseLike)[]): void;
                                                                                                       

node_modules/@crawlee/http/internals/http-crawler.d.ts:460:102 - error TS1005: ';' expected.

460     protected _extendSupportedMimeTypes(additionalMimeTypes: (string | RequestLike | ResponseLike)[]): void;
                                                                                                         ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:460:108 - error TS1109: Expression expected.

460     protected _extendSupportedMimeTypes(additionalMimeTypes: (string | RequestLike | ResponseLike)[]): void;
                                                                                                               ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:464:5 - error TS1128: Declaration or statement expected.

464     protected _handleRequestTimeout(session?: Session): void;
        ~~~~~~~~~

node_modules/@crawlee/http/internals/http-crawler.d.ts:464:45 - error TS1109: Expression expected.

464     protected _handleRequestTimeout(session?: Session): void;
                                                ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:464:55 - error TS1005: ';' expected.

464     protected _handleRequestTimeout(session?: Session): void;
                                                          ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:464:61 - error TS1109: Expression expected.

464     protected _handleRequestTimeout(session?: Session): void;
                                                                ~

node_modules/@crawlee/http/internals/http-crawler.d.ts:465:5 - error TS1128: Declaration or statement expected.

465     private _abortDownloadOfBody;
        ~~~~~~~

node_modules/@crawlee/http/internals/http-crawler.d.ts:469:5 - error TS1128: Declaration or statement expected.

469     private _requestAsBrowser;
        ~~~~~~~

node_modules/@crawlee/http/internals/http-crawler.d.ts:470:1 - error TS1128: Declaration or statement expected.

470 }
    ~


Found 140 errors in the same file, starting at: node_modules/@crawlee/http/internals/http-crawler.d.ts:372

To reproduce:

1. rm -rf package-lock.json node_modules
2. edit and remove in package.json crawlee, playwright*, puppeteer-extra-plugin-stealth, @crawlee/types
3. npm install
4. npm install crawlee playwright playwright-extra puppeteer-extra-plugin-stealth @crawlee/types
5. tsc

Note this is my tsconfig, note how it has exclude for node_modules as well as skipLibCheck true (though may not apply because of microsoft/TypeScript#41883)

 tsc --showConfig     
{
    "compilerOptions": {
        "target": "esnext",
        "module": "nodenext",
        "moduleResolution": "nodenext",
        "sourceMap": true,
        "outDir": "./dist",
        "esModuleInterop": true,
        "forceConsistentCasingInFileNames": true,
        "allowUnreachableCode": false,
        "noFallthroughCasesInSwitch": true,
        "noImplicitReturns": true,
        "strict": true,
        "skipLibCheck": true,
        "types": [
            "reflect-metadata"
        ],
        "lib": [
            "esnext",
            "dom"
        ],
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true,
        "allowImportingTsExtensions": false
    },
    "files": [
        <redacted>
    ],
    "include": [
        "src/**/*"
    ],
    "exclude": [
        "node_modules/**",
        "node_modules",
        "dist/**"
    ]
}

When I run npm install crawlee playwright playwright-extra puppeteer-extra-plugin-stealth @crawlee/types it installs versions as follows, in package.json:

    "@crawlee/types": "^3.8.1",
    "crawlee": "^3.8.1",
    "playwright": "^1.42.1",
    "playwright-extra": "^4.3.6",
    "puppeteer-extra-plugin-stealth": "^2.11.2",

Of note:

tsc -v                                                                                       
Version 5.0.4

I have looked for explicit dependency requirements for typescript/other libraries that Crawlee needs to be within but didn't see anything on https://crawlee.dev/docs/upgrading/upgrading-to-v3 or https://crawlee.dev/api/core/changelog though #2279 suggested you need 5.3 in a comment (where is that documented?)

Code sample

No response

Package version

crawlee 3.8.1, others listed above

Node.js version

v21.4.0

Operating system

macos

Apify platform

• Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

No response

[jawspeak]

UPDATE:

I think this is an error in my environment. Hope this can help someone else.

I think I globally installed typescript as well as locally in my project. Thus there was a version conflict as tsc was running the older global version. See https://www.typescriptlang.org/download to install locally in your project (not globally, or upgrade it globally)

which tsc shows it to be /opt/homebrew/bin/tsc, which is:

ls -al /opt/homebrew/bin/tsc
[.....]  /opt/homebrew/bin/tsc -> ../lib/node_modules/typescript/bin/tsc

If I run instead with npx tsc I don't have any of these errors.

tl;dr npx tsc instead of tsc. Or upgrade typescript globally.

The docs could still benefit from clarity of the required min version of TS.