How to work better when the target has an anti-crawler mechanism(-headless)？

[hktalent]

hi，How to work better when the target has an anti-crawler mechanism(-headless)？
I encountered some targets in actual work, such as:
https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950
They seem to have some anti-crawler extremes, which will cause the page to load very slowly, eventually causing the page to load. Failed, unable to obtain required information

katana -hl -u 'https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950' -sc -debug

Of course, this address can only be accessed within China.
out

[INF] Current katana version v1.0.4 (latest)
[INF] Started headless crawling for => https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950
[WRN] "https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950" on wait load: context deadline exceeded
[WRN] "https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950" on wait idle: context deadline exceeded
https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950
[WRN] Could not request seed URL https://www.aliyundrive.com/s/y3j5ej7diA8/folder/650e59b7845336306e1d4a90870b672e5751c950: [hybrid:RUNTIME] context deadline exceeded <- could not get dom

At the same time, I found that chrome did not close properly after the program ended

[hktalent]

I noticed a strange phenomenon, normally I get

However, if I step through the code, the page loads fine
However, if I debug the code step by step, the page loads normally without this error: "你打开的链接有误，请重试"

[MetzinAround]

First, just a reminder that all our tools are only to be used on targets you have approval for. If this site doesn't want you crawling, it might mean it's out of scope for a bounty program or pentest.

Second, have you tried it with a longer time for the -timeout flag yet to see if you can wait out the anti crawling measure?

Some rate limit options might work, too.

[hktalent]

thanks