Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WARNING: [Javinizer] Unable reach Javlibrary, enter websession/cookies to use the scraper #169

Open
jvlflame opened this issue Dec 25, 2020 · 15 comments · Fixed by #170
Open
Assignees

Comments

@jvlflame
Copy link
Collaborator

jvlflame commented Dec 25, 2020

------ Update July 6, 2021 ------

JAVLibrary has updated their cloudflare cookies that are required for site access. An update is incoming to resolve this. Resolved in release 2.5.1.

You can also point your javlibrary.baseurl setting to http://www.n53i.com ONLY if cloudflare blocking isn't enabled on that site. Otherwise you are required to use the site cookies.
image

------------------------------------

JAVLibrary has re-enabled their cloudflare DDOS protection on the site, meaning that using the scraper requires some additional steps.

I will also need to update the Get-Cfsession function to correctly create the required cookies to scrape the base javlibrary.com site. Fixed with 2.2.4

Current workaround:

---------- OPTION ONE ----------
Navigate to https://javlibrary.com and view the site cookies after completing the captcha.
Copy the content of __cfduid, cf_clearance, and your browser's user agent (as a single line) during sort when prompted.

image
image

---------- OPTION TWO ----------

Update setting javlibrary.baseurl to use b49t.com as it is a direct JAVLibrary mirror.

# Original setting
"javlibrary.baseurl": "http://www.javlibrary.com"
# New setting
"javlibrary.baseurl": "http://www.b49t.com"
@jvlflame jvlflame pinned this issue Dec 25, 2020
@jvlflame jvlflame self-assigned this Dec 25, 2020
@jvlflame jvlflame linked a pull request Dec 26, 2020 that will close this issue
@jvlflame jvlflame reopened this Dec 26, 2020
@cavalia88
Copy link

I have tried changing the javlibrary.baseurl setting to http://n53i.com, still getting the same error message.

Can you provide more detailed instructions on copying the cookie contents and user agent details? It's not very clear from your instructions

@jvlflame
Copy link
Collaborator Author

jvlflame commented Jul 8, 2021

I have tried changing the javlibrary.baseurl setting to http://n53i.com, still getting the same error message.

Can you provide more detailed instructions on copying the cookie contents and user agent details? It's not very clear from your instructions

@cavalia88 Can you try setting the javlibrary.baseurl to http://www.n53i.com instead? Not sure if that will change anything but I'm not having any issues from that mirror right now.

Basically you want to access javlibrary.com (or whatever your javlibrary.baseurl is set as) via your browser and complete the cloudflare captcha when prompted. After doing so, check the site's browser cookies and copy each of the corresponding cookie content over to Javinizer when it prompts you.

image

Though I did just try it right now and javlibrary isn't giving me the proper cookies in the browser. It's possible that Cloudflare is now properly detecting legitimate (browser) vs bot (javinizer) traffic and isn't providing the cookies in the browser...

EDIT: If the proper cookies aren't showing up, try deleting the cf_clearance cookie from your browser and then refreshing. Rinse and repeat until all the proper cookies show up:

  • cf_chl_2
  • cf_chl_prog
  • cf_clearance

@cavalia88
Copy link

Thank you for the clarification. I realized what the initial problem was on my side. I updated the Javinizer version to the latest version, but was amending the settings file of the earlier version. Changing the javlibrary.baseurl setting to http://n53i.com works now for me

When Javinizer updates to a new version, the old settings are not automatically migrated? We need to manually bring the settings over? I'm using the CLI version via Powershell on Windows 10

@jvlflame
Copy link
Collaborator Author

jvlflame commented Jul 8, 2021

Glad it's working for you now!

If you use the native PowerShell method, Update-Module Javinizer it won't migrate settings.
If you use the custom update function Javinizer -UpdateModule then it will.

@damenootoko
Copy link

It seems like n53i.com now have cloudflare as well, and I've been trying to fill the cookies values from javlibrary while on it's baseurl + cookies value from n53i while using it's baseurl as well but both has been failing consistently with errors that shows i need to input captcha

@jvlflame
Copy link
Collaborator Author

@damenootoko I'm using the default HTTPS javlibrary site to get the cookies and it's still working for me. Make sure when you paste your browser useragent it's all on a single line. I noticed when copying from google it gets separated into 2 lines so you'll need to paste it somewhere first and copy it again if you're using that method.

image

@cavalia88
Copy link

Same here, n53i.com no longer works. Does Javinizer allow scraping from other websites? Or it currently only works with Javlibrary?

Since Javlibrary is causing so many issues, thought it may be better if i switch to another website.

@jvlflame
Copy link
Collaborator Author

jvlflame commented Jul 12, 2021

@cavalia88 Yes, you can disable the javlibrary scraper completely and set your metadata priorities to any scraper of your choice.

Open the settings via the commandline by using Javinizer -OpenSettings.

@jvlflame
Copy link
Collaborator Author

I found an issue where not using the -Path parameter when running Javinizer would fail to append to necessary cookies onto the scraper.

E.g. running Javinizer -Recurse would fail, but Javinizer -Path . -Recurse would work.
Going to push a new release with this fix which will hopefully resolve some people's issues.

@yumesakiri
Copy link

I ran into this issue because I was using https://www.b49t.com as a mirror (can't remember why I started using b49t). https://www.n53i.com wasn't working either, so I changed the base url back to https://www.javlibrary.com itself and it worked. I noticed I couldn't find any of the aforementioned cookies for any of the 3 sites on any of Chrome, Firefox, Safari, and I'm not sure why. It looked like I was failing n53i's cloudflare check when I looked. Anyway, hope this helps someone.

@natsocenis
Copy link

Looks like this one is happening again. Started a week ago.

Have tried the options but my cookies for the JAVLibrary site do not match what the tool needs, and the mirrors either ask for cookie https://www.n53i.com info or dont seem to work (https://www.b49t.com)

Has anyone got this to work?

@Flute7
Copy link

Flute7 commented Feb 24, 2022

Looks like this one is happening again. Started a week ago.

Have tried the options but my cookies for the JAVLibrary site do not match what the tool needs, and the mirrors either ask for cookie https://www.n53i.com info or dont seem to work (https://www.b49t.com)

Has anyone got this to work?

Load the site (www.o58c.com) in an incognito browser window (chrome) to generate the full list of cookies. Even after clearing cookies and site data in a normal tab, I couldn't get all the cookies until I did this.

Should say there are 9 cookies. You will need to select cookies under o58c.com and www.o58c.com directories as the required info is spread across both entries.

@chronotank79
Copy link

Having this issue again. I've tried all the different solutions in this thread but I cannot get any of the various mirrors suggested here to give me the cookies that Javinizer wants. Another thing that is strange is that if I run the CLI Javinizer multiple times, it will intermittently sometimes be able to scrape one or two movies (that it had given an error for previously.)

@chronotank79
Copy link

UPDATE: using a VPN while scraping seems to have fixed the issue. Turn off VPN, error again. Turn on VPN, works again.

I think if you are sorting a large number of files, cloudflare may eventually decide to require additional security from your IP address. What is strange about this is that Javlibrary works perfectly normally in my normal web browsers.

@S1lv3rL10n
Copy link

Cannot locate all cookie values. Have tried Firefox, Chrome and Edge. Are there other workarounds?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants