Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] TikTok seems to have changed things #82

Open
bobcogneato opened this issue Dec 10, 2023 · 2 comments
Open

[BUG] TikTok seems to have changed things #82

bobcogneato opened this issue Dec 10, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@bobcogneato
Copy link

Describe the bug
TikTok got rid of #SIGI_STATE and replaced it with #__UNIVERSAL_DATA_FOR_REHYDRATION__, now the data exists in ex /__DEFAULT_SCOPE__/webapp.video-detail/itemInfo and has different keys so the mapper doesn't work anymore.

To Reproduce
Steps to reproduce the behavior:

  1. Scrape any page. Timeout occurs because SIGI_STATE isn't attached.

Expected behavior
Well, you know...

Version Information
Please include what versions of pydantic, playwright, and tiktokapipy you have installed (can be found with pip freeze).

playwright==1.30.0
playwright-stealth==1.0.5
pydantic==2.5.2
pydantic_core==2.14.5
tiktokapipy==0.2.4.post2

System Information
Please provide information about the system you're running the library on. What's the OS? Is it a desktop, server, VM, or container?
Various linux (ubuntu/debian) machines
Region Information
Where in the world are you located? If you're not in the US, it's likely that TikTok treats your region differently, which may present some challenges.
US
Additional context
Add any other context about the problem here.

@bobcogneato bobcogneato added the bug Something isn't working label Dec 10, 2023
@RobertModus
Copy link

As of today, i only see static and user data in the new state object, eg __$UNIVERSAL_DATA$__.__DEFAULT_SCOPE__["webapp.user-detail"] but no webapp.video-detail. Video data should be gathered through the DeferredItemListIterator(self._api, "post", self.sec_uid) magic anyways, so shmaybe not a big change needed?

@Russell-Newton
Copy link
Owner

I'll need some time to address this. At the moment, I'm pretty busy with research and various applications, but I'll try to get to this soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants