Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION] Video Downloading Error 403 #35

Open
alexempie opened this issue Feb 20, 2023 · 19 comments
Open

[QUESTION] Video Downloading Error 403 #35

alexempie opened this issue Feb 20, 2023 · 19 comments
Labels
duplicate This issue or pull request already exists question Further information is requested

Comments

@alexempie
Copy link

Cannot download any video by down_addr from challenge collection. It returns 403 Forbidden error

for video in challenge.videos
requests.get(video.video.download_addr)

How to fix that?

@alexempie alexempie added the question Further information is requested label Feb 20, 2023
@Russell-Newton Russell-Newton changed the title [QUESTION] [DUPLICATE] Video Downloading Error 403 Feb 20, 2023
@Russell-Newton
Copy link
Owner

Russell-Newton commented Feb 20, 2023

I'm not experiencing this problem, so I don't have a good way to debug it. I need some more information:

  1. Is the code sample you provided exactly what you're using and experiencing issues with?
  2. What country/region are you in?
  3. Are you able to access TikTok and watch videos normally through a desktop browser?

@Russell-Newton
Copy link
Owner

Related to #24

@Russell-Newton Russell-Newton changed the title [DUPLICATE] Video Downloading Error 403 [QUESTION] Video Downloading Error 403 Feb 20, 2023
@Russell-Newton Russell-Newton added the duplicate This issue or pull request already exists label Feb 20, 2023
@alejandroarmas
Copy link

alejandroarmas commented Mar 8, 2023

I am also getting a 403 error.

My code:

 async with AsyncTikTokAPI(emulate_mobile=False, navigation_retries=3, navigation_timeout=60) as api:
       async for video in user.videos:
              dwn_link = await self.save_video(video)
    
...

async def save_video(self, video: Video):
        async with aiohttp.ClientSession() as session:
            async with session.get(video.video.download_addr) as resp:
                print(f'{resp=}')
URL = https://tiktok.com/@kyliejenner/video/7202869446044650795
resp=<ClientResponse(https://v16-webapp-prime.us.tiktok.com/video/tos/useast5/tos-useast5-pve-0068-tx/7cccbd62e1f9451b8b14f8b10b280a99/?a=1988&ch=0&cr=0&dr=0&lr=tiktok_m&cd=0%7C0%7C1%7C0&cv=1&br=1474&bt=737&cs=0&ds=3&ft=4KJMyMzm8Zmo0FVD1A4jVwoWdpWrKsdm&mime_type=video_mp4&qs=0&rc=NTg7Ozw8ZWY5NTY5NjY0Z0BpM2VybTc6Zjt1aTMzZzczNEA2YF5iYDFfXzExYDAyYi9jYSNzc2g0cjRva25gLS1kMS9zcw%3D%3D&expire=1678277529&l=20230308061154A6172C27ED6C374EF78B&policy=2&signature=8491010f44ef385623181ab2cc6d805e&tk=tt_chain_token) [403 Forbidden]>
<CIMultiDictProxy('Server': 'AkamaiGHost', 'Mime-Version': '1.0', 'Content-Length': '408', 'Expires': 'Wed, 08 Mar 2023 06:11:56 GMT', 'Date': 'Wed, 08 Mar 2023 06:11:56 GMT', 'X-Cache': 'TCP_DENIED from a23-67-78-238.deploy.akamaitechnologies.com (AkamaiGHost/11.0.0-46340752) (-)', 'Connection': 'keep-alive', 'x-response-cache': '', 'x-tt-trace-tag': 'id=16;cdn-cache=miss;type=static', 'Server-Timing': 'cdn-cache; desc=MISS, edge; dur=0, origin; dur=0', 'Content-Type': 'video/mp4', 'Access-Control-Expose-Headers': 'Content-Length,Content-Range,content-type,expires,last-modified,via,X-Cache,x-response-cache,x-response-sinfo,x-response-cinfo', 'Accept-Ranges': 'bytes', 'Access-Control-Allow-Headers': 'range', 'Access-Control-Allow-Credentials': 'true', 'X-Akamai-Request-ID': '42391e2d')>

@alejandroarmas
Copy link

Is there any resolution? Thanks!

@Russell-Newton
Copy link
Owner

Russell-Newton commented Mar 29, 2023

I haven't had a whole lot of time to try to debug this, but I can share my initial thoughts:

It's likely you may need to pass the cookies TikTokPy stores to aiohttp or requests.

@alejandroarmas can you try modifying your code to have this:

async with AsyncTikTokAPI(emulate_mobile=False, navigation_retries=3, navigation_timeout=60) as api:
       async for video in user.videos:
              dwn_link = await self.save_video(video, {cookie['name']: cookie['value'] for cookie in api.context.cookies()})
    
...

async def save_video(self, video: Video, cookies={}):
        async with aiohttp.ClientSession(cookies=cookies) as session:
            async with session.get(video.video.download_addr) as resp:
                print(f'{resp=}')

Note the changes here pass the api's cookies to the client session.

@alexempie can you try changing your code to use

response = requests.get(video.video.download_addr, cookies={cookie['name']: cookie['value'] for cookie in api.context.cookies()})

Note the change here passes the api's cookies to the requests.get call.


If these changes work, I can update the documentation to include the fix.

@papayyg
Copy link

papayyg commented Apr 1, 2023

@Russell-Newton
I tried the code you sent. It gave the error

'method' object is not iterable

I modified it a little bit:
cookies = {cookie['name']: cookie['value'] for cookie in await api.context.cookies()}

And then it worked. That is, the cookies that were used in the API I passed to the session. And the cookies themselves look like this:
{'ttwid': '1%7CAVI7ulvcK00n-W6OElzY2emHneSrLvx4Slnjrnv_WJ0%7C1680351019%7Cb0453342efb89d920194f0eb407c408be94cecf3590b3161b44690ecdc527462', 'tt_csrf_token': 'CoTiG9bD-iWUO27U5OCm8oHNUzApu7tztK28', 'tt_chain_token': 'iM+Thjhn20W44pqVH04FLQ==', '__tea_cache_tokens_1988': '{%22_type_%22:%22default%22%2C%22user_unique_id%22:%227217052627791332865%22%2C%22timestamp%22:1680351030840}', 'tiktok_webapp_theme': 'light', 'csrf_session_id': 'cb3a4f15cb7bc696c753918d02bca160', 'msToken': 'ks6BGCynadZVeb0ztd_itBwQ2wOPJnCQp2-ldD539h1ddUxwhXpDkvIqWiqx2tbU0PQcWvyVdCQIue9pPp8FqE-FmN3dwspQ9tBG4-oDam7QFZ0jcTxOip8d-g=='}

But the answer was still: Access Denied. Then I checked how it worked on the TT site itself. And there I realized that the direct link to the video only opens if it is opened by the same cookies as the normal video link. And there I saw one cookie that does not appear in api.context.cookies(), namely s_v_web_id. And if you look at the cookies that are used when viewing the video via a direct link, you can only see those cookies:
image

And if we look at the cookies I wrote, we see that everything is there except s_v_web_id.

I don't know why it's not issued when using api.context.cookies(). But I think sending this cookie in a session would solve the problem, or send the same cookie in both api and session initially.

@Russell-Newton
Copy link
Owner

Every time a video or user page is loaded, TikTokPy clears its cookies. It's possible that s_v_web_id doesn't consistently get populated every visit. I probably need to adjust the cookie management. I don't have a whole lot of time in the next couple of days, given I'm a full time masters student, but I'll try to address this soon.

@Russell-Newton
Copy link
Owner

After looking into it a bit, s_v_web_id seems to be a cookie that TikTok isn't really using anymore. It doesn't seem necessary for me when viewing or downloading videos on my desktop.

For whatever reason, I seem to be completely unable to recreate this issue consistently. On my end, looking at the cookies in my browser (I used a VPN connecting to an IP in Europe, Asia, and a different location in the Americas), and it looks like the only cookies that are required are msToken, ttwid, tt_chain_token and tt_csrf_token. I used incognito mode to get the effect of having no preloaded cookies, so only the necessary ones would be included.

I suspect these tokens are related to the download link, and any change in either makes the video not downloadable.

Could you try copying over only those 4 cookies and seeing if that makes any difference? Something like:

await self.save_video(video, {cookie["name"]: cookie["value"] for cookie in await api.context.cookies() if cookie["name"] in ("msToken", "ttwid", "tt_chain_token", "tt_csrf_token")})

If that doesn't work, you may need to use a VPN.

@papayyg
Copy link

papayyg commented Apr 11, 2023

It turned out to be much simpler and more obvious.

You follow the link to the video and do the following

  1. Take away the "src" attribute from the
  2. Taking the "tt_chain_token" value from the cookie
  3. Create header
'referer': 'https://www.tiktok.com/'

Send a GET request to the link from src with the cookie and header

It's roughly like this:

import requests

cookies = {
    'tt_chain_token': '8J8/JTDl922BTqv9SaYTpw==', #Get it out of cookies when you open the regular video page
}

headers = {
    'referer': 'https://www.tiktok.com/', #I don't know why, but it doesn't work without it.
}

response = requests.get(
    'https://v16-webapp-prime.tiktok.com/video/tos/useast2a/tos-useast2a-pve-0068/oUIa0yuCmAiBcoIhPzEAwfhgqVzDiGQcWNFvWk/?a=1988&ch=0&cr=0&dr=0&lr=tiktok_m&cd=0%7C0%7C1%7C0&cv=1&br=2626&bt=1313&cs=0&ds=3&ft=_RwJrB0rq8ZmokDNTc_vjdsT_AhLrus&mime_type=video_mp4&qs=0&rc=OGc3aGRoOTk4ZmVkPDxmM0BpajVoO2k6ZnE4ajMzNzczM0AxL2FeMjIvNS0xYDMuMF4uYSMvcWJfcjRnXm5gLS1kMTZzcw%3D%3D&btag=80000&expire=1681225744&l=20230411090855B646C0771A5183040DD5&ply_type=2&policy=2&signature=34ae9a57c2f8c8b8ca7ca0fff86e8fab&tk=tt_chain_token', #src link
    cookies=cookies,
    headers=headers,
)

with open('test.mp4', "wb") as f:
    f.write(response.content)

@Russell-Newton
Copy link
Owner

Russell-Newton commented May 4, 2023

I'll add this to the documentation ASAP! Thank you for the help @papayyg. I'll reference this thread and your account when I add it.

Russell-Newton added a commit that referenced this issue May 5, 2023
Include extra note that can help resolve 403 errors when downloading videos
@Russell-Newton
Copy link
Owner

This has been added to the documentation: https://tiktokpy.readthedocs.io/en/stable/users/usage.html#download-videos-and-slideshows

I'm closing the issue because it's been finished.

@jinfu-leng
Copy link

Hi, @Russell-Newton and @papayyg, I followed your example, but I still got the access denied error. Could you please help?

Here is the code:

async def fetch_video_bytes(video: Video, api: AsyncTikTokAPI):
    async with aiohttp.ClientSession(cookies={cookie["name"]: cookie["value"] for cookie in await api.context.cookies() if cookie["name"] == "tt_chain_token"}) as session:
        print("video.video.download_addr: " + video.video.download_addr)
        async with session.get(video.video.download_addr, headers={"referer": "https://www.tiktok.com/"}) as resp:
            return await resp.read()


async def download_tiktok_videos(user_handler, download_directory):
    video_download_directory = os.path.join(download_directory, user_handler, "video")
    os.makedirs(video_download_directory, exist_ok=True)

    async with AsyncTikTokAPI(navigation_timeout=60*1000) as api:
        user = await api.user(user_handler, video_limit=2000)

        async for video in user.videos:
            print(f"{video.id}, {video.desc}")
            if not video.image_post:
                video_bytes = await fetch_video_bytes(video, api)
                print("video_bytes:" + video_bytes.decode())
                async with aiofiles.open("output.mp4", 'wb') as file:
                    await file.write(video_bytes)
            return

Here is the error message:

video_bytes:<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
 
You don't have permission to access "http&#58;&#47;&#47;v16&#45;webapp&#45;prime&#46;us&#46;tiktok&#46;com&#47;video&#47;tos&#47;useast5&#47;tos&#45;useast5&#45;ve&#45;0068c003&#45;tx&#47;oMCEchwzgYMBpmAft2kRDQAGIKBzngUYQfUajS&#47;&#63;" on this server.<P>
Reference&#32;&#35;18&#46;1e32c517&#46;1686182375&#46;8ffae9f
</BODY>
</HTML>

@Russell-Newton
Copy link
Owner

@jinfu-leng Have you tried using a proxy and/or VPN?

@papayyg
Copy link

papayyg commented Jun 9, 2023

They seem to have changed something. Now I can’t access the direct link in any way, it doesn’t work with vpn either

@Russell-Newton Russell-Newton reopened this Jun 9, 2023
@Russell-Newton
Copy link
Owner

I'll see what I can find out. Once again, it's working fine on my end in the US, but I may be able to figure something out.

@Russell-Newton
Copy link
Owner

It looks like TikTok is now validating the X-Bogus parameter now, and from what I can tell this is generated right before fetching based on the query parameters. I'll keep looking into this.

Russell-Newton added a commit that referenced this issue Jun 15, 2023
* Create functions for executing API calls of 4 kinds:
    * comment/list/ - video comments
    * post/item_list/ - user posts
    * challenge/item_list/ - popular videos tagged with a challenge
    * related/item_list/ - videos related to this one

* Opens up potential future resolutions for #35, #38, #40, #43, and #44
@ExpressGit
Copy link

I have same problem

<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
 
You don't have permission to access "http&#58;&#47;&#47;v16&#45;webapp&#45;prime&#46;us&#46;tiktok&#46;com&#47;video&#47;tos&#47;useast5&#47;tos&#45;useast5&#45;ve&#45;0068c003&#45;tx&#47;oMCEchwzgYMBpmAft2kRDQAGIKBzngUYQfUajS&#47;&#63;" on this server.<P>
Reference&#32;&#35;18&#46;1e32c517&#46;1686182375&#46;8ffae9f
</BODY>
</HTML>

code:

async def save_video(video: Video,api: AsyncTikTokAPI,cookies):
     async with aiohttp.ClientSession(cookies=cookies) as session:
        # Creating this header tricks TikTok into thinking it made the request itself
        async with session.get(video.video.download_addr, headers={"referer": "https://www.tiktok.com/"}) as resp:
            return await resp.read()


async def get_tiktok_trend_video():
    async with AsyncTikTokAPI() as api:
        user = await api.user("odapolf", video_limit=10)
        async for video in user.videos:
            print(video)
            video_bytes  = await save_video(video,api,{cookie['name']: cookie['value'] for cookie in api.context.cookies()})
            print("video_bytes:" + video_bytes.decode())
            file_path = os.path.join(directory,'{}.mp4'.format(video.desc))
            print(file_path)
            async with aiofiles.open(file_path, 'wb') as file:
                    await file.write(video_bytes)
        return "OK"

@Russell-Newton
Copy link
Owner

This may be fixed in version 0.2.4 with the video download function. It doesn't work on slideshows and requires the optional yt-dlp dependency. This can be installed with pip install yt-dlp or pip install tiktokapipy[download]. It may not work 100% depending on where you are, but it works well for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists question Further information is requested
Projects
None yet
Development

No branches or pull requests

6 participants