Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Instagram threads.net site request #7523

Open
9 of 11 tasks
ehyoitsdavid opened this issue Jul 6, 2023 · 8 comments · May be fixed by #9852
Open
9 of 11 tasks

Instagram threads.net site request #7523

ehyoitsdavid opened this issue Jul 6, 2023 · 8 comments · May be fixed by #9852
Labels
site-request Request to support a new website

Comments

@ehyoitsdavid
Copy link

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Canada

Example URLs

Provide a description that is worded well enough to be understood

Note that Instagram "threads.net" uses a CDN to store their videos and downloading videos from the CDN is not easy, just like Instagram reels. One needs to fake user agent and most importantly cookie support to prove that it is a real browser and not a script to get videos.

First I suggest implementing support for single videos first, then for whole accounts (which is probably risky because of rate limits and possibly bans)

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://www.threads.net/t/CuXnwmrMIZL', '-F']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481] (pip)
[debug] Python 3.11.4 (CPython arm64 64bit) - macOS-13.4.1-arm64-arm-64bit (OpenSSL 3.1.1 30 May 2023)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
yt-dlp is up to date (stable@2023.07.06)
[generic] Extracting URL: https://www.threads.net/t/CuXnwmrMIZL
[generic] CuXnwmrMIZL: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] CuXnwmrMIZL: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://www.threads.net/t/CuXnwmrMIZL
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1560, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1688, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 710, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/yt_dlp/extractor/generic.py", line 2568, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.threads.net/t/CuXnwmrMIZL
@ehyoitsdavid ehyoitsdavid added site-request Request to support a new website triage Untriaged issue labels Jul 6, 2023
@ehyoitsdavid ehyoitsdavid changed the title Instagram threads.net Instagram threads.net site request Jul 6, 2023
@bashonly bashonly added the account-needed Account details are needed to test/fix this label Jul 6, 2023
@pukkandan pukkandan removed the triage Untriaged issue label Jul 6, 2023
@ehyoitsdavid

This comment was marked as resolved.

Sign in to view
@bashonly bashonly removed the account-needed Account details are needed to test/fix this label Jul 6, 2023
@october262
Copy link

link to video 1 - https://scontent.cdninstagram.com/v/t50.2886-16/354809962_1213825392613482_2368464029860537544_n.mp4?_nc_ht=scontent.cdninstagram.com&_nc_cat=106&_nc_ohc=omg_ffWRix0AX9EYSXr&edm=APs17CUBAAAA&ccb=7-5&oh=00_AfDBz9VgLy_gbWpgoSL7IjvObkC4IHOJLprH6MFKoJM3Hw&oe=64A95D24&_nc_sid=10d13b

@jinthoa
Copy link

jinthoa commented Sep 22, 2023

Here's another example:

https://www.threads.net/@pubity/post/Cxd59tZLMrd

URL to video: https://scontent.cdninstagram.com/v/t50.2886-16/380787197_335451142264743_8090553461744062787_n.mp4?_nc_ht=scontent.cdninstagram.com&_nc_cat=103&_nc_ohc=V_mAxTiQzyoAX81uwqC&edm=APs17CUBAAAA&ccb=7-5&oh=00_AfDE--bQHO8yF-ZnCdh3_EuKl1MW6uKXbsyz_SnlfDeGTw&oe=650F64A2&_nc_sid=10d13b

Would be nice to see this supported.

@dirkf dirkf mentioned this issue Dec 15, 2023
Open
5 tasks
@SirElderling
Copy link
Contributor

One thing I noticed (at least from my end) is that downloading a threads video through yt-dlp is way slower than if we fetch it via curl.

time yt-dlp "https://scontent.cdninstagram.com/v/t50.2886-16/411700231_1013302036637199_8020624842325890217_n.mp4?_nc_ht=scontent.cdninstagram.com&_nc_cat=103&_nc_ohc=6pTQ9_0AGqUAX-NvjDf&edm=APs17CUBAAAA&ccb=7-5&oh=00_AfBGkJziC0PsIcyv8hFAXVWMInc2zXKahH24OAVMmpbUnA&oe=658077B4&_nc_sid=10d13b"
[generic] Extracting URL: https://scontent.cdninstagram.com/v/t50.2886-16/411700231_1013302036637199_8020624842325890217_n....077B4&_nc_sid=10d13b
[generic] 411700231_1013302036637199_8020624842325890217_n.mp4?_nc_ht=scontent.cdninstagram: Downloading webpage
[info] 411700231_1013302036637199_8020624842325890217_n.mp4?_nc_ht=scontent.cdninstagram: Downloading 1 format(s): mp4
[download] Resuming download at byte 130048
[download] Destination: 411700231_1013302036637199_8020624842325890217_n [411700231_1013302036637199_8020624842325890217_n.mp4？_nc_ht=scontent.cdninstagram].mp4
[download] 100% of    2.78MiB in 00:01:12 at 39.08KiB/s

real	1m14.511s
user	0m1.707s
sys	0m0.215s

time curl 'https://scontent.cdninstagram.com/v/t50.2886-16/411700231_1013302036637199_8020624842325890217_n.mp4?_nc_ht=scontent.cdninstagram.com&_nc_cat=103&_nc_ohc=6pTQ9_0AGqUAX-NvjDf&edm=APs17CUBAAAA&ccb=7-5&oh=00_AfBGkJziC0PsIcyv8hFAXVWMInc2zXKahH24OAVMmpbUnA&oe=658077B4&_nc_sid=10d13b' \
  -H 'authority: scontent.cdninstagram.com' \
  -H 'accept: */*' \
  -H 'accept-language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'dnt: 1' \
  -H 'range: bytes=0-' \
  -H 'referer: https://www.threads.net/' \
  -H 'sec-ch-ua: "Not_A Brand";v="8", "Chromium";v="120"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Linux"' \
  -H 'sec-fetch-dest: video' \
  -H 'sec-fetch-mode: no-cors' \
  -H 'sec-fetch-site: cross-site' \
  -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36' \
  --compressed --output ~/Temp/aaa.mp4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2841k  100 2841k    0     0  3316k      0 --:--:-- --:--:-- --:--:-- 3319k

real	0m0.881s
user	0m0.114s
sys	0m0.053s

@kartagis
Copy link

Hi, I came across this and is there a time frame threads.net will go into yt-dlp? I just updated to 2024.02.21.232721 nightly and still no trace of this :(

@seproDev
Copy link
Collaborator

@kartagis Are you offering to work on this? If you need any help, we can try to assist you.

@kartagis
Copy link

@seproDev I'm not actually because I don't know python but I'd be willing to help in any way that I can.

@renandecarlo
Copy link

Any news?

@renandecarlo renandecarlo linked a pull request May 3, 2024 that will close this issue
Open
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
site-request Request to support a new website
Projects
Major sites
Status: instagram
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[threads] Add extractor (Threads.net) renandecarlo/yt-dlp
9 participants
@kartagis @seproDev @pukkandan @renandecarlo @jinthoa @october262 @bashonly @ehyoitsdavid @SirElderling