[YouTube] New Channel renderer fails extracting various metadata like channel_follower_count

[bbilly1]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

• I'm reporting that yt-dlp is broken on a supported site
• I've verified that I have updated yt-dlp to nightly or master (update instructions)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

Europe

Provide a description that is worded well enough to be understood

Since a few days, I'm not able to reliably extract channel_follower_count when extracting info from the main channel page.

I've observed, that it fails around 50% of the times, but it's not consistent, making me assume there is some A/B testing going on.

Bellow verbose output doesn't show anything, but it is produced like this from latest commit on master:

import yt_dlp

url = "https://www.youtube.com/@Computerphile"

yt_obs = {
    "skip_download": True,
    "playlist_items": "0,0",
    "verbose": True,
}

response = yt_dlp.YoutubeDL(yt_obs).extract_info(url)
response.get("channel_follower_count")

Where channel_follower_count in response is None for me around 50% of times.

I can only reproduce this on channel pages, not on video pages.

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• If using API, add 'verbose': True to YoutubeDL params instead
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] API
[debug] params: {'skip_download': True, 'playlist_items': '0,0', 'verbose': True, 'compat_opts': set(), 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.18 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-us,en;q=0.5', 'Sec-Fetch-Mode': 'navigate'}}
[debug] Lazy loading extractors is disabled
[debug] Python 3.12.3 (CPython x86_64 64bit) - Linux-6.8.9-arch1-2-x86_64-with-glibc2.39 (OpenSSL 3.3.0 9 Apr 2024, glibc 2.39)
[debug] exe versions: ffmpeg 6.1.1 (setts), ffprobe 6.1.1
[debug] Optional libraries: certifi-2024.02.02, requests-2.31.0, sqlite3-3.45.3, urllib3-1.26.18
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests
[debug] Loaded 1810 extractors
[youtube:tab] Extracting URL: https://www.youtube.com/@Computerphile
[youtube:tab] @Computerphile: Downloading webpage
[debug] [youtube:tab] Selected tab: 'videos' (videos), Requested tab: ''
[youtube:tab] Downloading all uploads of the channel. To download only the videos in a specific tab, pass the tab's URL
[download] Downloading playlist: Computerphile - Videos
[youtube:tab] Playlist Computerphile - Videos: Downloading 0 items
[debug] The information of all playlist entries will be held in memory
[download] Finished downloading playlist: Computerphile - Videos

[bbilly1]

Some further investigating shows, that I'm getting wildly different input between tries in _extract_metadata_from_tabs. Inspecting the content of the data argument going into that function there.

On successful requests I'm seeing all relevant content in c4TabbedHeaderRenderer in header. On failing requests, that is not present and I'm seeing a pageHeaderRenderer instead. That looks quite different, but the subscriber count is nested in contentMetadataViewModel.

[bashonly]

Seems like A/B testing similar to what YT is currently doing with comments.

[oifj34f34f]

Might be related to iv-org/invidious#4681

[bbilly1]

Looking further into this, it's not just the channel_follower_count extraction that fails, channel thumbnails like banner fails too, plus probably a few other things, supporting the thinking of @oifj34f34f with the linked issue.