Downloading X number of videos based on a hashtag/trend

[savvas17]

I have seen good work done on downloading user videos. What I am struggling to figure out is how to scrape videos based on a certain hashtag or trend. I am struggling to find a way to do this because the video links don't appear in the source of the page. These pages only respond to GET requests and not POST.

• These type of pages are found on the discover page of TikTok.

• These pages are in two forms: tiktok.com/music/... or tiktok.com/tag/...

For example, is there a way to download X number of videos from a page like this: https://www.tiktok.com/music/No-Idea-6764115859423152901

Thank you :)

[socialmediaguy]

@savvas17 Did you manage to figure this out?

[savvas17]

@socialmediaguy I didn't manage to do it in python. But what I did manage to do is write some javascript which you inject into the browser and pulls video URLs from the page and scrolls down etc. (it is not that complicated). Those URLs then can be downloaded in a batch. Unfortunately it is a manual process for now as you have to physically navigate to the trend page in a browser. But it's something I guess.

[socialmediaguy]

@savvas17 Thanks for your reply!
Does that pull them out based on most popular or certain number of views or likes? I.e the most likes videos or viewed videos...?

[savvas17]

@socialmediaguy hmm as far as I can see TikTok seems to show you the trending ones first based on your location. Here is an example page https://www.tiktok.com/tag/catchthesplash
The videos you see here are likely different to the ones I see which are local to my country. All the code does is go through these videos in a page sequentially and get the links for them (it just keeps scrolling down). Then those links can be downloaded. I'm sure there would be a nicer way to automate this. In fact, im sure that an extension can be written where you just click it on a page and it produces the video links.

[socialmediaguy]

@savvas17 I noticed that too, but it doesn't show them in order or likes or views. Maybe based on trending in x time. I want something that gives me even just links of videos in order of most viewed to least in a hashtag.

[socialmediaguy]

I can't seem to edit my last comment but @savvas17 I guess you are halfway there. Now if you can read the likes/views per video then just order the links in ascending order

[savvas17]

@socialmediaguy It is definitely possible to order them. The issue of my implementation is that it requires a bit of manual work by visiting the page in the browser. But thereafter it should be simple. Depending on your use case (if I may ask?) this might not be appropriate to manually visit links like this. But it can work for some situations.

[socialmediaguy]

@savvas17 I'm curious to see what kind of videos in a particular hashtag get the most likes or views so for my use case i won't need to 100s, just the top ones and checking on adhoc basis so I don't mind if it is manual (of course automated is better) Speaking to a local friend who is pretty good with javascript to see if it can quickly knock something together, otherwise I will spend some time on it over the weekend.

[savvas17]

@socialmediaguy Yeah your javascript friend will accomplish the task easily. My only concern is that we don't really know what order tiktok displays those videos... but maybe grabbing the first 100 or 1000 is a safe bet to find the highly liked ones