Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use oauth to save private likes or blogs marked dashboard-only #128

Open
nightpool opened this issue Dec 5, 2018 · 4 comments
Open

Use oauth to save private likes or blogs marked dashboard-only #128

nightpool opened this issue Dec 5, 2018 · 4 comments

Comments

@nightpool
Copy link

Many of my blogs, and many of those i follow, enabled what is known as the "dashboard-only" option, which disables the https://username.tumblr.com link and (more importantly, for this issue) disables unauthenticated API access. Some blogs (those marked sensitive under the previous regime) have had this state forced on them. Additionally, many users don't make their likes public, and would like to back them up anyway.

Being able to specify an access_token for requests would make it possible to back up these blogs. Having a tumblr app and oauth flow built into the script is not required, but may be a nice improvement, once the basics have been implemented.

@MrEldritch
Copy link

MrEldritch commented Dec 5, 2018

Agreed - this is a critical feature, especially as most of the blogs that would be in need of urgent archival are NSFW blogs. Right now TumblThree is the only archiver I know of that offers this, but TumblThree is primarily an image ripper and lacks the ability to download a complete blog in readable format. (It ought to be possible to automatically reconstruct a blog from what it spits out as metadata, but it'd be a pain in the ass.)

I know it must be possible somehow, because TumblThree does it. I'm actually trying to look over TumblThree's code to see how they're doing it and if that can somehow be integrated into tumblr-utils or if they're using completely different approaches, but I've never tried to write something to do web scraping before and it's all going over my head.

@cebtenzzre
Copy link
Collaborator

This comment sheds some light on how it is done in TumblThree.

@MrEldritch
Copy link

Likewise, this is the part of the code most relevant for how TumblThree can scrape hidden blogs; it shows how they use that hidden svc api.

@Hrxn
Copy link

Hrxn commented Dec 9, 2018

I should probably mention this here, could be of interest for @cebtenzzre and others:

https://github.com/mikf/gallery-dl

CLI, cross-platform, written in Python as well (3) and supports Tumblr, among many others.
The point is, it supports OAuth. Sans external dependencies (only requests for Python).
So for anyone well versed enough in Python and willing to give it shot, here are probably good starting points:
https://github.com/mikf/gallery-dl/blob/master/gallery_dl/oauth.py
https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/tumblr.py

I've used this (still am, actually) myself, and it worked for the kind of special blogs you mention here.

But please note, this was never about doing a full Tumblr backup unlike this project here.
That means that text posts will probably not work. By design. Definitely not out of the box, you have to change this config setting here: https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractortumblrposts

(Which, to be fair, you have to do anyway, when setting your personal tokens for OAuth in the config file)

I can only say that I've never done any text posts so far.

But if someone is willing I'm sure that the OAuth implementation can be adapted from there. Or, alternatively, send over a PR to enhance support for other post types.
Whatever you prefer..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants