Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tumblr] metadata files saving with incorrect (different posts) filename #5557

Closed
Sey-81 opened this issue May 5, 2024 · 4 comments
Closed

Comments

@Sey-81
Copy link

Sey-81 commented May 5, 2024

ran gallery-dl to download a bunch of tumblr posts together with the text and tags, I end up with metadata files that don´t match the filenames correctly to which posts data is actually written in them.
specifically, the first saved posts metadata is written into "metadata.txt", then any following posts have the metadata written into a file that has the name of the previous saved post, causing all the filenames to be offset by a post.

so metadata.txt would contain img1.png´s data, then img1.txt contains img2.png´s data, and so on. this happens when saving a whole gallery.
ideally I´d like each posts txt file to just match its post name.

not the main issue at hand, but another oddity that I´ve noticed with metadata.txt is that if I do something like
gallery-dl (URL of single post) (URL of another single post)

both posts images are saved, but the metadata of only the second one.
doing it separately

-gallery-dl (URL of single post)
-gallery-dl (URL of another single post)

results the same so it seems the second one in this case overwrites the first posts metadata, when ideally I´d expect them to have separate .txt files with the filenames of their respective saved posts.

this is my config file

{
    "extractor": {
        "tumblr": {
            "api-key": "-",
            "api-secret": "-",
            "fallback-retries": 9,
            "metadata": true,
            "ratelimit": "wait",
            "postprocessors":[{
                "directory": "posts",
                "name": "metadata",
                "event": "post",
                "mode": "custom",
                "format": "content: \n{text} \n{caption} \n{summary} \n{tags} \n \nURL: https://tumblr.com/{blog_name}/{id}"
            }],
            "filename": "{blog_name}-{date:%Y%m%d}-{id}_{num:>02}-{summary}.{extension}",
            "directory": { 
                "": ["{category}","{blog_name}"]
            }
        }
    }
}
@Hrxn
Copy link
Contributor

Hrxn commented May 5, 2024

so metadata.txt would contain img1.png´s data, then img1.txt contains img2.png´s data, and so on. this happens when saving a whole gallery. ideally I´d like each posts txt file to just match its post name.

not the main issue at hand, but another oddity that I´ve noticed with metadata.txt is that if I do something like gallery-dl (URL of single post) (URL of another single post)

both posts images are saved, but the metadata of only the second one. doing it separately

-gallery-dl (URL of single post) -gallery-dl (URL of another single post)

results the same so it seems the second one in this case overwrites the first posts metadata, when ideally I´d expect them to have separate .txt files with the filenames of their respective saved posts.

You also need to set metadata.filename for this. With a filename string including their post ID.
You're aiming for 1 post (on tumblr's side), 1 one metadata file, right?

@Sey-81
Copy link
Author

Sey-81 commented May 6, 2024

You also need to set metadata.filename for this. With a filename string including their post ID. You're aiming for 1 post (on tumblr's side), 1 one metadata file, right?

thank you, this helps a lot. and yes Im trying to match the text files filenames with the images filenames (besides the .txt extension of course). so that I´d have image1.png and image1.png.txt, image2.png and image2.text, and so on

{
    "extractor": {
        "tumblr": {
            "api-key": "-",
            "api-secret": "-",
            "fallback-retries": 9,
            "metadata": true,
            "ratelimit": "wait",
            "postprocessors":[{
                "directory": "posts",
                "name": "metadata",
                "filename": "{blog_name}-{date:%Y%m%d}-{id}_{num:>02}-{summary}.{extension}.txt",
                "event": "post",
                "mode": "custom",
                "format": "content: \n{text} \n{caption} \n{summary} \n{tags} \n \nURL: https://tumblr.com/{blog_name}/{id}"
            }],
            "filename": "{blog_name}-{date:%Y%m%d}-{id}_{num:>02}-{summary}.{extension}",
            "directory": { 
                "": ["{category}","{blog_name}"]
            }
        }
    }
}

this gives me almost the desired result, but with this the metadata files names always seem to read the extension of the images as "none"
so instead of img1.png being saved with img1.png.txt, I have img1.png and img1.none.txt
previously the metadata files were saved without having that "none" part issue. I would ideally like to include the images extension there instead instead of just cutting it out
from how I´m thinking a solution could be to somehow set it to just copy the images filenames for the metadata text files filenames, rather than this, if there´s any way of doing that.
(apologies for accidentally clicking close issue earlier- misclick)

@Sey-81 Sey-81 closed this as completed May 6, 2024
@Sey-81 Sey-81 reopened this May 6, 2024
@Hrxn
Copy link
Contributor

Hrxn commented May 6, 2024

Edit

Spoke to soon, you were already doing what I suggested first.

Yeah, the metadata.filename, i.e. and the "regular" filename output option
must match if you are going for something like "image1.png and image1.png.txt, image2.png and image2.png.txt, and so on"

Although I'm not sure if this makes sense at all - the way Tumblr used to work, with one Post containing multiple images (like a small gallery), but the Post is still the same, and therefore the metadata is also always the same (unless Tumblr has changed something in the meantime, I can't tell).

You're already using "event": "post" in the post-processor options, which should be correct here.

@mikf
Copy link
Owner

mikf commented May 9, 2024

but with this the metadata files names always seem to read the extension of the images as "none"

There is no {extension} value available during "event": "post".
You'd have to use "event": "file" for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants