Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] - Import Code Fails with "Invalid or unsupported zip format. No END header found" #933

Open
Dolphindalt opened this issue Jan 6, 2023 · 7 comments · May be fixed by #1018
Open

[BUG] - Import Code Fails with "Invalid or unsupported zip format. No END header found" #933

Dolphindalt opened this issue Jan 6, 2023 · 7 comments · May be fixed by #1018
Labels
bug Minor issue

Comments

@Dolphindalt
Copy link

Dolphindalt commented Jan 6, 2023

Describe the bug
This issue is occurring with Valheim, but I suspect that #916 is experiencing the same issue with RoR2. A friend of mine is exporting their mod profile as a code so that I may import the mod profile as a code. Attempting to do so produces the error, "Invalid or unsupported zip format. No END header found".

To Reproduce
Steps to reproduce the behavior:

  1. Install Thunderstore app
  2. Create a profile for Valheim and install mods
  3. Launch Valheim for initial setup of mod files
  4. Export profile with a code
  5. Install and launch r2modmanPlus
  6. Select Valheim as the game
  7. In Profile Selection, choose "Import/Update"
  8. Choose "Import new profile" or "Update existing profile" (both provide the same result)
  9. Select "From code"
  10. Paste the code into the input element and press "Import" (screenshot provided)
  11. See error (screenshot provided)

Expected behavior
A new profile should be created containing the mods associated with the exported profile represented by the code.

Screenshots
2023-01-06-024103_646x245_scrot
2023-01-06-023841_650x181_scrot

Additional context
The code that we were using: 0185868e-7969-ef67-b878-7db08f3a0d45

The mod zip file appears to be found over HTTPS. Perhaps the zip file is not downloaded completely before the extraction process begins?

The error is caught and displayed in the following code from src/pages/Profiles.vue and would be a good place to start investigating. Will do this myself if there is time; otherwise this could be a good first issue for others.
2023-01-06-024724_679x158_scrot

My setup may be a factor in the presence of this error.
2023-01-06-025249_342x306_scrot

@Dolphindalt Dolphindalt added the bug Minor issue label Jan 6, 2023
@MythicManiac
Copy link
Collaborator

On my machine importing the profile code you provided works as expected. Or in other words, I'm personally unable to reproduce this.

Let me know if you manage to log anything more detailed. For starters, can you check what's happening with the network traffic if you have the inspector open? (Ctrl+Shift+I should work I think)

@Dolphindalt
Copy link
Author

Dolphindalt commented Jan 7, 2023

I am having trouble getting the devtools console to open at all; be it on the available production build and running locally with npm run-script run. I was able to verify (using the error display function and changing a line) that the get request for the import.r2z file succeeded with status 200. Need to get the console up to be able to provide more details. Is there a way to run this application in the browser natively? Otherwise, I will try have to try the application on a more mainstream WM.

Edit: After doing some digging, it appears other Linux users are having this issue with Electron 11 and the devtools not appearing. Once I get this resolved, I will return to posting more detail on this issue relating to the mod manager.

@otDan
Copy link

otDan commented Jan 15, 2023

I actually get the same thing when using thunderstore exported profiles for ROUNDS.
image

@MythicManiac
Copy link
Collaborator

This has been narrowed down a bit from my understanding, and the issue seems to be that Thunderstore Mod Manager creates ZIP exports via some C# built-ins, and the resulting archive does not seem to function on the nodjs side (r2modman). This results in exports created from TMM to not be loadable in r2modman.

We're not sure what specifically causes this disparity and I've not personally verified it either. In case someone is interested in looking into it more, here's a sample of the c# zip creation code:

namespace Zipper
{
    public class ZipCreator
    {
        private static readonly Dictionary<String, KeyValuePair<MemoryStream, ZipArchive>> Archives =
            new Dictionary<string, KeyValuePair<MemoryStream, ZipArchive>>();

        public void RequestKey(Action<object> callback)
        {
            string key;
            while ((key = Guid.NewGuid().ToString()).Length > 0 && Archives.ContainsKey(key)) { }

            var stream = new MemoryStream();
            Archives.Add(key, new KeyValuePair<MemoryStream, ZipArchive>(stream, new ZipArchive(stream, ZipArchiveMode.Update)));
            callback(Utilities.JsonEncode(new Result(ResultType.STRING, key)));
        }

        public void AddBuffer(string key, string name, byte[] buffer, Action<object> callback)
        {
            Archives[key].Value.CreateEntry(name);
            Archives[key].Value.GetEntry(name)?.Open().Write(buffer, 0, buffer.Length);
            callback(Utilities.JsonEncode(new Result(ResultType.SUCCESS, null)));
        }

        public void AddFolder(string appName, string appId, string key, string folder, string folderName, Action<object> callback)
        {
            callback(Utilities.RunSafe(appName, appId, folder, () =>
            {
                AddFolderInternal(key, folder, folderName);
                return new Result(ResultType.SUCCESS, null);
            }));
        }

        private void AddFolderInternal(string key, string folder, string folderName)
        {
            var safeFolder = Utilities.SanitisePath(folder);
            foreach (var file in Directory.GetFiles(safeFolder))
            {
                var zip = Archives[key].Value;
                zip.CreateEntryFromFile(file, Path.Combine(folderName, new FileInfo(file).Name));
            }

            foreach (var directory in Directory.GetDirectories(safeFolder))
            {
                AddFolderInternal(key, directory, Path.Combine(folderName, new DirectoryInfo(directory).Name));
            }
        }

        public void Create(string appName, string appId, string key, string output, Action<object> callback)
        {
            Archives[key].Value.Dispose();
            var buffer = Archives[key].Key.GetBuffer();
            new FileOperations().WriteFile(appName, appId, output, buffer, callback);
        }
    }
}

@Dolphindalt
Copy link
Author

This might be a hint. I tried unzipping the archive the mod manager downloads and tries to read with Linux tools. Firstly, there is unzip, which fails. 7z produces warnings about extra data being included at the end of the zip file; perhaps this is a clue?

zip_hint_unzip

zip_hint

@MythicManiac
Copy link
Collaborator

The file downloaded by the manager isn't a raw zip file, but rather a base64 encoded zip with a string prefix of #r2modman\n.

The code snippet encoding the content is as follows:

const profileBuffer = '#r2modman\n' + (await fs.base64FromZip(exportPath));

@wagyourtail wagyourtail linked a pull request Apr 26, 2023 that will close this issue
@MythicManiac
Copy link
Collaborator

This should be addressed on the thunderstore mod manager side now in that profile exports crated with version 1.37.0 or upwards no longer contain extra data at the end of the file. Older profile exports are still of course incompatible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Minor issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants