Import: Better handling of truncated WebDAV uploads.

[kevincox]

Describe what problem this solves and why this would be valuable to many users

Sometimes WebDAV won't complete. Maybe the network dropped, maybe the battery ran out or maybe PhotoPrism was restarted.

The exact behaviour will depend on the upload/sync tool that you are using but with my scenario (FolderSync) what will happen is this:

1. Upload partial file.
2. Partial file is imported.
3. Upload complete file.
4. Complete file is imported.

This results in two items in my PhotoPrism library. Ideally this common scenario could be handled so that only the complete file ends up in the library.

Describe the solution you'd like

When a file is imported and there are existing files with the same original name and a smaller size PhotoPrism should check if the existing file is a prefix of the new file. If so the old file should be replaced.

Possible tweaks:

1. What if there are tons of files with the same original name and smaller size. Maybe this should be limited in some way? Maybe only consider the most recent 100 or only files uploaded in the past 30 days?
2. Are there common situations where a non-truncated file is a prefix of another file? Maybe we do this if the original is damaged in some way? A recency check like above could also mitigate this.

Describe alternatives you've considered

1. If a WebDAV upload connection drops delete the file.
   • Pro: Simple.
   • Con: This would prevent resuming the upload.
2. If a WebDAV upload connection drops flag that as "unfinished" until a new connection overwrites/completes the file.
   • Con: What if the upload did complete, then the next upload would never come.
3. Detect corrupt files and don't import them.
   • Pro: Upload can be resumed forever as the file will sit in the imports folder even past an import.
   • Con: The photo can't be managed in PhotoPrism until the full upload occurs.
   • Con: What if the full file never comes, it may be better to import the partial file. Would this need an alternative process?
4. Import the file but don't delete it from the upload folder.
   • Pro: Upload can be resumed forever as the file will sit in the imports folder even past an import.
   • Pro: Can be managed immediately.
   • Con: If the full file never comes the download folder will be cluttered. (Possibly causing issue if someone wants to upload a file with the same name).

Additional context

The files I were uploading were JPEG files from the Google Camera app. The truncated files were handled relatively well by PhotoPrism with the image just grey after some point (where the truncation happened). However needing to review all of the uploads later for files with the same name and deleting the smaller one was annoying, tedious and error prone. In some cases I may not have realized that this happened and had these truncated versions filling my library.