Proposal: Support chunked file uploads

[MarvinJWendt]

I have my server running behind a Cloudflare reverse proxy. Cloudflare only supports requests that are <= 100 MB. In practice, this means that I cannot upload files larger than 100 MB to the server.

I propose that chunked file uploading is added to PocketBase. This would split the file on the client side (could be done in the frameworks, for user-friendliness) and then upload the chunks in multiple requests. PocketBase will then stick the parts back together on the server side and save the full file.

Benefits of chunked file uploads:

• Faster Uploads: Different chunks can be uploaded simultaneously across multiple connections, potentially increasing the use of available bandwidth and reducing the overall upload time.
• Bypassing Size Limits: It allows bypassing the maximum file size limits imposed by servers, reverse proxies, or middleware (such as Cloudflare's 100 MB limit).
• Improved Reliability: If the upload process is interrupted due to a network issue, only the current chunk needs to be resent rather than the entire file, which can save time and bandwidth.
• Progress Tracking: Chunked uploads make it easier to provide accurate progress indicators to users, as each chunk can be tracked individually.
• Network Efficiency: Smaller data packets can reduce the likelihood of timeout errors that might occur with larger uploads, especially on slower or unstable connections.

[gedw99]

https://github.com/tus/tusd

[xFrann]

I have encountered the same issue while running pocketbase behind cloudflare, this would be a great addition

[gedw99]

Cloud flare should not affect it ..

[MarvinJWendt]

Cloud flare should not affect it ..

As stated in the proposal, it does limit client requests to 100MB, which includes file uploading.

[gedw99]

https://github.com/tus/tusd

I don't know if this will fix the Cloudflare issue btw

[thangngoc89]

Maybe this could be circumvented by uploading using pre-signed S3 url?

Pocketbase SDK could generate the pre-signed URL and keeping track of the metadata after uploading somehow.