server: add dynamic configuration for download variables

[jzila]

Issue

I've found that downloads can be unreliable for large models, either due to errors during the download or during processing the digest.

Workaround

Add some environment variables to the server to allow runtime configuration of the number of parallel downloads and how big the chunks can be.

Follow-up

Ideally the server would be configured to respect the ?partNumber query parameters that the returned x-amz-mp-parts-count header implies should be supported. I'd also like it to support x-amz-checksum-mode=ENABLED (currently returns a 501 NOT IMPLEMENTED), so that each part number returns an expected digest in the response headers for GET and HEAD. This would enable us to split the digest by parts, so that if a part download fails, we don't need to retrieve the full model each time.

It seems that the current implementation is done via github.com/distribution/distribution, which delegates its Range handling to http.ServeContent, and as such does not support the partNumber functionality implied by the CloudFlare response headers. So any such support would have to fork github.com/distribution/distribution. CloudFlare also does not appear to support these headers.

[jzila]

@jmorganca what's the protocol for contribution reviews? I'd love to get this merged if it meets the bar.