NonBlockingFileIO.read(...) don't error if more than 4 GiB is requested

[weissi]

A ByteBuffer can never hold more than 4 GiB of data. But the NonBlockingFileIO.read(...) functions don't complain if more than that is requested. They should error saying that this request can't be fulfilled.

Likewise the readChunked functions cap out at (IIRC) 2 GiB and don't complain if you request more than 2 GiB chunks.

Both need to error.

So

• read: Work through 4 GiB, error > 4 GiB
• readChunked: Work up through 2 GiB, error > 2 GiB (the reason for the 2 GiB limit is that IIRC the underlying posix functions don't work if > CInt.max, this should be checked though).

[dnadoba]

We should also upgrade some of the Int types that express file sizes to Int64 so 4GB+ sizes can actually be supported on 32bit systems. I think this only affects the byteCount parameter of readChunked. This can be done separately and will need to preserver and deprecate the old function signatures. Note that offset is already Int64.

[weissi]

We should also upgrade some of the Int types that express file sizes to Int64 so 4GB+ sizes can actually be supported on 32bit systems. I think this only affects the byteCount parameter of readChunked. This can be done separately and will need to preserver and deprecate the old function signatures. Note that offset is already Int64.

We don't need > 32 bit chunks. Reasonable chunk sizes are in kiB or MiB

[dnadoba]

Agree but I'm talking about the byteCount parameter, not the chunkSize parameter of the readChunked(...) method.

[weissi]

Agree but I'm talking about the byteCount parameter, not the chunkSize parameter of the readChunked(...) method.

Fair enough!