mjtokelly

Sounds like my understanding must be out of date as of this year! (Which makes me very excited about your upcoming library.) My conclusion was based on research ~1 year ago finding no OSS or commercial software supporting it, and forum posts for those projects by developers sadly explaining why they just couldn't do multipart yet.

My bad experience with files disappearing was entirely with web-based POST requests, ~2 years ago. Large file transfer from EC2 to S3 was reliable, but our POST requests on slower connections (even those that were very reliable) would return with a false report of success.

Direct uploading like this has been possible for at least two years. However, in our tests it turned out that Amazon was silently losing some files >100MB (and almost all files ~1GB).

On the AWS forums it was clear that AWS was aware of this consistent problem, but was not able to fix it, and not willing to document it.

They eventually released "multipart upload", which remains the only reliable way to get large files to S3. Unfortunately, multipart upload is nearly impossible to implement as a web app (short of resorting to e.g. Java).

Can you share the URL you're citing?