I recall Joyent's solution to this (similar) problem where you have an object stored somewhere (e.g. S3) and you want to use that object in a container, but you have to copy it over HTTP or something to do any work on it and the object could be very large.
With Joyent's Manta[1] you would spin up a container right where an object is stored (instead of bringing the objects to the container via NFS.) Also has map reduce support.
I recall Joyent's solution to this (similar) problem where you have an object stored somewhere (e.g. S3) and you want to use that object in a container, but you have to copy it over HTTP or something to do any work on it and the object could be very large.
With Joyent's Manta[1] you would spin up a container right where an object is stored (instead of bringing the objects to the container via NFS.) Also has map reduce support.
[1] https://apidocs.joyent.com/manta/jobs-reference.html