The HuggingFace datasets python package is free and can handle images inside Apache Arrow just fine. Combined with a few Exos, NVMes, and dm-cache, storing TBs of data becomes very affordable and I have direct 10G access and can mmap on the network share.
In my opinion, this space will be won based on price per throughput.
But you're charging $1k monthly for 1TB? And my hobbyist speech recognition projects (30TB) vastly exceed your enterprise plan? That discrepancy makes it challenging to believe that your team has as much experience with actual AI projects, as the reference to self-driving cars would suggest.
Anything that easily fits onto a consumer NAS is probably not an enterprise data lake. A one-time €1700 purchase (equivalent to 2 months of your service) will buy me a TS-464-4G with 32TB RAID1 and 5 years of warranty.
In my opinion, this space will be won based on price per throughput.
But you're charging $1k monthly for 1TB? And my hobbyist speech recognition projects (30TB) vastly exceed your enterprise plan? That discrepancy makes it challenging to believe that your team has as much experience with actual AI projects, as the reference to self-driving cars would suggest.
Anything that easily fits onto a consumer NAS is probably not an enterprise data lake. A one-time €1700 purchase (equivalent to 2 months of your service) will buy me a TS-464-4G with 32TB RAID1 and 5 years of warranty.