Comment by derefr

It's one of those paradoxes inherent to software development: even if a tool is designed to work with Big Data, developers are too impatient / feel too unproductive to actually work with Big Data during development, so they only bother testing their tool using small data for 99.9% of development. Problems the tool has under actual Big Data tend to only start being found as of v1.0-rc1.