Absolutely a believer in the power of single machine analytics on GB or TB vs PB datasets. But I've found that whether it's a new data system or just showing csv/tsv tool magic, if people aren't into data systems (for the former) or unix cli (for the latter), it's a whole culture/training thing vs. most technically efficient wins.
So I think the latter issues dominate (culture/training) for new implementations as well as existing.
Re: efficiency overall, and of distributed systems, it's interesting to see both that MapR has made a business selling more efficient Hadoop-ishness, but also depressing how infrequently I see them deployed, even for pretty massive clusters.
Unfortunately, it's hard to see a startup focusing on single-machine solutions for GB->TB sets (even scaled out with replicas) as the way many in the space get started is with an open core model and the thing they charge for is the clustering and/or monitoring needed to become a distributed system.
But... I am optimistic we'll see a generational effect over the next 10 years in openness/interest in composable ad-hoc analytics tools, especially with Windows incorporating unix cli components.
So I think the latter issues dominate (culture/training) for new implementations as well as existing.
Re: efficiency overall, and of distributed systems, it's interesting to see both that MapR has made a business selling more efficient Hadoop-ishness, but also depressing how infrequently I see them deployed, even for pretty massive clusters.
Unfortunately, it's hard to see a startup focusing on single-machine solutions for GB->TB sets (even scaled out with replicas) as the way many in the space get started is with an open core model and the thing they charge for is the clustering and/or monitoring needed to become a distributed system.
But... I am optimistic we'll see a generational effect over the next 10 years in openness/interest in composable ad-hoc analytics tools, especially with Windows incorporating unix cli components.