tlipcon
Joined 17 years ago
Karma
366
Submissions
75
Member since
Jul 10
Recent submissions
Comment · 4 years ago
Comment · 6 years ago
Just a small note: Kudu shares no code or provenance with Hadoop. Only commonality is ecosystem (eg you can use spark or Impala to query it)
Comment · 9 years ago
For those interested in a technical overview, http://kudu.apache.org/kudu.pdf is our academic-style (but not submitted to any journal) paper.
Comment · 9 years ago
Apache Kudu project founder here:
It's true that the project was initially developed at Cloudera, and employees continue to be the main driving force behind development. That said, we have committers and contributors from other companies as well. Roughly half the people who contributed a patch in the last 3 months have been non-Cloudera. Additionally we are very strict about doing all development upstream (eg with the first open source release we spent a lot of effort to open the entire development history going back to 2012, including JIRA, git, etc).
As for users, here are a couple examples off the top of my head who aren't currently paying for any support:
- Xiaomi (world's 4th largest smartphone maker) collects ~2TB/day of event data from >5million phones into a cluster which simultaneously runs analytics workloads (SQL, Spark, etc) - CERN is looking at using Kudu to store high energy physics experiment data from the ATLAS detector at the LHC. You can find some code at https://gitlab.cern.ch/zbaranow/kudu-atlas-eventindex and a poster here: https://indico.cern.ch/event/505613/contributions/2230964/at...
(of course lots more too whose names I dont have permission to mention)
Feel free to join our slack if you're interested in chatting with more - usually plenty of people online here: https://getkudu-slack.herokuapp.com
-Todd
Spanner stores its data in Colossus, so there would be some bootstrapping issues to resolve to move it to Spanner over Bigtable. (Bigtable also has the bootstrap issues but has solved them already and there are additional difficulties due to some details that I probably am not at liberty to share)
Spanner is used for metadata for many other very very large storage systems, though.