gane5h

The culture around cryptocurrencies can be a little insular – so we're sharing our onboarding guide that teaches you to buy cryptocurrencies step-by-step. I think it's broadly applicable regardless of what you think about the viability of crypto.

I've used this in the past, in high performance math.

If you have data (vectors, matrices, etc.) that doesn't fit neatly into a SIMD block size, you'll have to zero out fields after the calculation. At this point, it's cheaper to generate a zero on the register than load via memory (cheaper as in the number of CPU instructions.)

Congrats to the compose folks! IIRC, they acquired Cloudant a little while ago.

We store our event stream data in Elasticsearch. Two features that made it appealing:

  * the ingest-side can be scaled up by adding more shards
  * the query-side can be scaled up by adding more replicas

To compute rollup analytics, we make heavy use of Elasticsearch's aggregation framework to compute daily/weekly/monthly/quarterly active users.

From my understanding Postgres has many of these features, but the distributed features of ES are killer!

Really cool write up – thanks! First time I’m hearing about CitusDB. They appear to be building a columnar, distributed database while preserving the Postgres frontend (similar to redshift, aster, greenplum, etc.)

It’s all in the details. I’m planning to investigate the following during my next weekend hack. Hope somebody can answer some pre-sales questions for me:

  - how complete is the postgres functionality (e.g.: lateral joins)
  - can you set a sharding key to control the shard distribution
  - does the database do multiple passes for queries with subselects
  - usually one increases the replication factor (limited by budget) to improve query times, with the limitation that it slows down loading time. does the DB stage intermediate writes to batch them, so does the user need to do this? this works really well for append-only, timestamped event data.
  - do you have a job manager or scheduler, needed when you have multiple views that need to be updated without melting your infrastructure
  - how easy is it to operate? does the database expose operational metrics so that you can see the load on each shard to potentially detect unbalanced shards?
  - tips on hardware configuration (big advantage of redshift here is that you don’t have to run your own warehouse.) maybe partner with MongoHQ?

It’ll be nice to see some sample query plans graphically visualized.

I learnt this the hard way. Initially, I designed the API with bigints for ids and found some older versions of PHP didn't support bigints. I had to switch to using strings.