jethroalias97

Assuming you are using Riak's default configuration each range query hits 1/3 of the cluster, which could get pretty hairy on large clusters that have lots of requests. Also, there is no pagination, so if an index has a million objects you'll have to be prepared to wait even if you only want the first part of the query.

You could solve this by putting a sort value in the key and using a range query, but this wouldn't work if you want the most recent items keyed with time, because the items could be unevenly spaced back in time. Also, Riak, like many databases based on Dynamo, thrives on fat data which one would think would favor lists. LevelDB is also supposedly slower than Bitcask, the default backend, but I'm not sure if this is still true.

I've been trying to think of ways around these problems. A simple thought I had was to simply cache the response as pages in Riak. Although this introduces new problems like how to know how often to reset the cache, too often and I may as well not have this cache, too infrequently and users get stale data. I would also have to handle this using worker threads because I wouldn't want the odd 100th user to get a big latency hit. The database would also either have to be continually polled, wasting CPU, or potentially not have the data cached when needed.

Another solution I've been considering is to write a secondary index layer on top of Riak using a skiplist or btree to know where to add and remove data when it gets to be very large. This seems like a cool idea, but might be tricky to implement and do conflict resolutions on.

My last idea was the most ambitious, which was to implement a separate distributed database specifically for secondary indexes and range queries which would not be bound by Dynamo. The idea here is to have each node in charge of a segment of the key space (like Big Table) and then have it split and coalesce not only based on size, but also on frequency of reads and writes to handle the bottleneck problem.

I would think that any new bitcoin wallet could be introducing a substantial security risk before it has been extensively field tested. If someone where to sneak some code in, or even just screw up a security protocol, all the stored bitcoins would be at risk.