asharp

It's more of a capability thing. If you're running, say, Piston cloud you're using ceph over ethernet to back your disks, so you can easily decouple disk usage and ram usage. If you're stuck using local disks (ie. rackspace/joyent/linode/amazon to a point/etc.), then it's a lot harder to provide that sort of product.

That being said there are providers out there that sell it, and have been for years.

This makes sense when you keep in mind how the older clouds work (essentially VPS providers 101).

You have a server with some disks, some ram and some cpus. You aggregate the disks together, then split them to form the individual disks for the virtual machines. You then use kvm/xen to provide isolation as well as to split the ram/cpu between the virtual machines.

So to answer your question: Storage/ram/cpu is sold in lock step because otherwise there would be resources sitting on servers that are unable to be sold. Bandwidth isn't constrained like that because bandwidth isn't a thing tied to a machine.

There are some providers out there that don't lock ram/disk together. This is mostly because they use a distributed storage pool rather than local disks. This is significantly more complex and is a 'fairly' new addition to the scene (~2010?).

This is also why certain providers still charge you for ram even when your machine is turned off, and why backups/migra tions/plan upgrades can be a bit of a pain in the neck at times.

Key value stores are useful, and they are especially useful in this form factor. On the other hand, you now have a very large black box that you have to somehow navigate in order to create a workable system. Given that this is likely an arm core running linux on the inside, I would have considered a slightly more open approach to be 'Here's a working KV store using backing db X and here's how to reflash it if it doesn't quite work for you'.

Bitrot? Error recovery? SMART data? ...?

Using external kernels isn't in any way fundamental to the way Xen works. Most standard setups (even PV) have the kernel inside the VM, which allows for standard upgrades, etc.

I think you should take a quick look at oracle's exadata machines. Mostly because they are both distributed and ACID compliant.

Does this method have a name/are there any reasonable papers out there describing it?

I saw that, but it doesn't seem to implement any of the indexes you'd need in order to run a bloom query efficiently.

That being said, if you stored your bitvector as an array of powers of two, then it would work. But that would be horribly inefficient in terms of space usage.

I believe that you can index the bloom column, to some extent, with some hacks.

A major problem is a lack of indexes in array elements, which then forces you to build an otherwise unneeded table.

That being said, it's definitely doable, to some extent.

Cloud done right. Hardware that is approximately indistinguishable from tin sitting on your datacenter floor, pay by the second.

To elaborate slightly. I mean hardware that you can run anything on, complete with full BMC, fast persistent disk storage (1K IOPS+), and any number of full layer 2 cross connectable networks. All pay by the second.

Already a major company in Australia with enterprise and government clients. Coming to the US soon.

Cool speed hacks.

An an improvement though, you only need two independent hash functions to run your bloom filter[1]. Strangely enough, this isn't well known and as such isn't implemented anywhere near as often as it should be (ie. it's not implemented here).

[1] www.eecs.harvard.edu/~kirsch/pubs/bbbf/rsa.pdf

Mmmm. Strange.

Anyways, the major problem is that shares and options are taxed like income at the time of issue, with no easy way to defer the tax bill. This makes it essentially untenable to hand out equity to employees. This then makes it a lot harder to employ people as a bootstrapped startup.