Depending on which kernel version you are, C5 (and M5) instances can be real sources of pain.
The disk is exposed as a /dev/nvme* block device, and as such I/O goes through a separate driver. The earlier versions of the driver had a hard limit of 255 seconds before I/O operation times out. [0,1,2]
When the timeout triggers, it is treated as a hard failure and the filesystem gets remounted read-only. Meaning: if you have anything that writes intensively to an attached volume, C5/M5 instances are dangerous. We experimented with them for our early prometheus nodes. Not a good idea. Having the alerts for an entire fleet start flapping due to a seemingly nonsensical "out of disk, write error" monitoring node failure is not fun.[ß]
If you run stateless, in-memory only applications on them (preferably even without local logging), then you should be fine.
0: https://bugs.launchpad.net/ubuntu/bionic/+source/linux/+bug/...
1: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1729119
2: https://www.reddit.com/r/aws/comments/7s5gui/c5_instances_nv...
ß: We handle nodes dying. The byzantine failure mode of nodes suddenly spewing wrong data is harder to deal with.
The disk is exposed as a /dev/nvme* block device, and as such I/O goes through a separate driver. The earlier versions of the driver had a hard limit of 255 seconds before I/O operation times out. [0,1,2]
When the timeout triggers, it is treated as a hard failure and the filesystem gets remounted read-only. Meaning: if you have anything that writes intensively to an attached volume, C5/M5 instances are dangerous. We experimented with them for our early prometheus nodes. Not a good idea. Having the alerts for an entire fleet start flapping due to a seemingly nonsensical "out of disk, write error" monitoring node failure is not fun.[ß]
If you run stateless, in-memory only applications on them (preferably even without local logging), then you should be fine.
0: https://bugs.launchpad.net/ubuntu/bionic/+source/linux/+bug/...
1: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1729119
2: https://www.reddit.com/r/aws/comments/7s5gui/c5_instances_nv...
ß: We handle nodes dying. The byzantine failure mode of nodes suddenly spewing wrong data is harder to deal with.