mpetrov

We kept the design simple, otherwise a "batching" mechanism could be introduced that would replay a single batch of 50 messages but that would make everything a second delayed. However, part of the map allows you to login and see your messages live on the screen as you're sending them to your partner and for that the real time streaming is pretty critical.

Comment · 12 years ago

I tried using faye and it died very quickly under load with the redis backend. For every message that you send it queues it up into a redis list for each faye server in the cluster to read which doesn't scale very well in their very chatty redis queue system.

I would do significant load testing for each use case before using it (or socket.io for that matter). I needed to push 30-50 messages per second to each connected client and faye started choking as soon as there were more than 20-30 clients. socket.io would choke at around 50-100 connected clients. Raw websockets were able to reach closer to 100 clients but performed more or less similar to socket.io.

Comment · 12 years ago

Sticky load balancing was a deal breaker for us when we built the https://map.couple.me visualization with socket.io. At the last minute I just pulled older browser support and left it with raw websockets instead of socket.io because the cluser module couldn't be used otherwise.

We had two issues with sticky load balancing:

1. AWS ELB had to run in TCP mode (for websockets) which meant no stickiness

2. Within each instance we have a node-cluster running to utilize multiple CPUs and putting haproxy with stickiness there increased complexity

We could avoid using the ELB but then you lose the ability to use an EC2 Auto Scaling group, introducing significant admin overhead.

Ultimately we didn't need the nodes communicating between each other (it's a one way stream from us to the client) so raw websockets ended up being the simplest solution.

Comment · 12 years ago

Good catch! There is a small variation depending on the server that you hit but the variation should never be more than 1,000 messages. It's just a side effect of how we're scaling the system.

Comment · 12 years ago

It's the geographic center of Canada, that's the coordinate that the geo-ip database gives us back when no specific city is available. Works well for smaller countries, but in Canada it's quite visibly "off" :)

Comment · 12 years ago

Those data points appear as circular flashes without a line, there are quite a few of them there. The app appeals to a lot of long distance Couples so the map is somewhat biased towards long distance communication.

Also if you look at big cities (LA, NYC, SF) you'll see a continuous stream of circles flashing without lines. So the data just overlaps a lot.

Comment · 12 years ago

That's pretty much exactly what's happening with some locations. We do our best to get an accurate location based on a few sources, but ultimately Maxmind is the database we fall back to for pure IP based location if that's all we have.

Comment · 12 years ago

We're planning to do a larger tech focused write-up on this infographic soon. On the backend it's powered by Node and Redis and runs within an EC2 auto scaling group.

Let me know if I can elaborate on any of the tech behind this visualization.