colinwilson

Joined 13 years ago

Karma
30
Submissions
14
Member since
May 6
Data engineer who loves hiking, running, and beer.

email: user at gmail dot com

View on Hacker News

Recent submissions

Comment · 3 years ago
Comment · 3 years ago
Comment · 6 years ago
Systems like Hadoop and Spark will run multiple versions of the same task writing out to a FS at once but to a temporary directory and when the first finishes the output data is just moved to the final place. It's not uncommon for a job to "complete" writing data to S3 and just sit there and hang as the FS move commands run copying the data and deleting the old version. It is just assumed a rename/move is a no-op in some systems.
Comment · 9 years ago
For runs it's mostly just for fun. I use Strava so it has sections they identify along your run that they rank you against your friends and strangers. I do end up running very similar routes most of the time.

I mostly find the GPS invaluable in hiking. It can be a great thing to have on the top of a mountain in the winter with low visibility. If you get up to a summit and the weather turns, you can have the watch send you back down the way you came. Nice and safe.

Comment · 9 years ago
You can see how off the trace can be here: https://i.imgur.com/eHJI8FH.jpg . You could probably do some smoothing to help fit some of this to a street but I've been on runs where I zig zag down streets just to add distance or to do hills.
Comment · 9 years ago
As an avid hiker and runner I'm more excited to see what this will do for things like Garmin smart watches. I run through downtown Boston a few times a week and the accuracy is very bad at times making you look like you run much farther than you do.
Comment · 9 years ago
I would say it can be used at it's simplest as a replacement for cron. It supports running programs on a schedule and you can set concurrency rules, SLAs, and triggers around even single commands or programs. You also get nice graphs of task run time and email alerts if jobs take longer than the SLA you've set.
Comment · 10 years ago
Comment · 10 years ago
Comment · 11 years ago
I might be making this all up, I'll need to actually test it out now. I swear when watching Netflix on a computer, you start an episode and your cursor is auto-hidden. At some point, maybe even when the second episode starts, I swear the cursor becomes visible again on the screen. I then instinctively giggle my finger on my touchpad to make it hide again. After watching couple episodes in a row and doing that, if I don't I feel like it pauses and ask if I'm still there within 20 or so seconds.

Has anyone else noticed that? My theory is that they reveal the cursor after every new episode to try to get you to move it and hide it. If you do that each time, they could assume you're not paying as much attention.

Comment · 11 years ago
I was reading through the wikipedia page for the building and it mentions that when it was built there was a bylaw that required kitchens and bathrooms to have a window in them.
Comment · 11 years ago
I've just started using Deis at work recently. Dokku looks like it was a small project sponsored by the company behind Deis. I haven't looked much at dokku-alt.

The thing that I really like about Deis is it supports deployments off of docker images in a private repo on top of git push deploys.

One of the things we do is host a REST service backed by data we generate. We can have jenkins automatically build and publish a docker container to our internal repo, then it's two REST calls to Deis to update the docker container that is running. We generate the data in Hadoop through a longish oozie workflow weekly and we can make two REST calls to repoint the REST service to some new database tables.

I also like that each layer or part of Deis can be swapped out. Want to run haproxy for the load balancing layer instead of nginx? Someone probably has a container that can be dropped in instead. Want swap a more advanced scheduler for etcd, you can swap in Mesos, YARN, or even Flynn's scheduler.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton