tessro

Right. And you can also ratchet up the statistical power you want, at the cost of increased sample size requirements.

> you do have good reason to believe that the choice is harmless.

The issue you will run into here is that 95% confidence means that you will only have a false positive 5% of the time. It does not mean a neutral finding is 95% likely to be neutral. The lever that controls that is statistical power, which is oft-ignored in conversations about A/B testing. Most statisticians use 80% power, which means a full 20% of neutral findings were false negatives.

Cool! I'm happy to provide guidance, and I'll take a pull request if you can get it working with some unit tests.

Yeah, that's a neat project. I was already thinking about building this before they launched, and their positive reception spurred me on.

If you're already using Heroku Postgres there's a ton of overlap between the services, but I'm hoping to provide a decent open source alternative for the rest of us. :)

If you can run Rack apps in your environment, it would be pretty straightfoward to add a SQL Server driver to Oculus. Right now it supports MySQL and Postgres.

Thanks! (Sadly I think the demo cron job will delete that after an hour or so :))

It's designed to be used within a team environment, where you trust everyone. (Although even then, I highly recommend using it with a readonly SQL account!) I would be skeptical of any situation where you could run raw untrusted SQL without any risks, at least without reimplementing MySQL's AST parser.

As for the demo, it's isolated, and locked down as well as MySQL will allow. (For starters, just SELECT privileges.)

Author here. I do a lot of analysis each day and am constantly sharing it with my team by taking screenshots of Sequel Pro. I built Oculus so I could keep a centrally available repository of my research, that everyone could access and collaborate on. It definitely has a ways to go, but I would love feedback!

Also, this is my first attempt at providing a live demo of software that can execute arbitrary SQL. I'm probably crazy for doing this, but I thought it was a lot more interesting than doing a video. If the demo suddenly disappears, you'll know I wasn't able to harden it enough... :)

This is an interesting study, but I found the HN title to be misleading. Here are the bullets from the concluding paragraph:

* The American public has long held generally positive attitudes toward the space program, but is not very familiar with its details.

* Over the history of the space age, an average of more than 60 percent of those polled rated the job done by NASA as either ‘‘excellent’’ or ‘‘good.’’

* Most Americans have shown support for space exploration and view it as important over the years, but also believe that federal money could be better spent on other programs.

* Most are also in favor of NASA as an organization, but are relatively unfamiliar with the majority of its activities and objectives.

* These polls also suggest historically close relationships between public perceptions of NASA and spaceflight depictions in popular culture, especially film. These images from popular culture, coupledwith real-world accomplishments in spaceflight, work together to create powerful visions affecting the public consciousness.

This is a powerful approach when you can quantify your regret. For many startups, however, it's important to understand the tradeoffs involved in moving one metric upward or downward. To take Zynga as an example, they care about virality at least as much as engagement (or perhaps moreso). Adding or removing a friendspam dialog is likely to trade some virality for user experience. What percentages make or break the decision? Sometimes this is a qualitative call.

In environments where you need to look at the impact of your experiments across multiple variables, and make a subjective call about the tradeoffs, it's really important to have statistical confidence in the movement of each variable you're evaluating. This is a key strength of the traditional A/B testing approach.

Unfortunately, screen sharing no longer works in Skype 2.8 on Lion.

I love Giles. His first Archaeopteryx talk was one of the best Ruby-related talks of all time. He's way too harsh on Rails though. The original "blog in 15 minutes" demos that catapulted Rails into the limelight are still possible with Rails 3. Things like 'gem install rails' still work. If you are determined to skip bundler, you can do that too. What's more, unlike 2005, in the 15 minutes it takes to build a Rails blog, you can deploy to Heroku and have a live site running.

Node is fun. V8 smokes MRI. But Node's web framework ecosystem has a long, long, long way to go before one can reasonably say that Rails is a bad investment. If anything, we should be celebrating the number of options that are open to us. It's a great time to be developing for the web. Good things will come from Node's ascendancy, whether its frameworks dethrone Rails or not. The negativity is unnecessary.