I like this idea but I think the design needs to be pretty drastically rethought. Node.js is not really the right medium for this sort of test, although it might function as a first barrier for screening the interested; "you must be able to follow readmes to install software" etc.
What I'm more concerned is the idea that this is apparently structured like an automated test. If you used it that way, then here is a function which would pass the first test file:
fn = function (a, b) {
switch (b) {
case 3: return 2;
case undefined: return 10;
case 2: return '1 3 4'.split(" ");
case 10: return [0, 0, 0, 0, 10];
case 'z': return '1 2 z 3 4'.split(" ");
default: return '1 2 3 4 a b c'.split(" ");
}
};
It was built by just looking at the "expect" calls, finding the input to the function and the output from it, and duplicating that output.
That's obviously not what you want to test. You really want to say, "you give me an object, and I'm going to generate a couple random test vectors and make sure you do the right thing with those vectors." You shouldn't hardcode the result, but the right algorithms, so that someone else's solutions can be measured against the right way to do things. And then you need to ask people not to modify the right solutions, but to duplicate them.
What I'm more concerned is the idea that this is apparently structured like an automated test. If you used it that way, then here is a function which would pass the first test file:
It was built by just looking at the "expect" calls, finding the input to the function and the output from it, and duplicating that output.That's obviously not what you want to test. You really want to say, "you give me an object, and I'm going to generate a couple random test vectors and make sure you do the right thing with those vectors." You shouldn't hardcode the result, but the right algorithms, so that someone else's solutions can be measured against the right way to do things. And then you need to ask people not to modify the right solutions, but to duplicate them.