njohnson41

Even with the birthday problem, sqrt(52!) is still about 10^34, which is still huge. It's unlikely that any two (sufficiently random) shuffles in history have been the same.

Or maybe it's just a good excuse to carry fewer cups and pitchers around?

You're right (and nitpicking nitpicks seems appropriate to me :P).

But, I'm pretty sure the assumptions of logistic regression are even stronger than just that. The inputs are assumed to be independent given the output class, and the log odds of the output vary as a linear function of each input. The first one is essentially the naive Bayes assumption, and the second one is completely unreasonable for almost any problem ever (roughly equivalent to assuming every dataset has a multivariate normal distribution). If they are both correct, though, you get a perfectly good Bayesian posterior probability of each output class.

I think the lesson is that gradient descent will build a decent function approximation out of pretty much anything powerful enough, which is why neural networks still work even when probability theory has been thrown completely out the window.

Still, in any consistent way of assigning probabilities to events, if A implies B, then P(A) <= P(B).

Neural network outputs are not probabilities. I think that's the main lesson here.

I personally liked "Godel's Proof" by Ernest Nagel. It's pretty old, but it's to-the-point and inexpensive.

Fair enough; I'll be a little more precise.

I think the correct way to state it is: "all true statements are either tautologies, or can be determined empirically (with arbitrarily high but not necessarily measure-1 probability)". This statement is itself a tautology, because tautologies are by definition true, and because "determined" implies some method of determination, which if it actually can be used to determine truth, means it can be used empricially. Determination and empiricism are secretly defined in terms of each other, basically. The reason this tautology is worth stating is that it gives a simple criterion for discarding non-questions: questions that have no method of determining whether they are true or false (with arbitrarily high probability), are always non-questions.

Especially if you assume that the scientific method is always valid--that all true statements can be determined empirically--you can't make statements like "Ultimate Meaning doesn't/does exist". If Ultimate Meaning cannot be defined, it cannot be tested for or measured, so no statements about it can be true or false. The answer to meaning would not be false, but null (or maybe 42).

Just like 42, "atoms and the void" here is just a science-flavored attempt to answer a non-question.

> ... all possible combinations of 8-16 characters with 100 character possibilities ...

Yes, but this doesn't even come close to describing the typical users' password, which is most likely a 6-letter English word with a capital letter and a 1! appended to the end. Your calculation here isn't really relevant, because it's all about the worst or common case. (You also assume that people are using a GPU for a compute-bound problem, when much faster FPGAs are also available, but either way it's moot.)

Security through obscurity, which is what you're proposing with the shuffled salt idea, is also not normally considered the right way to go. If you wanted to use a similar but much simpler and straightforward method, you could just encrypt the salted hashes before storing them in the database.

This should be marked (2004), not (2014).

There could also be a simpler causal link: books and lucrative jobs could both caused by higher family socio-economic status.

Kolmogorov complexity is definitely meaningful, but it's not (Shannon) entropy, just conceptually similar. Many people think of something like Kolmogorov-complex sequences when they think of "random" sequences, which is (IMO) why they have trouble thinking of entropy as being about a probability distribution.

The one case where they coincide (sort of) is if you believe your random sequence is generated by a randomly chosen Turing machine, which I've only really seen in philosophical settings.

A uniformly chosen 64-bit integer still has exactly 64 bits of entropy, regardless of how much Kolmogorov complexity the actual bits you generate have.

Right, the amount of entropy per bit of sequence is always between 0 (deterministic) and 1 (every bit is independent and 50/50) (... or between 0 and log2(k) in general if the element varies over a set of k things). These "weak" sources just have low entropy per bit. They could be biased (more 0s than 1s) or correlated (long runs of 0s/1s or periodicity), or just have some other pattern that sometimes holds.

A deterministic PRNG's sequence has exactly the entropy of it's seed, actually, but it has 0 bits of entropy per symbol, because its sequence is infinite.

The thing most people get confused about with entropy is in thinking that entropy is a property of some single object, like a bit string. Really, entropy is always a measurement about a probability distribution, just like mean or variance is. In the usual case with random streams, the distribution is P(x_i | x_i-1 ... x_0) for bits x_i in the stream, i.e. the distribution remaining for the current bit even if we know all previous bits. For a deterministic PRNG, once we can extract the key from the history (given unlimited compute power) that distribution becomes deterministic, so the entropy is 0.

That's because this result is not about combining weak deterministic PRNGs, it's about combining entropy sources (like two hardware random number generators).

This has always been possible, but it sounds like they've lowered the minimum entropy needed in the source streams to produce a high-quality output.

Interestingly, that may be the new ordering if the disks are SSDs, but the typical seek latency on a spinning disk (~5 ms) is definitely higher than the latency to read data from another machine's memory across ethernet (a few hundred us), and even the bandwidths are comparable (~150 MB/s).

So, now it has jumped from (disk -> network -> memory -> ...) to (network -> disk -> memory -> ...), which is a big change.

Sounds a lot like python's "pickle" module (which is super-useful for prototyping), but with the same achilles' heel: all of your serialized objects can now run arbitrary code when you deserialize them!