obastani

At the very least, many of the papers in other domains are co-authored with experts in that domain.

I don't know if this is the future, but if it is, why bother building one version of the software for everyone? We can have agents build the website for each user exactly the way they want. That would be the most exciting possibility to come out of AI-generated software.

It's not about training directly on the test set, it's about people discussing questions in the test set online (e.g., in forums), and then this data is swept up into the training set. That's what makes test set contamination so difficult to avoid.

If I'm understanding correctly, by "income", this article means "actual individual consumption", which is the amount of money spent by a household. Thus, the article is saying that for countries where households spend more, they spend more on healthcare. Given that healthcare is a huge fraction of household expenditures (almost 20%), this seems tautological. Am I misunderstanding something?

This breaks down because it's not easy to statically reason about when a variable is a NonZeroNumber. For example, what is the type signature for subtraction of two NonZeroNumbers? You can't guarantee that it isn't zero, so it has to be a Number. Thus, you can't divide by the difference. You could use a more powerful type system to reason about these kinds of constraints, but then type checking quickly becomes undecidable (or at least very, very expensive).

Important caveat with some of the results: they are using better prompting techniques for Gemini vs GPT-4, including their top line result on MMLU (CoT@32 vs top-5). But, they do have better results on zero-shot prompting below, e.g., on HumanEval.

> So, the 2% growth rate for world energy consumption should be a conservative assumption.

An important caveat: this article assumes that energy consumption will continue to increase exponentially to get the 1000 year timeline of draining the rotational energy of the Earth.

I don’t really understand the argument that corporate profits are driving inflation. If demand for a product goes up, we would naturally expect prices to rise since there is more competition for a fixed set of goods. Since the cost of producing the product has not increased, corporate profits necessarily rise. In the long term, we would expect competition to drive prices back to marginal cost as production of that product increases, but this process can takes time.

In this case, how do you separate inflation due to increased demand from inflation due to corporate profits? The two seem inextricably tied to me.

Just out of curiosity, in what sense is Codex is better trained than CodeGen?

I’m not sure I buy the conclusion. The point in the original story (at least the version I’ve read) was that Mel’s main job was to write code that could make the computer appear as fast as possible to potential customers. Of course one hack isn’t going to noticeably improve performance, but a collection of hacks can significantly improve performance.

I'm not sure about that. If Mary knows all the physics of light, she can design and set up a sensor to detect the color of the apple and the instructions, just like we have sensors that detect ultraviolet and infrared light. Just as we will never know how it feels to "see" ultraviolet light, this thought experiment is about Mary's subjective experience perceiving color, not her ability to determine the color of the world around her.