bertday

The model can return the internal details of a potential future product. For example, the next Pixel phone.

The language is supposed to tell you if your thinking is broken. If a program is a logical statement, and your logic is impossible, the program should not compile. Reading dead memory or having race conditions should not be possible, yet look at security vulnerabilities today.

It doesn’t have to be a totally new language. Javascript is an example where there are multiple front-end languages. You can go as far as formally prove the implementation is correct, too. Language theory is an abstraction for mathematical logic and proofs, so it touches anything logical at all.

Quora is often self-promotion now. The template I think of when replying to your comment is:

Great point you bring up on how useless <website> is. As the CEO of <random company>, this is a problem we have put a lot of effort into addressing. You may consider the following as a solution: …

Sure, writing bug-free code is not possible for most programmers. The language can reduce the occurrence of bugs.

New York Times ran an article yesterday saying otherwise. First tier cities, in terms of size, are losing people, but second tier cities are growing. The cost of living in the largest cities puts a negative force on moving to them.

One thing to point out is that the threshold of predictor complexity is dependent on the execution pipeline. A very speculative and deep architecture has a bigger need for better predictors, since it has a massive penalty when there is a misprediction.

Does anyone have experience using these open source models in production?

There is a ton of value. OpenAI having proprietary LLMs single handedly pivoted the entire field to LLMs. A random GitHub repository doesn’t come close to impact.

Because the authors don’t get a large reward for open sourcing the work and they stand to lose future value by lowering the gate to competition. You may want the code, but Google will not care (or it might dislike it).

Look at GPT-3+, OpenAI gets fame and fortune while people struggle to reproduce their last-gen models.

They asked Watson of course.

Agreed.

Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making.

Now imagine if you were a peasant from 1000 AD who was given a car or TV to examine. Could you really be confident you understood how it worked by just running experiments on it as a black box? If you give a non-programmer the linux kernel, will he/she think it’s magical?

Things look like magic especially when you can’t look under the hood. The story of the Mechanical Turk is one example of that.

The APIs were messed up early on, which is a reason TF2 happened. Every team started making their own random implementations of stuff. You had the TF Slim API, you had Keras, etc. The API just got fatter and fatter and then libraries would make cross dependencies to bake in the API mistakes.

Nah, TF has had dynamic execution since TF2 and it’s still losing users, it seems. The execution model and API is simply more complicated. What’s a session, placeholder, constant, tensor, …? PyTorch was sold as numpy with GPU support and it is pretty close to that. JAX is an attempt to approach language simplicity and purity.