kavalg

Good questions. I guess we will never know unless someone with enough money and data tries the approach on a larger scale.

Their premises are:

- If you work with the raw bytes in a non-hierarchical (isotropic) model, such as a byte transformer, you get better transfer to downstream tasks, but at a huge computational cost.

- if you use a tokenizer (pretrained or regexp based), you gain a lot of computational efficiency, but loose semantics and the transfer to downstream tasks may degrade

- finally, with H-Nets you get the best of both worlds. The model learns how to chunk the data in a better way than a hardcoded/separately trained tokenizer

Authors also demonstrate that this approach works in other modalities, such as DNA sequences and images, where we don't normally have lots of ready to use tokenizers.

50/50 is bad enough

Highly recommended! Sebastian does a great job explaining complex AI topics in a very intuitive way.

While I have the same sentiment, I am also curious why a competitor can't accelerate their developments with AI. In the end, hardware development is to a large extent also software development. Is it the patents that NVIDIA has that are preventing this? Or maybe some trade secrets? AFAIK the NVIDIA hardware is manufactured entirely by TSMC.

These are two different things. For example, it is possible that the fridges connect to a central, more secure MQTT endpoint to report telemetry and receive commands. They don't have to be discoverable on the internet. Also, not setting up a VPN, given the fact these vulns were known for so long is pure negligence.

yes, but why do they need to be exposed to the internet?

You guys are so funny :)

The most subtle thing to me is who has stake in the game. It is true that humans make mistakes too, but they would generally strive not to damage money-generating software/hardware infrastructure. If we remove all they people that actually understand how the system is working (up to a level of abstraction), how will the ambitious PO (or whoever is driving the AI) be safe from ruining the whole thing or simply landing in a deadlock situation where one agent wants to make the change (e.g. dev agent) and the other agent is refusing (QA agent)? I guess we are still trying to find the right level of abstraction and only evolution will show where it is.

This quote shall really resonate with all of you who use Tinder similar dating apps: "Commercial apps need you to keep swiping but this app is designed to get you off the platform. Singapore built the first dating app whose success metric is deleting its own users".

Even for a non-linguist like myself, the article didn't feel right. I expected to find more information about the Thracians, who inhabited those lands for quite some time and left some extraordinary (albeit non-written) heritage.

Since when has intelligence had a strong relationship to status and respectability? I've met intelligent people that don't get much respect or status either because they don't look good, are shy or they don't have money.

True, but the relationship is certainly not linear and not a Markov process. Also, the $$$ a company makes are rarely directly translated to incentives for developers. I remember a case where a company was doing an OK engineering/product job, but had problems with sales. Then a new sales driven CEO came into place and for the next 3-5 years the company had great financial results, while at the same time eroding the engineering culture (who cares if money is flowing anyway). Then that guy left and the next CEOs had to take care of the mess.

How come? the github page says the license is MIT.

I kind of agree with you at this point. When ChatGPT was rapidly gaining popularity I thought that they will eventually replace search (esp. for shopping), which would have given them a huge ad revenue. Maybe they could have even tried social networking e.g., to help you sort out the huge flow of information that today's social networks are and get to the important/rewarding/whatever posts. But now ChatGPT is kind of getting commoditized. I would even dare say that gemini feels to me a bit better now, so the search route for ChatGPT is clearly gone.