airspresso

Noticed this as well! Was looking for details on the software support and this FAQ entry didn't exactly instill confidence...

> it's extremely long in it's thinking, it just goes on and on

Known issue with this model, I recommend setting thinking to 'medium' instead of default 'xhigh'.

> I looked into running a local model, and it's way outside the budget of an individual or even a small or medium business (hundreds of thousands of dollars).

That is a big exaggeration. You can have a perfectly usable local LLM setup that will power your agent for single digit thousands of dollars. Can even power multiple agents simultaneously, depending on the hardware and setup. Won't be fast and won't be frontier intelligence, but definitely useful.

> What if all the agents involved in these incidents have in fact had the full stack of alignment applied

A big part of this developing story is that it happened during training of a new model that ended up misaligned. And training happened without the usual safeguards applied like chain-of-thought monitoring. So OpenAI has already admitted that the full stack of aligment had certainly not been applied in this case.

We are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.

No, those use LPDDR5(x), not HBM.

From what I'm seeing elsewhere, context size up to 128k should be possible on this hardware. It really matters for agentic workloads to push that context size headroom up. Anthropic are spoiling us with models that do 500k context and beyond.

I'm with you on this, the knowledge sharing on YT is massive and in general not replicated in written form anywhere else. Anything that needs fixing involves a YT search for me these days, since that has the highest chance of turning up something useful and relevant. Only downside is that it's time consuming to find the right spot in the video and pause/rewind to understand how to do the thing properly. But having someone actually demonstrate how to do it, rather than just describing it with words, is so helpful and makes replicating it a lot easier.

Clicked the links just to see preview screenshots, that is hilarious with all the cows XD

It gave the card an aura of mystique (pun intended) and I remember really wanting it. I guess that means they succeeded in making a statement with it.

Nvidia is also pushing for local inference IMHO. They want open models and competition in the model layer, not two big labs controlling all of it.

This depends heavily on what the use-case is. Yes, if it's a coder making software and having to read LLM output then writing style matters. If the LLM is used in an automated data processing pipeline with a capped level of complexity, entirely different aspects matter and LLMs become more interchangeable.

By leveraging the experience Broadcom has in this area. Still remains to be seen how that goes when they want to scale production.

Most of the generative art is hosted on IPFS.