rhdunn

It's interesting how all the recent SOTA models are in the 1T-3T range. This even has a large number of active parameters (A52B). It is also likely that OpenAI, Anthropic, et. al. have similar sized models.

This has two effects: 1) it prevents effective use of these models on consumer-grade hardware; and 2) it pushes the requirements to train and run these models on existing and future data centres, making them more expensive to run.

It would be nice to have more slimmer models and more active research that isn't just "let's throw more compute at the problem". Maybe they'll train smaller models from the large model like Qwen have done in the past.

It's easier to call someone a bot or other names than to engage with them or to try and understand their point of view.

We're increasingly pushed into different groups and sides with little to no toleration of each other. It's been progressing more over the past 10-15 years in the UK, US, and possibly elsewhere.

I think it's a combination of both...

Before AI people would still try asking questions on SO despite the hostility because it was the only/main place to get answers. Over this time, SO popularity was deteriorating.

After AI people who had bad experiences with asking questions on SO would ask the AI instead. This is because the AI wouldn't mark your question as a duplicate, wouldn't question its validity [1], etc. It would just answer your question. You could even ask it follow-up questions if the answer wasn't quite what you were looking for. All of this in real time, not waiting to get an answer.

[1] Unless the question triggered the safety gates that the AI companies have in place.

And with light/dark mode support and CSS variables the colours in a print mode should be easy to define and support.

So you're primarily left with hiding content that shouldn't be printed (like navigation or menu buttons), defining page breaks, and some other quality of life changes for printing.

Printing also covers things like exporting to PDF, or rendering in an eBook format (EPUB, Kindle, etc.) for reading the content offline while travelling, etc.

There are three return states that a parser (or partial parser) can return:

1. an item (`T`) for a successful parse;

2. an error (`ParseError`) for an unrecoverable parse error;

3. a partial item (`T with ParseError`) for a recoverable parse error or warning (known statically).

You may also want to differentiate between:

1. leaf nodes (numbers, identifiers, string literals, etc.);

2. composite/nested nodes (addition, multiplication, variable assignment, etc.).

The A3B part of that model means that each "expert" in the mixture of experts (MOE) model use 3B parameters. At 4-bit quant, you can fit 2 parameters per byte. That gives you 1.5GB per expert (plus a bit for overhead), which makes it usable on an 8GB card.

It's all about trade-offs: how much of the model you want to load into VRAM; how much VRAM you want to reserve for context; how much quantization you/the model can tolerate; how many tokens/second you can tolerate; whether you want to use the VRAM for anything else (graphics, Text-to-Speech, etc.); etc.

Stonking is British slang for large/huge/etc. For example, "Labour won by a stonking majority."