nessex

(Old) English had æ first!

Many of the Nemotron datasets are gated behind approval, and a license agreement.

The preamble on these datasets is: "This repository is publicly accessible, but you have to accept the conditions to access its files and contents."

I don't know what others have experienced, but I requested access to multiple Nemotron datasets and those requests were ignored for months before all but one request was rejected. There's no explanation for why, nor anything I can see which would lead to a rejection. So it's purely anecdotal and YMMV, but I don't see these as being particularly open.

Paying the list price Anthropic decided upon for outputs from an LLM, then using those outputs for your work? Or sourcing those outputs from others that paid for the service, and using those? If that's "distillation", it seems fine to me. And a lot like what most are doing with these same providers.

Anthropic: training AI is "transformative", it's not copyright infringement if we don't re-transmit the copyrighted books we trained on

Anthropic: training AI models on our outputs is stealing our secret sauce, outputs that could only be produced by us

Anthropic: AI model outputs are unreliable and do not reflect Anthropic's views, we are not liable if they harm you

(not exact quotes, they're "distilled")

So Anthropic "distills" knowledge of others with reckless abandon, packages it up, sells it to you, claims it's your fault if anything bad happens but then also lobbies to treat you as a criminal if the outputs you paid for end up being transformed into any sort of competition for them. By you, or others that use the outputs you paid for.

Forgot to mention, but it was after the 4.7 release when I was still using 4.6 that I saw those loops too... Before that, 4.6 had been a pretty seamless experience.

They would quantize the model. That'd make it cheaper to run, and have slightly worse output but it would still generate outputs with a similar feel, derived from a compressed version of the same knowledge base etc.

They wouldn't even need to do this uniformly, quantized versions of the model could be routed only a subset of the requests. They could do this to nerf the old model, or more likely just to give themselves more hardware to run the new one on by handling more requests on less hardware. Or to handle increased request volume as traffic ramps up faster than hardware can be provisioned.

Playing with local models at various quants, the degradation can be hard to spot. Sometimes it's only noticeable in aggregate. And even then, you never really know if you just got unlucky with a bad response due to RNG.

I've had Opus 4.6 fall into some weirdly incoherent loops that I rarely see from even Sonnet, that felt like the kind of thing I got frequently with Qwen3.5 9B on local. And the above applies... Was that just bad RNG? Or was my request to Opus routed to some lower quality variant? There's no great way for me to tell for any given request, nor any way to guarantee Anthropic _didn't_ do that.

This is a great post that covers a lot of the recent ground. I have a very similar setup after a very similar journey, minus the RTX6000. Worth noting though that a lot of the recent changes make a single 3090/4090 much more viable here too. MTP and the recent improvements to kv quantization in particular, as well as model-specific template & quant fixes. I run a 4090 with the 4-bit quantized variant of the same model now and have had a great experience. Qwen3.5 was already a big step up, but with 3.6 and the rest of the improvements it's substantially more reliable as a daily use tool and I find myself reaching for hosted models a lot less. Feels like I could work entirely without them if they were to disappear without going back to typing every line of code myself.

To make 4-bit fit on one card with reasonable (100k+) context needs a bit more care though. And tuning can be highly specific to your machine, gpu and use-case. But I use a headless server, offload multi-modal to CPU, use fit-target to reduce wasted memory and use q8_0 kv since the 4090 performs well with it... In addition to most of the same config as the author elsewhere. I get 50-60tps generation with a power limit of 275W (450W is default), more than enough to offer a roughly an Opus-speed feedback loop.

I haven't seen many of the issues with looping the author mentions. But I did with Qwen3.5 and in particular other 4-bit quants in the past. But the difference is probably a mix of the improvements above, as well as habits changing to avoid cases where models will loop. For what I'm doing, it seems like I loop Qwen3.6 on the same kind of prompts I'll make Haiku or Sonnet loop on (the latter hide some of their existential loops behind "thinking"). Usually it's cause I was too vague about some aspect of what I'm wanting them to do or I forgot to include some context that smaller models just don't have access to in their smaller knowledge base. But at least for what I'm doing (Rust, React, kubernetes) it's not been a notable problem at all with the latest iteration of this whole stack. And knowledge of standard libraries and default k8s resource kinds has been almost flawless.

There's still plenty of more complex stuff where I'll choose to jump straight to Claude or GLM-5.2, but if it's not worth that jump I've stopped paying for the middle ground as it's usually not much better than just one more iteration through qwen.

All this to say, if you have a 3090/4090, feel free to give the same setup a go. It's come a long way in recent weeks.

I've got a similar setup with a k3s homelab and a bunch of small projects that need basic data storage and caching. One thing worth considering is that if someone wants to run both redis and postgres, they need to allocate enough memory for both including enough overhead that they don't suddenly OOM.

In that sense, seeing if the latency impact of postgres is tolerable is pretty reasonable. You may be able to get away with postgres putting things on disk (yes, redis can too), and only paying the overhead cost of allocating sufficient excess RAM to one pod rather than two.

But if making tradeoffs like that, for a low-traffic service in a small homelab, I do wonder if you even need a remote cache. It's always worth considering if you can just have the web server keep it's own cache in-memory or even on-disk. If using go like in the article, you'd likely only need a map and a mutex. That'd be an order of magnitude faster, and be even less to manage... Of course it's not persistent, but then neither was Redis (excl. across web server restarts).

Sure, but if you insist it's like a tide you downplay the risk of the initial hit of the wavefronts and the potential for it to slam up the coast or a seawall becoming a larger local wave. And if you insist it's like a wave, you downplay the persistent risk of both follow-up waves and ongoing flooding that won't subside quickly.

So saying it's not waves is dangerous, and saying it's not a sea level rise is dangerous. It's not useful to try and delineate between a tsunami being one of the two when it's in reality an event that consists of both.

(Ignoring that a sea level rise and a long-wavelength wave are the same thing)

It's a distinction without value I think. There are waves, and many of them. There is a rise in the sea level. For anywhere affected, both certainly matter. Like you mentioned, tsunami isn't a brief event. And here in Japan, they are talking about tsunami waves, not a singular tsunami. And talking about sea level rise and checking the local power poles for sea level indicators from previous tsunami events and floods.

There are planes, buoys and other things being mentioned on the news here in Japan as ways things are being tracked. Maybe not what you meant, but tracking the wave isn't necessarily correct. There are many waves, and the initial wave is often (in this case also) not the largest.

The news mentioned a previous similar event where the largest wave was 4 hours later.

Not true. As the news reporters here in Japan are repeating every few minutes, there will be many waves and they can get bigger over time. They already have, 20-30cm initial waves had 40-60cm later waves.

Waves can get bigger due to earthquakes not being instantaneous or necessarily a single movement, due to amplification by geography, by reflections, by aftershocks, and many other things. The news is suggesting waves lasted about a day for a previous event in a similar area.

If you haven't tried OpenAI's advanced voice mode, it's a mind blowing version of exactly what things like Siri really ought to become with a little more development. If that's what you mean by LLM Siri, I totally agree.

Being able to chat casually with low latency, correct yourself, switch languages mid-sentence, incorporate context throughout a back-and-forth conversation etc. turns talking to these kinds of systems from a painful chore into something that can actually add value.

> Because there are so few of them. Most of the time you can walk in the middle of the street, so rare is the traffic

This is a bit of a stretch. There are cars everywhere you go in Tokyo, it's pretty well set up for driving given the size and population. That said, I've lived here without a license for years, and rarely had the need to hop into any cars. Speed limits are generally low, and lights are everywhere, which often makes the train or sometimes even a bike a faster option than a car or a bus.

It's only when you have multiple connections in your route or when you get well outside the city that you start to see consistent benefit from a car.

One reason it's so easy to get by without a car in most of Tokyo is that the shops and attractions are distributed well. Zoning means there are tiny shops everywhere, and the bigger shops are present at many of the train stations in the city. You are pretty much always within walking distance of everything essential. One more reason is that the postal system is excellent. If you order something from Amazon, you'll receive it about 24 hours later in most parts of Tokyo. Who needs a car when you pick up everything essential on a 10 minute walk from your house, and everything you want will be at your door tomorrow?

I ride in Tokyo and surrounding areas often. There are bike lanes scattered around, though some places have far more than others. In general though, there is a lot of patience on the part of drivers here in the city (at least relative to Australia) that makes riding a bike on roads without a bike lane generally a fairly safe option.

Also, there are plenty of terraces to have a nice coffee on. The difficult thing in Tokyo is you need to search in advance for what you want. It's so crammed full of options, that you might never realize all the interesting things you're walking past. It's a very three-dimensional city, there's stuff hidden everywhere.