CapsAdmin

What kind of mistakes?

I've used local models extensively (mostly qwen 3.8 27b) and now Claude sonnet/opus 5.5 since launch.

Sometimes I really wonder if not being able to see what's going on in thinking is a large contributor to thinking the models are stronger.

That assumes you bother skimming through thinking traces and the output in-between.

In my experience both qwen and claude fumbles and misunderstand, but will in a lot of cases self correct.

Qwen models can feel very indecisive but somehow makes it in the end. If the harness was able to hide that and only highlight the end results, I'd probably think qwen models are smarter.

Reading the tool calls Claude does while thinking shows a lot of trial and error about even simple things like editing code and getting the whitespace wrong. But it happens so fast compared to qwen that it's sort of acceptable. With qwen, it can feel frustrating because it spends so much time on it.

This is all assuming that's what you meant by mistakes, but maybe you meant something different.

In my experience, the science/math/code crowd don't care about copyright/plagiarism as much as the video/art/literature crowd, so the first crowd turns a blind eye to most of the latter crowd talks about.

Code can be art, and copyright/plagiarism is real. It sort of boils down to how much it bothers us.

My biggest clue is the odd repeating details. Especially on grass, tree leaves, pores, etc

Not all image models (or sampler settings) do this, but with gpt it's very obvious.

With this in mind, I misjudged 3 images on the default game.

This feels very condescending and reminds me of how theists "feel sorry" for atheists because they don't believe in god.

I would say opus was in some ways more accurate, but missed the higher level curvature feel of the site that astra picked up on.

It sounds completely trivial and likely I'm wrong here, but could it be that opus saw the reference image squished? That might explain the sharper horizontal curvature

I've been vary of using ai to search considering all the spam out there. I think I'd rather, perhaps naively, whitelist wikipedia, reddit, arxiv, some news sources, etc than include everything.

Is there nothing out there that does this? I'm paying for kagi and I can see that it has an api, is that maybe sufficient if configured properly?

Is this a normal thing now? I remember seeing this talked about as a surprising thing, but now I'm seeing posts about this as if it's normal.

(I switched to using local models as usage limits, api instability and the concept of paying per token stresses me out)

It comes with MTP already, they just don't mention it in the filename as it's now the default.

I've tracked it for a few years with whatever the latest samsung watch is, and it usually says it's good to excellent.

My total sleep time is about 6 hours. deep sleep is about 1 hour and 30 min. rem sleep is about 1 hour and 20 min.

I usually go to bed around 11pm and wake up at 6 am (give or take, I'm married)

It does however warn me about bad sleep if I've been drinking, have a fever, traveling across timezones, or staying up very late for some reason. One of these things may happen every 2 months or so.

I feel mentally good, I'm in shape and in general good health according to yearly checkups. I've been drinking coffee for about 18 years now.

Whenever I stop drinking coffee for a few days or even months, I find that it becomes harder to wake up. I seem to sleep a lot heavier, but in the morning it's difficult to open my eyes as they feel heavy.

It doesn't matter if I have 1 cup in the morning or drink coffee all day. As long as I have it in my system from the previous day, it's very easy to wake up the next day.

Something that bugs me about these new demos is that they benchmark the model and harness at the same time. The harness steer the model to expand the vague prompt, self correct with tools, etc.

I think it's useful when evaluating "model+intended harness", but I'm more interested in seeing raw model improvements than harness improvements.

That was pretty much my takeaway from reading the website. Not my cup of tea, but I guess it might valuable to some?

I've had to use the aws cli at work a few times (not an expert), but I could see aws trying to compete here if this becomes a thing.

I think nothing can be done anymore, but I don't buy that security through obscurity never worked in practice. A better way to look at this is effort rather than obscurity. The effort it takes to do something with AI is going down, not up.

Almost everything was possible given you put in the effort, but few people possess the will to put in the effort AND pursue a malicious goal.

I think an obvious example is all the fake ai content flooding the internet made to trick people in exchange for money (ad revenue, scams, likes, etc). This existed before ai, but I think it's fair to say pumping out content now requires less effort than it did before.

Most physical locks are an example of security through obscurity/effort. You can after all just pick a lock if you go through the effort to learn the skill. But once a universal lock picker is made available to everyone, you will simply see more locks getting picked.