time0ut

I wonder if I can hook failed git push into betting on github's status on polymarket.

I have heard of small towns that generated income by running speed traps. There was a notorious one near where I am from where the speed limit changed like a dozen times seemingly randomly over the course of a couple miles. Maybe someday we will see some enterprising sheriff set up some trap for Tesla robotaxis and fund their local government by fining Elon. They can beta test it by sticking a cop at the nearest school right now!

Well it is a Wednesday.

I noticed because SSO was down. They don't even report on that as far as I can tell.

HN works great on Delta flights. What more do you need?

The early days of stack overflow were amazing, but 2013 was a long time ago. A few bad experiences when trying to help people was enough of a turn off to stop me from logging a decade ago. I do agree that we lost something and LLMs are a poor replacement.

I have been running a series of experiments on Jev since its release targeted at understanding it, seeing how it handles real use cases I have, and maybe figure out what it is inside.

Some of my tests do point towards what this post says. I was not successful in getting it's score to align with an existing rubric I had. It 'worked' but it was off and compressed from where I wanted it to be. Not a bad starting point, but I couldn't get it to move to where I intended the rubric to be. It wasn't the most robust test and I didn't spend a lot of time trying, but it wasn't just instantly magical.

However, it does seem genuinely useful just by being fast and cheap and good enough, so I am still a bit hyped.

The sales pressure from them on their agent core stuff has been really shocking over the last six months. Never seen anything like it.

Yes! I really hope they release some papers on their techniques. I am very curious.

I ran it through MMLU a few days ago and it scored ~90% so seems to have a lot of general world knowledge trained in. Makes me think your speculation is right. I have some credits left, might try and think of an experiment. I saw a gist where someone was asking it which model it was and it was picking qwen a lot, but who knows...

Anyway, thank you for the interesting discussion!

Yes, agreed. I was speaking in general, of course. This particular topic is of interest to me, so thinking of the edge cases and confounds vs Jev.

In your example, I would expect an LLM to do fine and if you have access to the raw logits you can measure whether or not it was confused and assign a confidence to the answer it gave.

I do think that Jev handles more than this though and, in my early testing, does things that are not easily accomplished with guided decoding techniques.

Grammars do risk pushing models off distribution in a way that impacts their output quality in a way Jev allegedly does not suffer from. Additionally, Jev's ability to answer questions independently is also exciting. Using an LLM to answer multiple questions in one generation has the property of earlier answers influencing later ones. TBD how many of TypeSafe's claims stand up, but my testing so far is promising. I hope they author some papers on their methods as well, but that might destroy their moat.

I am a total outsider when it comes to this topic, but it reads like the author is doing the thing they are accusing the Bend 2 guy of. Good juicy reading while I sip my coffee!

If I had to pick just one, these days it’s Composer. It is just a cheap, fast, surgical workhorse. My workflow uses other models as well for various phases: Opus for planning, Sonnet for review, etc.

I prefer Cursor at this point just because of Composer. Claude Code is passable but the lack of a good, fast, cheap workhorse sucks. Sonnet and Haiku aren’t it.

I also do like Codex and Sol, Terra, and Luna. They are decent but I don’t find they stand out enough to use over the others.

Additionally, I have not tried Astra and found Fable to really not worth the cost for the tasks I do.

Finally, the latest Grok is actually a beast of a model, but expensive enough to not be a stand out.