GodelNumbering

I spam Yare Yare to the extent that my friends had it on my birthday cake

This is such a good use of tech! Although I have heard in the past that youtube is quick to ban any use of its content on a third party site. I hope I am wrong and this stays.

"It is possible to commit no mistakes and still lose. That is not a weakness, that is life." - Jean-Luc Picard

Good startup advice. Good life advice.

As of their latest update (AA's), I find their index hard to trust, and a bit shady. 60% of the benchmarks contributing to the AA index are now private, nobody can replicate them. And they explicitly say that they offer benchmark consulting to labs. This is a serious conflict of interest.

The latest update knocked down only openweights models from the leaderboard.

Argon(18), Carbon(6), they have Helium next and some quark after that.

> I'm all ears. Our intention to make things clear and simple, even if we miss the mark sometimes.

Thanks. I think the very response you wrote here is a better form of marketing for the software engineers than what openAI puts out there. When everything is groundbreaking and everything tops charts, nothing is.

I suspect a lot of target demographic (developers) prefer simpler and honest communication about not just the models high points but low points too. With the linked Terra post, if I saw 1 chart that clearly highlighted the usecases for which Terra is _worse_ than higher tier models, I would be more convinced overall.

Everything trying to occupy the top spot in my brain's limited "top model" representation space is not helpful.

It sounds like it is trained on a lot of podcasts, judging by a quick few second skims over their demo tracks.

Also, the trend of posting on HN with much more sensationalist title first, getting to the front page, then getting the title adjusted by a moderator should probably be penalized; how is 'most authentic ever' not a marketing term? (not just this post, but a bunch of posts recently got to frontpage by making unreasonable claims)

Yes, reading more files than necessary. Was curious about how their model handles a series of edits over longer tasks, but then tested it using openrouter myself, it worked quite well.

No novel technique can avoid this, but certain techniques can make it worse.

This looks noticeably more capable than previous gen. I purposely gave it a bit of a vague prompt to create an html renderer for UML diagrams, it produced impressive work. Then i asked a question, and in the third prompt I asked it to update the html which it did correctly.

Another thing that stood out, and this might just be a novelty thing, is that it picked different and less painful-to-eyes color schemes for each theme. Congrats to the team behind it.

Question (since their team has reps here and I have never used Inception via api): when you ask for a code change in your codebase (via coding agent), how much code does it need to read? How well does it handle tool calls? Since diffusion is a different way of doing things, I am curious about the code-needed for each incremental change in the codebase, say over a larger task.

Might want to reach out to the author of the above benchmark. I think having a working group of multiple distinct benchmark owners might help quickly evaluate new model developments

'Thrust' version takes $245,000 of downpayment (not the flying version that is 'invite only'). That money is neither escrowed nor earns an interest.

With high interest rate and tight money supply, this could get them a few billion dollars of 'customer financing'.

Unsure, or as LLMs say, I’d gently push back on that. In my opinion, the jaggedness has increased. While this type of capability has improved, I still see longer running Astra agents making the assumptions that any employable person at any level wouldn't make. Also, generally, all models feel more robotic and less pleasant to talk to (subjective) while still lacking judgement.