thornewolf
Joined 4 years ago
Joined 4 years ago
A bad universal classifier does suggest a good one later. And that is exactly what I would call "theoretically doable"
That said, I don't think that Jev is a magic breakthrough or anything. I think it is just a particularly good narrative with an easy way to try it out.
Write a new prompt, evaluate, it now succeeds 99% of the time. Measure what changes between prompt #1 and prompt #2, understand what contributed to the performance jump.
Write a third prompt, this one succeeds 100% of the time. Increase the size of your evaluation set, find a 1/5000 error-class and a 1/10000 error-class, add some explicit code to correct for this cases.
Roll out to production, collecting usage metrics. You make some tweaks to your harness, your prompts. Eventually you have confidence that your system has fewer mistakes than 1 in 100k.
Now, multiply this iteration across all your different prompts and different ways that they might interact with one another.
v = setup()
if v == 1:
side_effect_1()
elif v > 1:
side_effect_1()
side_effect_2(v)
else:
raise Exception()
then we can "refactor" v = setup()
if v < 1:
raise Exception()
side_effect_1()
if v > 1:
side_effect_2(v)
i know that this might seem "dumb" that the code was ever setup the first way but code can grow into that shape pretty easily. this refactor "removes" the v==1 branch. this new code also follows the "early return" pattern, which improves readability.Google's last frontier model release was Gemini 3.1 Pro, which was in February of this year[1]. At the time, it was ahead of the (at the time) flagship models of Opus 4.6 and GPT 5.2/5.3. From my recollection of the time, it was the best model in the world.
Anthropic released Opus 4.5 Nov '25, 4.6 in Feb '26, 4.7 in April, 4.8 in (late) May. Then Fable in June. 4.7 beat 3.1 Pro on multiple metrics. Fable eats it for breakfast. However, I want to note the 3 month gap between those first two Opus versions.
OpenAI released 5.2 Dec '25, 5.3 Codex Feb '26, 5.3 Instant Mar, 5.4 Mar, 5.5 (late) May, 5.6 July. 5.4 beats 3.1 Pro on agentic benchmarks[2], seems to be similar/losing on non-agentic. 5.5 seems stronger than 3.1 Pro[3].
Gemini 3.5 Pro is alleged to be launching within the week. Why do I type this all out? Because I think Google is getting a bad rap. They are delayed on a frontier release by a month or two and are being regarded as if they cannot release frontier models. I think their last release demonstrates strength and we need to see a weak release before we call them "behind" (in any reasonable sense). These companies swap back and forth constantly. I recall a multi-month span where 2.5 Pro was just the best thing out there by a large margin (in my opinion).
[1]: https://blog.google/innovation-and-ai/models-and-research/ge...
--pdf-engine=typst
-V template=article.typ
Despite the input content being a .md file, we see that the .md contents populate the .typ file. Pandoc then understands how to convert the populated .typ file into a .pdf. The author also notes in their hyperlinked .typ document where the body content is placed: // THIS IS THE ACTUAL BODY:
What is not clear to me is the actual mechanism that tells pandoc where to place the body contents in the template. I presume it's some "magic" from the pandoc docs. The pandoc templates documentation[1] does reveal to us that `-V` is setting a variable called `template`. I don't have pandoc locally, but from another post[2] from the same author and a chatgpt query, it seems that the `body` symbol in the template is the chosen substitution symbol.[1]: https://pandoc.org/demo/example33/6-templates.html
[2]: https://imaginarytext.ca/posts/2024/pandoc-typst-tutorial/