kyxsc
Joined 3 months ago
Joined 3 months ago
That being said, perhaps this Parseable project is much more efficient (RAM) in storing that 1b samples compared to Mimir (RAM). Once we add in object stores which cheapen the storage cost by orders of magnitude (by going from memory to disk), the comparison is even weaker. Mimir and Cortex/Thanos are quite happy to pull cold data from S3.
So I too expected to see “billions” as well.. hmm.
Works really well and is a great use case for work laptops. Same shell, same memories, same sessions
Remove the one thing that differentiates these 2 SOTA models (long-horizon task scaling) and you're left with assessing the raw intelligence of both models.
Both models are really fucking smart. We should be intentional when discussing effort levels when it comes so SOTA models because currently, effort levels are the essential lever to evaluate task scaling.
I've never seen that leaderboard link but I think my sentiments reflect exactly the findings: Sol 5.6 is smart, Fable is smart, but Sol is more value for the end user (even more so when you lower effort levels because that doesn't degrade the model's raw intelligence/knowledge). Not to mention Codex resets, that's just the cherry on top!
But smart != capable and this is evident once you start assessing both models on long-horizon tasks with higher effort levels. While Fable is (imo) at least a little bit better, both are still very good. If you want to test the raw intelligence of said models, you should lower the effort level.. if you want to test the model's capabilities fully, you should increase the effort level.
Economics aside, Fable is the better model (imo), but there's no need for a binary stance here. Both models are very good yet there is a clear winner on the value front.
Given the 50% discount on Sol and how smart it is, yeah it's unprecedented value. If you only want to use low effort, there's a clear winner here on value and it's not even close!
For small tasks, you can just use something like low or medium effort and it can usually avoid mistakes; after all, the model will test the code anyways and can do some baseline level of iterating.
In regards to cost, we need to acknowledge how generous OpenAI was in the last couple months with Codex usage credits (no weekly limits) and usage resets. It afforded me many a dive with Codex! Yes it uses more tokens, but sometimes it's worth it -- just depends on what you're working on.
Finally, Ultra(code) isn't that bad when it comes to cached tokens. I think folks overstate the general token usage of ultra effort on both providers.
---
Both models are great at green-fielding a project when given detailed specs.
Both models overthink too liberally (imo) during these larger multi-shots. Sol overthinks more than Fable.
Both models are really smart and perform great for general knowledge and regular coding tasks.
After extensively using both on Max 20x plans, I've concluded that Fable is better for problem solving and coding, whereas Sol 5.6 Ultra shines in debugging specific issues: tackle a problem with Fable then leverage Sol to clean up, double check, or fix specific issues.
Fable (imo) had the edge on the $200 plan, but after this 50% reduction I'd say Codex is better value by far and there's no contest.
---
Using Fable as the orchestrator and delegating tasks to Sol 5.6 Ultra via the codex plugin in Claude Code yielded good results, but still there was a lot more over-engineering (thus time and tokens spent) than Fable by itself would've done.
Both models suffer from doing-too-much. But both models are fundamentally really smart and knowledgeable. I think it's really close and pricing cuts really spice things up for us consumers! Sol is a clear winner in the value department and the $100 plan is enticing!
---
*Claude Code usage is reducing by 33% in 2 days, Wednesday August 19... cmon anthropic: clau.de/cc-50-promo
(fable/sol are ~60 t/s, and OpenAI just announced their Cerebras partnership(?) for "ultrafast" mode of 750 t/s)
models aren't able to run that fast right now on our consumer/prosumer hardware. M5 Max for example has a memory bandwidth of 600 GB/s. a 5090 has 3x that, so running the same model on a 5090 is that much faster (provided the model is within 30GB).
running a bigger model on an M5 Ultra is still much slower than running it on a Blackwell chip with sufficient vram, CUDA being a major difference. if apple can bridge this gap, interesting things will happen... and just imagine if M7 Ultra has comparable speeds to Blackwell (or even Rubin)!
I use CachyOS but don’t really play many games on Linux. I’ve enabled secure boot so I can seamlessly swap to Windows for certain kernel-level anti-cheat games like Valorant. Using a shell alias/function (and menu bar buttons), I can boot into Windows (yay Limine one-shots!) in one command/click and next reboot, while in windows, gets me back into Linux. I really like this setup and believe it’s the most seamless you can get, besides a Windows VM with GPU handoff in Linux, which when I tried a couple years ago, was able to work with Apex.