cmenge
Joined 11 years ago
Joined 11 years ago
A) searching using BM25 and via vector embeddings are two different methods of search; both have their uses, many practical use cases benefit from having both. The search method is independent of who calls it when, i.e. whether you just have a static pipeline, or a dynamic one where the search tool calls them.
B) you're describing the static pipeline as if it were SOTA. We dropped that well over a year ago IIRC. There might still be cases where that is good enough, but in general, agentic search has pretty much become the default since LLMs became reasonably good at tool calling.
By the end of the first sentence I'm pretty sure it would take me quite some time to understand what it's about (the first sentence of the abstract, not the whole paper)
In my view, from the testing I did since Thursday, it's better than Fable. I had just finished a rather large task that Fable completed, including a /review and an /ultrareview.
Ox Alpha found bugs that Fable and Opus missed, and it continued to build things like a pro.
It does have issues with availability - but it's on a free promo right now. That also means that I don't know how much it would have cost if I had to pay API prices for it, which might not be cheaper than the subsidized small company / consumer usage, but for large companies paying API prices for either offering, the difference will be considerable.
Happy to chat and learn what you're up to: [email protected]
For all other roles the first few messages are similar: this is <role X> with key challenges a,b,c. You seem to be a good fit because of d,e,f. Would you be open to explore this? This requires relocating to <place>.
Traditionally, this was done by entry-level people. In either case, this isn't the person who will jump on a call with you.
"Desperate" and "clueless" seem very strong words here in reference to someone who gets actively approached from a recruiter.
I just spoke to a fried who is a headhunter and who's been trying to automate his processes for a while (he likes to fiddle and certainly has skills, but he's not an engineer). He kept trying, but it just wasn't good enough.
Now he said with GPT Work and Sol, it worked, but the key point is: all of it suddenly worked.
The problem was one of reliability, of handling edge cases. All previous attempts / model-harness-combinations were too brittle and needed too much observation and fiddling - cheaper to do it yourself.
Now he says "I don't know why I would ever hire a recruiter [the folks doing the cold outreach] again. I can focus on the candidate screening and acquiring projects, everything else is fully automated".
This doesn't come from an engineer or an AI lab, but a technically inclined power user, and I think this is where things get interesting.
Maybe the prompt was particularly well-suited for the model (I instructed it to put on a mathematician's hat, look at the mathematical substructure of the problem, identify invariants and general laws and verify them, then plan how to remediate).
It wrote a ca. 800 line in-depth analysis (at times spawning over 130 research agents...) with remediation plans, prioritized them and then implemented them. One issue was that this document was frankly over my head. Both the language it used and the mathematical parts were very terse, and in parts it felt like a post-C2-vocab exercise. The prose was much harder to understand than the code snippets / data models. As a non-native speaker, it lost me on the prose part, and had to ask it for a less elaborate version to actually understand it.
It burned the session limit four times, but it turned a huge mess of proof-of-concepts with patchy glueing into a coherent, stable application.
I'm also on the Max plan using Claude Code, and I have the feeling that the harness is much more important than the consensus expectation.
The deterioration was real and annoying, and shines a light on the problematic lack of transparency of what exactly is going on behind the scenes and the somewhat arbitrary token-cost based billing - too many factors at play, if you wanted to trace that as a user you can just do the work yourself instead.
The fact that waiting for a long time before resuming a convo incurs additional cost and lag seemed clear to me from having worked with LLM APIs directly, but it might be important to make this more obvious in the TUI.
In short, he proved that even AI agents exhibit all the dysfunctions one would normally attribute to human shortcomings / politics / laziness etc.
Either way, I think the point is strong: if the organization is bad, you end up doing mostly work about work which is exhausting.
Small, effective teams with super high accountability are more fun, but don't look "reproducible" or "repeatable".
Shameless plug on my take: https://www.menge.io/blog/where-to-cut/
Using Claude Code Max 20 so ROI would be maybe 2+ years.
CC gives me unlimited coding in 4-6 windows in parallel. Unsure if any model would beat (or even match) that, both in terms in quality and speed.
I wouldn't gamble on that now. With a subscription, I can change any time. With the machine, you risk that this great insane model comes out but you need 138GB and then you'll pay for both.
Also added a small side-project, https://www.revuo.ai for software-reviews and feature-tracking. This is only the start, obviously there are enough directories but I'm trying to dig deeper into the features. This one just started and is basically invisible as of now. Well, you gotta start somewhere I guess :)