skp1995
Joined 4 years ago
Joined 4 years ago
yeah this one is a bit weird, you can run the linux binary using WSL and that should work. We have a window flavor build but its not as heavily tested yet (we are figuring out a better testing flow for windows)
We worked quite heavily on the TUI, its written in ratatui and we did try to go the extra mile to make sure that there won't be any regressions from moving to an alt-screen rendering. Lots and lots of small details to manage and get correct, we also tried to get vim keybindings correct while keeping it mouse friendly.
What kind of SDK support are you looking for?
The big difference is the complete loop, each PR gets its own VM with the tool chains installed so the agent can run cargo check or cargo tests etc.
We do find the LLMs of today are not the best elite engineers but very very competent junior engineers. It's been a weird but eye opening workflow to use.
ngl the total expenditure was around $10k, in terms of test-time compute we ran upto 20X agents on the same problem to first understand if the bitter lesson paradigm of "scale is the answer" really holds true.
The final submission which we did ran 5X agents and the decider was based on mean average score of the rewards, per problem the cost was around $20
We are going to push this scaling paradigm a bit more, my honest gut feeling is that swe-bench as a benchmark is prime for saturation real soon
1. These problem statements are in the training data for the LLMs
2. Brute-forcing the answer the way we are doing works and we just proved it, so someone is going to take a better stab at it real soon
I used to think Arc/Rc was a shortcut to avoiding the borrow checker shenanigans, but have evolved that thinking over time.
You do mention it in your comment so wondering if you have anything to share about it
I know my way around this now, which is to literally binary search over the timeline of my edits (commenting out code and then reintroducing it) to see what causes the compiler to trip over (there might be better ways to debug this, and I am all ears)
Most of the times this error is several layers deep in my application so even tho I want to ticket it up, not being able to create a minimal repo for anyone to iterate against feels like a bit of wasted energy on all sides, do let me know if I should change this way of thinking and I can promise myself to start being more proactive.
Having said this, the benefits of borrow checker out weight the shortcomings. I can feel myself writing better code in other languages (I tend to think about the layout and the mutability and lifetimes upfront more now)
My rust code now is very functional, which seems to work best with lifetimes.
I would love to know more about the authors pain, I do hope rustc gets better at lifetime compilation errors cause some of them can be very very gnarly.