springtimesun

2 turns requesting information and indicating a preference for web search vs answering from memory and it fell over. Thinking block was a mix of Chinese and English, which is fine, but both questions were in English. Second turn answered in Chinese. When told “I don’t speak Chinese” it started to answer the first turn again, then crashed.

I find the benchmark results simply unfathomable after watching that performance.

Is the increase in commits graph in percentages because it’s fewer than Anthropic’s rate? A case of commit envy?

In all seriousness though I have enjoyed reading these posts. I will never work at this scale, but I like to read how different teams approach problems like this and I do wonder if some of these approaches will trickle down (or up?) into regular practice as everyone is committing more and faster.

I don’t know how many are still here, but do people who launched the project ever come back and get to give them a command now and then?

<thinking>

The voice recordings and non-linearity were what made it for me. I felt like I was piecing the world together as I explored. Even after doing the boss fight finding intel that explained something that wasn’t known before slotted it into my understanding of the story. By the time I reached the grind at the end, it was like I had a world in my head and it was ok to close the book.

I loved The Division. Some months ago I re-downloaded it and just ran around for a bit. The setting and world feel are still there and the story is deeper than some single player games. For a while I thought, wouldn't it be amazing if it was ever released (or leaked) so it could be player hosted or modded in a local kind of development like this. But, I didn't stay with that thought. There was just something about that game that had a beginning, middle and end for me. Maybe I'm just getting old.

I feel like I’m an outlier based on all these comments. This is still working through my personal benchmarks, but it’s spending 30% more time and performing worse than the previous 4.1 flash did.

Benchmark is historical PR reviews. Models are put in a sandbox with a git and db snapshot as it existed at the time. PRs are a selection of changes some bugs (major and minor), some with things that aren’t bugs that have tripped up models in the past and some no findings that they should approve and move along.

4.1 flash already wasn’t that strong at this task, but the speed edged out v4 pro for a spot in the roster.

I made a comment some time back that reviewing AI code was kind of bewildering (really any other code but my own). I’ve since dug in, crafted some processes and skills (in the /skill sense) and this is basically the shape it takes. Find the hot spots, what’s setting precedents, what’s not sitting right with me… it’s fast but it actually creates additional work (in a good way) because the way I have the skill set up the agent then scans through the whole codebase to find places where the changes demanded on a given PR also exist. Each PR spins out 1-multiple tickets. In a way, this has always been my dream. I believe in refactoring at some core level. Now I finally have the time (and tokens) to do it continuously.

LLMs have also pushed me to learn a lot, but I don’t find the shape to be terribly different than before. I rarely remember the exact sequence of commands or flags or syntax. What my learning was useful for before and after LLMs is to 1) build an intuition of how something behaves and what it should do and 2) expose me to non-obvious possibilities that I wouldn’t have come to on my own. With those two things in place, the next execution without the LLM is a docs search, same as it always was (for me maybe some people are better at remembering this stuff).

What’s different after LLMs for me is that there are some things that just don’t care about and I’m fine if they are a black box that the agent handles for me. I didn’t have that luxury before.

My take on picks and shovels was always a more cynical one. The beauty of the metaphor comes from the fact that after the miners went bust and left their picks and shovels where they lay, they could be collected and sold again to the next dreamer.

Claude has gotten pretty sensitive to copyright. In addition to the no song lyrics that was posted in another article, I also find it will not generally transcribe podcasts (you have to play with model versions and even effort inside that).

The “for agents” button answers a question I have had for some time: what happens when the frameworks I use that are well represented in the training data go stale? For the work I do, agents hit APIs nearly flawlessly from memory, but if the new releases from open weights well dries up, that won’t always be the case as the models age. I like this approach to building skills rather than relying on memory.

Man you people are hard ones. OP had an exchange (as a local) with someone who was impactful in their life. Not to slight Antirez, but I don’t think people are ruining his walks daily to talk about in memory caching. If it was really 5 minutes and a photo (he presumably agreed to) I feel like you all are blowing this way out of proportion.

I remember the first time I ever met someone IRL from a big OSS project. I was a little star struck. Of course he didn’t remember merging my code, but I didn’t expect him to. It’s an experience I will always remember.

I encourage anyone who has doubt about this question to try an experiment. Find a problem you are about to use an agent for or save one. Then ask a non dev to prompt the AI agent of your choice to try to fix it. You can even find a technical user or someone who uses AI regularly for non coding work.

I have done this a few times now and it is very apparent what experience, and skill, bring to the table.