solomatov

As far as I remember there was report of their run rate. How does it align with this?

>you could choose not to jump on the bandwagon

I think it's not an option. The benefits are just too large for me.

>I do know we will never go back to mainly programming through code again, that’s for sure.

I think, in some niches, i.e. where there's something not well represented in the training set, it still makes sense to write code by hand. But I am not sure that it will continue.

How many toks/s could you get this way?

Which remarks? Could you share a link?

It looks like HuggingFace shows Apache-2.0 but they have AUP. How does it work together?

Thanks for sharing.

Super impressive! Is it possible for you to share your methodology of using LLMs?

I tried a similar approach before, but it didn't work for me. I didn't get a lot of speedup if any from it. IMO, to get productivity you need some kind of YOLO mode (in a sandbox).

IMO, the goal should be to outsource as much work to the model, as possible, while minimizing effort required to understand and review what is did. For example: ask the model to find out why a bug happens, figure out proof of concept for thing X, incrementally optimize something, do a well specified refactoring with some guide, and similar things.

IMO, what people say about creating loops is a very similar thing. You maximize the work done by the model, while minimizing the amount you need to do to control it.

Did they mention the license?

> that he doesn't really understand how inference works from a technical perspective

Could you share what tells about it? I.e. where he was wrong about it?