aaquibahm

you can’t see any benefits to studying how LLMs behave in long horizon multi agent tasks?

i think it's unfair to assume the labs paying top dollar for cybersecurity engineers somehow have an incompetent bunch of nincompoops running their evals. it's a hard problem! to conduct effective evals, you sometimes need the model to have (restricted) access to the internet. at the same time, you have thousands (!!) of model instances with high cyber capabilities that will do whatever it takes to solve the answer. the infra surrounding model evals may soon (or already) genuinely be subjected to more hacking attempts from all sorts of weird angles than any other code on the planet.

cool! im @Aaquib111 on github if you would like to credit, but no pressure

why not mask attention and do it all in one forward pass ? tokens belonging to a question can just see that question and the main prompt