nee1r

are you specifically focused on non-invasive methods/why? seems like you reach a noise barrier which limits things like this

how do you pick good goal conditioning images/do you have to hand pick a dataset of good goal images? seems hard if you don't have full context. really cool though!

thanks! a lot of credit to the people who helped write/edit

giving back to the research community! releasing and talking about research helps everyone

thanks! i definitely love diffusion + pushed for it, as a non-causal generative method i think its pretty unique

we have an alignment blog post dropping soon! scaling up in the next couple of months, then hopefully opening up an API or licensing it.

Benchmarks are really fun—lots of secret ones. Our main thesis is that you should be using the same benchmarks to measure human ability to use a computer, as you would an AI model. Definitely a suite of continuous long term planning tasks (games) and things such as marking emails as spam etc.

definitely! we are looking into more interp + visualizations in general as we scale up.

planning on instruct tuning soon!

safety was important for the demo, the model didn't have access to the brake or accelerator.

thanks! the math and architecture of the FDM (no video encoder) is pretty simple, its a regular transformer with next-token predictions but with frames interleaved.

yeah! i love the BCO paper, i think its extremely intuitive and these methods are really interesting in a time where data without labels is abundant. i especially like the idea of iteratively making the inverse dynamics better—might lean closer to that in the future

cool thanks for the title idea!! hopefully when we scale up in the next month/two we can update the community

collected! no synthetic

thanks! the inverse dynamics model is trained first on 40k hours of data and then frozen to label all 11 million hours. yup! the idea is that it should take a small amount of data to generalize environment dynamics, then you can use a lot of data to understand actions.

real