a1j9o94
Joined 12 years ago
Joined 12 years ago
If you productize that it will be an experience a lot of people like.
And on the UI piece, I think most people will just interact through text and voice interfaces. Wherever they already spend time like sms, what's app, etc.
I spent the last few weeks exploring whether AI systems could benefit from generating video predictions before making decisions—like how humans mentally simulate "what happens if I pour this coffee?" before acting.
The idea: Show an AI an image, ask "what happens if I push this?", have it generate a video prediction, then compare that prediction to reality. If the prediction looks wrong, maybe the AI could catch its own mistakes.
The result: Current models can't do this. But I learned some interesting things along the way.
What I tested:
- 7 different architectures for predicting future video frames from VLM latent space
- Whether perceptual similarity (LPIPS) between predicted and actual video correlates with correctness
- Self-correction loops where the model gets feedback on its predictions
Key findings:
1. VLMs can't predict the future – Every architecture I tried performed worse than just copying the current frame as the "prediction." The model understands what's in an image but can't predict what will change.
2. Visual similarity ≠ semantic correctness – This one surprised me. Wrong predictions often looked MORE similar to reality than correct ones (LPIPS correlation: 0.106). You can't use "does it look right?" to catch mistakes.
3. Some things worked – Hybrid encoders (DINOv2 + VLM) preserve spatial information that VLMs lose. VLMs understand generated video well (93% semantic retention). Small adapters (10M params) work better than large ones (100M).
I'm releasing this as a benchmark proposal. Video generation is improving fast—capabilities that don't exist today might emerge in future models. Seems worth tracking.
Links:
- Demo video: https://youtu.be/YJxDt_zCrUI
- Code + paper: https://github.com/a1j9o94/foresight
- Live demo: https://foresight-demo-kappa.vercel.app
Built with Qwen2.5-VL, LTX-Video, Modal (GPUs), and the Something-Something v2 dataset.
Happy to answer questions about the experiments or methodology.When it gives you a link, it literally takes you to the part of the page that it got its answer from. That's how we can quickly validate.
If they answer your question with irrelevant context, then that's the problem, not that it was AI
In most of my work contexts, people want more formal documents with clean headings titles, detailed risks even if it's the same risks we've put on every project.
I feel like more time is wasted trying to catch your coworkers using AI vs just engaging with the plan. If it's a bad plan say that and make sure your coworker is held accountable for presenting a bad plan. But it shouldn't matter if he gave 5 bullets to Chat gpt that expanded it to a full page with a detailed plan.
Reading the exchange it seems like you got caught up on the illustrative examples the other person used. If a web server isn't a good example of a simple problem for F* the landing page should have an example of something that is. It shouldn't take going through that many steps to understand the so what.