I built Gemini Cursor, an open-source multimodal AI cursor that guides users through tasks on their desktop by pointing and speaking.
It leverages Gemini 2.0 Flash and Google's live multimodal API to analyze what's on screen and provide real-time assistance.
In this demo, my friend tries to add a payment method to Amazon, and the AI cursor walks them through the entire process with visual cues and spoken instructions.
I've also used it to interpret diagrams from research papers—curious to see what other use cases people find this useful for!
It leverages Gemini 2.0 Flash and Google's live multimodal API to analyze what's on screen and provide real-time assistance.
In this demo, my friend tries to add a payment method to Amazon, and the AI cursor walks them through the entire process with visual cues and spoken instructions.
I've also used it to interpret diagrams from research papers—curious to see what other use cases people find this useful for!
Demo: https://x.com/27upon2/status/1889128655672029582
Repo: https://github.com/13point5/gemini-cursor