grbsh
Joined 3 years ago
Karma
117
Submissions
72
Member since
Mar 20
Recent submissions
Comment · 1 year ago
Comment · 1 year ago
Why not just use Claude by itself? Opus and Sonnet are great at producing pixel coordinates and tool usages from screenshots of UIs. Curious as to what your framework gives me over the plain base model.
Comment · 1 year ago
I recently got the Pavlok device to stop a bad habit - compulsive nail biting. It worked extremely well! It works less well for thinks like distraction, because you have to manually trigger the negative stimulus.
I find the prospect of an AI continuously monitoring and course correcting very interesting. Like GPS/ maps -- if you go off course, it re-routes you. What will people build with real-time AI like this in the future? I think Cluely may be the best example of this.
I'd personally like a version of the Pavlok device that has a built in camera / mic that allows me to configure these type of triggers with real-time monitoring. It'll probably be a while until we can run an LLM on a watch-sized device in 1-3s but still have ~8hr battery life.
Comment · 1 year ago
Knowledgework is AI that can actually save you time, make you better organized, and do better work. Knowledgework is the answer to question, “Why isn’t ChatGPT more useful for real-world work tasks?”
While ChatGPT is a better Google search, Knowledgework is like asking a copy of your own brain for help. Today’s chat interfaces are flawed because they require the user to micromanage: to not only know exactly what can and should be done, but to describe in detail all the specifics required to do it. Knowledgework fixes this with two elements that I believe create a new paradigm for AI enabled software: proactivity and omniscience. Without these two properties, you can’t delegate meaningful real world tasks to AI.
ChatGPT isn’t useful for this. Not because it’s not intelligent enough (it is), but because it lacks the right knowledge. When I speak to knowledge workers about how they use AI for work, they tell me they’d like to delegate more tasks to AI, but that they experience this frustration of needing to micromanage it, re-teaching everything about their project and their team every time.
How does Knowledgework solve omniscience and proactivity to create an AI assistant that’s useful for delegation of real-world work? It’s a desktop vision AI that watches you work, like an intern who is shadowing you, or a pair programmer. It learns the rich, internal (often unwritten) knowledge specific to your projects that's required to contextualize them. It organizes this into neat, understandable documentation of everything you’re doing: a hyperlinked wiki that connects all of your team’s concepts, decisions, definitions, acronyms, tools, etc.
This explicit representation of your knowledge powers the AI assistant to enable useful delegation — but it’s also really useful in it of itself. Curious as to how your team came to a decision on something? Click through the wiki to get context.
The other main feature is the Timeline. It’s kind of like a log or an objective summary of how you spent your time. While the wiki mirrors humans’ associative and conceptual memory, the timeline represents episodic memory. This enables you to visually search through your time: imagine you remember solving a similar problem a few weeks ago, but you don’t quite remember when. By going through the Timeline, you can quickly scan to find the specific work session and ask about what you did.
Together, these representations of your knowledge and experience along with the AI assistant running on top begin to feel like a sort of “digital second brain”. Since I started using it, I’ve had the experience where I’m hesitant to even do things on other devices, because it feels like anything I do there is ephemeral.
If you’re excited to upload your mind and see what the future looks like with this tech, sign up for the waitlist here: https://knowledgework.ai.
Comment · 1 year ago
Ooh, I really like the idea about deciding whether to use the big or small model based on task specificity.
Comment · 1 year ago
I know moondream is cheap / fast and can run locally, but is it good enough? In my experience testing things like Computer Use, anything but the large LLMs has been so unreliable as to be unworkable. But maybe you guys are doing something special to make it work well in concert?
But let's assume you're much better than average at understanding code by reviewing it -- you have another frustrating experience to get through with AI. Pre-AI, let's say 4 days of the week are spend writing new code, while 1 day is spent fixing unforseen issues (perhaps incorrect assumption) that came up after production integration or showing things to real users. Post-AI, someone might be able to write those 4 days worth of code in 1 day, but making decisions about unexpected issues after integration doesn't get compressed -- that still takes 1 day.
So post-AI, your time switches almost entirely from the fun, creative act of writing code to the more frustrating experience of figuring out what's wrong with a lot of code that is almost correct. But you're way ahead -- you've tested your assumptions much faster, but unfortunately that means nearly all of your time will now be spent in a state of feeling dumb and trying to figure out why your assumptions are wrong. If your assumptions were right, you'd just move forward without noticing.