jamesmcintyre
Joined 17 years ago
Joined 17 years ago
https://github.com/tantaraio/voy
https://github.com/nitaiaharoni1/vector-storage
https://github.com/danielivanovz/indexed-vector-store
https://github.com/yusufhilmi/client-vector-search
Another area I've looked into briefly is also trying to generate vector embeddings fully in-browser. I know there will likely be tradeoffs with dimensionality / overall accuracy/performance but having a semi-decent way to do this in-browser would be awesome.
This project only runs in node (but there's an issue that talks about how it might be modified to run in-browser):
https://github.com/Anush008/fastembed-js
Looks like transformers.js supports embeddings in-browser:
https://github.com/xenova/transformers.js/releases/tag/2.1.0
https://github.com/pinecone-io/semantic-search-example/blob/...
Also it appears the client-vector-search supports both embeddings & vector indexing
https://github.com/yusufhilmi/client-vector-search
Hope this helps others looking into this stuff!
"the laser projector adds X more development months, X less hours of battery life, and X more costs to manufacture due to higher power needs therefore bigger battery, more heat management.. should we launch v1 without it? We can just push-notify your phone if we detect they need a display and they're reaching for their phone, they swipe notification and we take them to exactly what we were trying to show them. Or we can show it on their watch and it would serve as a forcing function for us to build out the voice UX to try and do more with voice."
"I get that we're introducing a new ux modality but shouldn't we give user's a pathway that blends between already heavily embedded habits (smartphone, watch) to ours? User's should be able to carry on existing txt conversations from their phones, call from their existing phone numbers, hear notifications from their phone, etc. In fact- these could be great hooks for us to get them used to using our natural language UX more frequently."
"I get that the device needs to look like beautiful jewelry but metal is heavier- can't we find a way to make a fully plastic enclosure look high-end?"
"Our software stack isn't ready! We can put compute and llms closer to the edge to reduce latency, train 'micro-llms' for very specific tasks that will run faster and cheaper, etc. etc."
For all of these hypothetical contentions I can equally imagine very valid reasons for choosing the paths Humane has chosen. Letting these objections always drive the product development could have resulted in a much worse outcome. There's a plethora of anecdotes about Steve Job's refusing to compromise on the many facets of the original iPhones design but if I had to guess there are less stories repeated about how many compromises he was ok with.
It's been notoriously difficult to introduce new consumer product categories- if it was easy or more obvious the Apple watch would already be at-parity with 80-90% of the features (which one would assume is what will happen in short order, watch & airpods as proxies to an embedded+edge llms on iPhone).
I can understand viewpoints from both those who are trashing the Humane ai pin and those excited about it but one thing I'm happy to see is a team of this caliber making bold moves (and a investment ecosystem to support it). With v1 out (remember, a lot of hardware startup never make it this far) and the community responding it will be interesting to see how deftly they can navigate rapidly iterating and improve the ux.
Retell focusing on all of the other weird/unpredictable aspects of human conversation sounds super interesting and the demo's incredible.
Things are moving so fast, wow.
Vercel is increasingly trying to offer all these integrated goodies too but imo right now most of them are off-the-shelf things like "a postgres db" (powered by neondb platform) a "kv redis store" (powered by upstash) and they seem to be betting their user's want to use their own pg/redis cli's & sdk's to manage & access the data from the edge functions (versus supabase's sdk's which unify auth, db, functions, etc.).
Both options do have their benefits but if you want everything tightly integrated with a friendly management UI, sdk & auth Supabase can save you tons of time!
In terms of speed I still think nothing beats cloudflare workers (way more geographically distributed points of presence so even closer to more users). However I think vercel's edge functions only use a subset of the total cloudflare workers PoPs. But I highly doubt the average use-case or user would ever notice the difference in latency between cloudflare workers & supabase edge functions so i don't know if that should be your determining factor. (And let's not get started on the debate of "edge doesn't matter if it's far from your db". If supabase is doing any magic to make their edge functions "close" to the db that's a huge win!)
Another option is to go full cloudflare ecosystem (they too have a kv store & now a limited sql db) but, again, I do not think their ecosystem gives you the all-in-one goodness that supabase does (no auth offering, definitely no way to tie together db/kv/auth with one sdk/management dashboard).
Yet another option is to go with fly.io + tursodb. Like deno & cloudflare workers they have their own geographically distributed PoP's that run lightweight, optimized firecracker vms running any of your dockerfile desires. With this you get even more freedom and control but you also need to get your hands a bit dirty in flyio's cli & platform magic. Also while turso is super interesting it's very new and they seem to be pushing more of a "have a database per user" than "host one giant multitenant db".
I have compared all these offerings, weighed the pros / cons. In the end don't let it paralyze you! Nearly all these offerings have enough standardized pathways in and out of them (or similar standard protocols like sql, postgres protocol, exports, cf runtime even has limited node/npm support for instance) to where lock-in should not too big a fear in the beginning. Worst case try to write abstractions for db-access, auth-hooks, kv-access, maybe runtime-specific logic so that you can adapt the abstraction to benefit app-wide if you ever need to migrate to another runtime/db.
Hope this helps!
I love the Mercury OS concept and think it's design both elegantly and sort of subversively packs a myriad of potentially breakthrough ideas.
I have been stewing with ideas around the same vision for years. The idea of a new type of UI where the UI seams to dematerialize, where you directly manipulate the object in your current context (like multi-touch's direct manipulation but at a higher layer of abstraction powered by deep api integrations, intelligent self-assembling relational graphs, and of course ai). For over a decade I've had this thought "the data becomes the UI" like an emergent UI from whatever given data, task or context you are currently in. When I came across the mercuryos concept I immediately smiled.
Conceptually, strategically and technically there are so many challenges to introducing such a new ux paradigm but I'm very happy to see the mercuryos concept has seemed to evolve to New Computer's Dot and I wish them the best!
For those immediately turning to negative sentiment based on privacy or "it's just a gpt4 wrapper" I can see why that could be the knee-jerk reaction but I wouldn't underestimate a sturdy design-philosophy approach like this one. I'd go as far as to make comparisons to Next Computer's NextStepOS. NextStep introduced so many groundbreaking UX concepts and to a large extent I think their personal computing contemporaries underestimated what potential it packed. And, yes, I know the business model and many other factors played into an inevitable doom for Next Computer but there's belief that Steve Jobs may have never intended for Next to become a dominant computing player and instead knew it'd be an irresistible acquisition target in a latent space of UX innovation. It's possible he saw the next evolution of personal computing UX and hedged his bet on not compromising on it. Yet another comparison could be that NextStepOS needed more cpu, graphics and connectivity power to truly display it's heightened level of UX much in the same way something like Dot or mercuryOS would inherently need to leverage cutting-edge computing to truly enable it's vision (obviously LLM's, Vector DB's, etc).
Ok, I'm done, lol.
Quick question, how does it know to choose plugins? I think you add a plugin node to the graph but are there ways in which you can describe what the plugin does so that an llm could dynamically choose amongst plugins? Or is this just building out a graph with nodes that explicitly choose amongst plugins via prompting?
Thanks so much for open sourcing this! Great work!
(also love that your docs authoring look to be assisted by llm's, so meta: https://rivet.ironcladapp.com/docs/api-reference/node/overvi...)
- investors may see more options for acquisition exits
- if in a crowded space you still see a gap it's likely real
Cons: - table stakes to reach parity with existing solutions may be higher
- existing entrants may be quick to replicate your killer feature
- with more noise demands more marketing/differentiation/ads/baked-in virality
I'm sure you've already thought of all that jazz. One idea is, if you can find any online communities which overlap with your target demographic maybe develop a survey (crude mockups could be even better in the survey) and ask if that feature set would entice them to switch from their current solution.
Another approach is to use the producthunt "coming soon" with decent mockups / landing page to gauge interest?
Maybe find a content creator that specializes in this space and propose a interview style video where you open the book and bounce ideas for what's missing in the market and what your solution would be. This would likely be the most fruitful move but you'd be putting it all out there which is the only way to make this type of video work for both parties.
Lastly, YC never seems worried about funding startups in a crowded space especially if the market doesn't seem too matured as there's still time for a new entrant to "get it right".
Good luck!
I know it's a POC but just some of my thought's / questions to see if you have any ideas or opinions on them:
- I briefly browsed workerd repo and it doesn't seem like there's an api to load a new worker script file to the existing workerd instance. Which is, i'm guessing, one reason why you're just restarting the whole fly machine to re-fetch the latest zipped resources at boot. - workerd repo mentions it's currently single-threaded - i'm wondering how fast workerd itself boots? - due to ^ those three points I'm wondering if a nice design would be: - publisher & worker use bullmq and it's sandbox-worker to spawn child process workers running workerd (helps utilize more threads & the bullmq worker code can take the job data input which contains the new user script to bootstrap config at job run). This would be good IF workerd boots fast.. - if workerd doesn't boot fast, and eventually you have many user scripts i can't see any other option than to have a third server that acts as a load-balancer/orchestrator. It would keep state of user scripts and which worker fly machines have told it it can run that script. Could still use a "buffer worker machine" which can "cold-boot" newly published scripts using the bullmq method i mentioned above.
Anyway, this is all really cool just wanted to spitball some ideas!
- Given how neon architecture decouples compute with storage using the safekeepers & pageservers will this still just work with neon? (Wondering because you mention the index is in-memory so unless your stateless compute nodes can somehow hot-swap the indexes I wasn't sure how that'd work?)
- Do you plan to offer vector search as a plug-and-play offering? If so does neon as a product offering plan to introduce more "out-of-the-box" functionalities like vector search? (Similar to xata's offerings like search & vector search.)
- unrelated question- I believe neon has very small cold startups for free/scale-to-zero configurations but are there also inherent small latencies for infrequently accessed data? In other words is it safe to say if there were a large table with records that are "old"/"archival" but also always a sort of ~last 30-days of records that are "fresh"/"accessed with more frequency" would there more likely be slight latency introduced when accessing the older records?
Neon looks awesome and thanks for neons open source contributions!
For various projects in the past I’ve dabbled in generating selectors so I’m curious what you’ve come up with!
My more recent interests have been to find a way to generate more robust selectors that could handle slight-to-mild DOM structure changes (also cannot rely fully on classes, ids, html attributes). This wouldn’t necessarily be for testing but could apply. There are applications that need the ability to find an element despite its location in the dom changing slightly (ie its now nested one div deeper). For instance most xpath or “traverse down the dom until find parent div” type selectors would be too fragile. What i’ve been trying to figure out is a way to “fingerprint” the target element and have a algorithm that has a few ways to look for that element, each failure would try a less strict, more flexible method.
Just curious if the work you’ve done ventures into anything like this or if in your research you came across anything like that?
https://twitter.com/jamesmcintyre https://www.jamesiscoding.com