rawsh
Joined 6 years ago
Karma
51
Submissions
19
Member since
May 5
Recent submissions
Comment · 2 years ago
Comment · 2 years ago
Bit confused what the value add is over a framework like DSPy. This still requires you to create an eval dataset with ground truth, basically the only hard part of using DSPy. Easily getting the optimized prompt and having some metrics out of the box is not worth nearly $1k/mo IMO
Side note: I’ve had a lot of luck combining automatic prompt optimization with finetuning. There is definitely some synergy https://raw.sh/posts/chess_puzzles
Comment · 2 years ago
I built a web version with WASM at https://pdfgrep.com a few years ago in case it’s helpful to anyone
Comment · 2 years ago
https://github.com/VikParuchuri/marker is solid, but slow and needs gpu(s) to be practical
Comment · 3 years ago
Is it possible to use this for hybrid search in combination with pg_embedding? My understanding is that hybrid search currently requires syncing with Postgres
Comment · 3 years ago
Documents actually never get uploaded! PDF text extraction happens on the client using a web worker and MuPDF compiled to WASM.
1. PDF parsed and chunked on the client
2. Sparse vectors are regenerated for the entire document corpus and the existing vectors are updated
3. Dense vectors are generated for the new text and upserted along with the new sparse values
The original documents stay on your device.
Comment · 3 years ago
Nope, it’s a serious project; I mostly made it for personal use during my last semester of college. I rewrote it a few times and packaged it up because I think it’s genuinely useful. Langchain gets you 80% of the way there but you run into issues with it very quickly.
On lichess puzzles gpt4o with the compiled prompt is around 70%, I think the 270M transformer is around 95%