rishsriv
Joined 12 years ago
Karma
39
Submissions
39
Member since
Dec 2
Recent submissions
Comment · 3 months ago
This is particularly frustrating on social media, and abases what might be good ideas.
Too often, my Twitter experience now is - start reading a post with some kernel of insight - read some claudism in the post - stop reading immediately
It seems like people turn over their raw kernel of insight to an LLM to give it polish, likely in order to increase its appeal to the audience. But most of that appeal is lost amongst people that consume LLM written content regularly.
It's also likely that social media engagement does not filter for this. A majority of people are still not neck deep in LLM-talk, and cannot distinguish between LLM and human written content.
It'll probably take us another year or two to develop herd immunity against meat proxism.
Comment · 3 months ago
Glad to hear it! We store point in time vintages going back 10 years (in most cases), with more coming.
Everything that's currently available will continue being free. We're introducing a tentatively $50/mo pricing tier for more data (insights from earning transcripts and leadership podcast interviews, bespoke KPIs from 10Qs, more macro and pricing data, insights from satellite imagery) - but are still figuring out what the right price point should be!
Comment · 3 years ago
Hey HN! We just open-sourced – our text to SQL LLM that outperforms OpenAI's gpt-3.5-turbo on out-of-training-set schemas, and matches gpt-4 when trained on a single business's schema.
SQLCoder is a fine-tuned variant of StarCoder, supplemented with a lot of hand-curated of data and slightly novel fine-tuning techniques.
We are also open-sourcing our framework for evaluating whether LLM-generated SQL is correct. SQL is tricky to evaluate. Two very different SQL queries can both be "correct". For the question, "who are the 10 most recent users from Toronto", both of the following are correct in their own ways – so we had to build a new framework algorithm to evaluate query correctness.
Query 1: ```sql SELECT userid, username from users where city='Toronto' order by created_at DESC LIMIT 10; ```
Query 2: ```sql SELECT userid, firstname || ' ' || lastname from users where city='Toronto' order by created_at DESC LIMIT 10; ```
The model is small enough to run on a single A100 40GB with weights in 16 bit floats, or on a single high-end consumer GPU (like RTX 3090/4090) with 8bit quantization. We will also release a ggml-based quantized version soon, and you should soon be able to run it on most M1 or M2 Macbooks with 32GB of RAM.
The model weights have a CC BY-SA 4.0 license. You can use and modify the model for any purpose – including commercial use. However, if you modify the weights (for example, by fine-tuning), you must open-source your modified weights under the same license terms.
Our evaluation framework is at https://defog.ai/blog/open-sourcing-sqleval/ Interactive demo: https://defog.ai/sqlcoder-demo/
Would love for you to give it a spin, and let us know what you think!
Comment · 3 years ago
This is so sorely needed. I used the app after the PH launch and loved how easy the self-serve was!
Do you have plans to let users define "types" of data that can be redacted (like monetary terms in a contract, code embedded in documents etc)? Also, any plans on making this an API that other developers could build on top of?
Comment · 3 years ago
This looks fantastic. Will try replacing our current fine-tuned FLAN-UL2 model with this.
I wonder how the devtooling around this will evolve. Seems like a matter of days until someone creates a GUI wrapper around this, and obviates the need to use programmer time for fine-tuning
Comment · 3 years ago
Looks promising! Would love to know how this is different from creating postman collections, and running tests on those
W: https://rish.blog X: rishdotblog