txtai
Joined 4 years ago
Karma
243
Submissions
198
Member since
Feb 25
Recent submissions
Comment · 3 years ago
Excellent work! Hope to see a version that has a friendlier commercial license in the future (current version is CC-BY 4.0 NC).
Comment · 3 years ago
If you want something similar to this that is fully downloadable and open source, check this out - https://huggingface.co/NeuML/txtai-wikipedia
It's an embeddings database of Wikipedia abstracts with page view data integrated to enable filtering pages based on popularity in addition to similarity.
Comment · 3 years ago
SetFit is a great framework for building a text classifier.
This is a pretty straight forward problem and a good fit for a standard text classifier as well.
Here is an example of fine-tuning a model with txtai: https://colab.research.google.com/github/neuml/txtai/blob/ma...
Comment · 3 years ago
txtai combines Faiss and SQLite to support similarity search with SQL.
For example: SELECT id, text, date FROM txtai WHERE similar('machine learning') AND date >= '2023-03-30'
GitHub: https://github.com/neuml/txtai
This article is a deep dive on how the index format works: https://neuml.hashnode.dev/anatomy-of-a-txtai-index
Comment · 3 years ago
txtai combines SQLite and Faiss to enable vector search. It also does a lot more than that.