Arimbr
Joined 14 years ago
Karma
318
Submissions
134
Member since
Nov 11
Data & Content Engineer. Building https://toolsfordata.com/
View on Hacker NewsRecent submissions
Comment · 1 year ago
I like the YAML abstraction. This should make it easier to programmatically try and evaluate multiple configurations for the whole AI pipeline (not just the LLM) against a dataset or real users through an API deployment.
Some feedback: It would be great to see in one place all the supported fields and values for the YAML config.
Comment · 2 years ago
What is an hybrid index?
Comment · 2 years ago
Nice, thanks! I was reading https://pathway.com/developers/user-guide/deployment/persist.... If I understand correctly you persist both source data and internal state, including the intermediary state of the computational graph. And you only rely on the backend to recover from failures and upgrades. So if I want to clone a Pathway instance, I don't need to reprocess all source data, I can recover the intermediary state from the snapshot.
Is it the same logic for the VectorStoreServer? https://pathway.com/developers/user-guide/llm-xpack/vectorst...
Comment · 2 years ago
If all the pipeline and the vector index is keep in memory... does Pathway still persist state somewhere?