antti909

Thanks for mentioning Haystack :)

Heh, you seem to keep asking :) You could also ask in our community Discord, tbh, there are people who have been trying both.. There's definitely a ton of great things about langchain, so I'd be curious myself!

To be precise - I don't think I'm saying 'local LLMs' above :) But technically possible, I guess, just hasn't been part of what's officially available. (There are also licensing issues still.) To answer your question about the APIs - the Agent itself queries OpenAI via REST to break the prompt down into tasks, then works with the underlying tools/pipelines using Python API (and then, e.g., a Transformer model that's part of the pipeline has to be 'loaded' into a GPU). Part of those pipelines might be using Promptnode (that can work with hosted LLMs via REST, but could also work with a local LLM). Re 'subsume' - well, that depends :) But arguably, you can build an NLP Python backend with Haystack only, of course.. Regardless of how complex your underlying use case is, or whether it's extractive, generative or both.

Thanks :) Working on it. Re local models - indeed, all started with using the Transformer models for extractive QA and semantic search. With the Promptnode, and/or the Agents it's also now possible to combine local models/pipelines & 'LLMs' freely.

For the Agents? Yes, indeed. Referred in the article.