gergelycsegzi

We’ve been building our startup at the intersection of retrieval and document processing. Having pitched it to customers, team members and investors, I realized that I had priors that I had not shared. Thus, I set up visualizations for the major search and retrieval methods that exist out there. Hope you find them interesting. Let me know what you think!

. .

P.S. Disclaimer before you jump in: every use case comes with different trade-offs and should be considered individually. I had to make generalizations to be able to create comparisons. So if you’re a die-hard fan of say Hybrid RAG, please know that I agree that there are use cases where the trade-offs work out in its favor.

Re 1 - that is a very kind offer! Our current public template library is very limited, so let me come back to you on this.

2. We see exactly the same thing. There is a trade-off in correctness vs token burning. However, some tokens (models) are cheaper and faster than others, so the small pieces can benefit from that. The token usage is also surprisingly variable, because it depends on the information density of the document and also on the information density of the question (e.g. is it a single needle in a haystack or are we analyzing the entire haystack from 10 perspectives). So the parsing for 1k pages may be on the order of millions of tokens, while a series of queries (extractions) on top of it could be 1-2 orders of magnitude more.

Yes, we do it by having multiple stages to the pipeline. First we would extract the independent data points (from say both page 4 and 40) and a second pass step establishes relationship (we call this resolution).

On the scale aspect, because we go in multiple passes, we break the scope into small enough pieces and then build it back up in a later step. Iirc the largest document I've seen a customer use was over 1k pages.

There are more complex data dependency scenarios where we find that the data that's extracted and combined (e.g. from page 4 and 40), needs to then be further transformed in different ways (e.g. having an evaluation and a clarification outcome at the end). To make these be aligned in value we are soon releasing a feature for what we call derived agents.

This does indeed look really interesting. We have deterministic validations (and some deterministic excel transformations) but using more deterministic transformations for text based on traditional NLP would be a nice complement.

If Claude is good enough for your use case then for sure. If you need scale, persistent structure and verifiability we can help:)

Haha thanks, the reader can try and guess which is which;)

We actually don't use embeddings or vector similarity, since those tend not to work well in specialist domains (e.g. for the OfficeQA benchmark where we have 90k pages talking about US treasury numbers, they would be mostly mapped to a very small embedding space because it's all the same topic, with small variations across years, expense categories etc.).

We use LLMs for the extraction and comparison as well, and we route between different models depending on the complexity of the comprehension of the given step required (and by this I mean routing between our pipeline steps; we currently do not dynamically try to judge individual cases for complexity like OpenRouter Fusion).

I can see why, it's tempting to go for full automation. The reason we go for fine grained sourcing is so that people can build their awareness quickly. Plus many of our customers work in regulated industries where full automation is prohibited.

Potentially, but at that scale cost and latency may actually become an issue, so probably better to consider some sort of indexing or keyword searching.

Hey, that's exactly it!