Depending on the use case, it happily runs on my MacBook air M2 16GB ram with mps for small pdfs, and searching over 100-150 documents with colpali takes a 2-ish minutes. Very rough numbers. For ingestion, takes around 15-20-ish seconds a page, which is on the slower end. On an A100, it takes 4-5 seconds per page for ingestion using Colpali to run (we haven't performance optimized, or optimized batch sizes yet tho). Without Colpali it is much faster. Ingestion doesn't change much as size grows.
I'd be happy to report back after some testing, we are looking to optimize more of this soon, as speed is somewhat of a missing piece at the moment.
I'd be happy to report back after some testing, we are looking to optimize more of this soon, as speed is somewhat of a missing piece at the moment.