mkaszkowiak

What was your approach to benchmarking an adversarial agent?

This is an open problem that I came across (in a different domain), as the search space can be really wide. It's hard to measure results for non-trivial tasks.

Would be really interested if you can share your eval approach :)

Google is killing Android. Along with the side-loading changes, I'm losing the desire to keep using it, as it's no longer an open OS.

What's the point of those changes? Does Google want to maintain its revenue from Play Store? Feels like a bad long-term decision, especially when Apple is releasing excellent phones.

Happy to see competition in rerankers! Good luck with your product.

My questions: what languages do your models currently support? Did you perform multilingual benchmarks? Couldn't find an answer on the website

Easier to handle edge-cases - real examples:

- What if certain rows in a table don't need to be embedded?

- What if we use a single API key for embedding database rows and user queries and it hits a rate limit - how to prioritize user queries?

- What if some rows should be vectorized using a different model, depending on an external configuration?

Glad to hear that :) Thanks for developing Marker!

Thanks for answering! In my case, I don't directly use RAG; but rather post-process documents via LLMs to extract a set of specific answers. That's also why I've asked about deduplication - asking LLM to provide an answer from 2 different data sources (invalid unstructured table text & valid structured table contents) quickly ramps up errors.

How do you combine the outputs? Wouldn't there be data duplication between unstructured text and tables?

Did you encounter hidden costs when using Azure Document Intelligence? I processed some PDFs using the paid tier, but the resulting costs were way higher than expected, despite using a prebuilt layout model for only structured extraction. Have no clue what could cause it, no extra details on the billing page. Not sure if the price is misleading, or if it's a skill issue on my part :)

For my use case, overall Marker seems to work pretty well - but it has issues with tables. Merged cells, misplaced headers, and so forth. I'm currently extracting Polish PDFs that are //not// scanned

When compared to Azure Document Intelligence, Marker is really cheap when self-hosted (assuming you fall under the license requirements), but it does not produce high quality data. YMMV.

This aurora was really powerful! I could see it with a naked eye from a town in central Poland, despite cloudy weather and light pollution. Feels great to finally see it in person

Surprised by the amount of negative comments. Kudos to the team! This is very impressive to accomplish in 24h with a 3-man team.

Thanks for the links! I'll read them :)