macklinkachorn

The users specify the schema and output format and a validation rule and we make sure the system adheres to that.

That’s awesome. Curious to hear your use cases and also happy to share notes on what we see work well. Feel free to ping me (email in Bio)

Super cool! This aligns with our experiences. These tools are great and can get to near 100% of accuracy but it's quite a lot of work on the Eng side to get it there reliably.

Great to hear that you worked saw similar use cases. Doing this before LLMs seem like a big challenge.

Yes, we combine LLMs as a judge with traditional checks like reverse search in original data sources, defining your own post-processing logic, and simple classifier for confidence score.

A few that we experience as we’re building Trellis out:

1. Managing end-to-end workflows from integrating with data sources, automatically triggering new runs when there’s new data coming in, and keeping track of different business logic that’s involved (i.e. I want to classify the type of the emails and based on that apply different extraction logic)

2. Most out-of-the-box solutions only get you 95% of the way there. The customers want the ability to pass in their own data to improve performance and specify their unique ontology.

3. Building a good UI and API support for both technical and non-technical users to use the product.