alrudolph

For the agent working, we're focusing on the user outcome, we think that the raw usage, number of turns, function calls are useful operationally but think of those as more observability than the core evaluation target. We do show some of these stats in our conversation view but don't aggregate to compare agents. Longer term we will look to add in more of these features so we can compare quality vs cost, for example

Yeah, right now I'm not supporting ranges but can see the value and will work on adding in

The demo is setup to match on the description, price and stock columns, when you send a message I try to parse it into searches for each column. I'm realizing with these kinds of searches that I probably have to do more work on that parsing, because its not super clear if that number should be part of the description search or matching against price, etc..

Thanks for sharing, what would you have expected the 80+ to return?