prathmeshmcp

You want to test the full request flow between client and server: connection, tool discovery, tool selection, calls, responses, and whether the user’s GOAL was actually achieved.

What's hard: an external agent sits between your user and your MCP server, in a client you don't control (ChatGPT, claude etc.). Your tests need to cover how that client and agent find and uses your tools.

Recommend checking for :

- (deterministic assertions + non-deterministic judge checks) at essentially every stage of the request flow from client->server (User Value Chain)

- across many clients where YOUR target users are at (unfort. these clients change behavior every other day) and across harness + models

- our default eval assertions cover things like input schemas, valid arguments, repeated calls, errors, latency, and response size; pair those deterministic checks with non-deterministic evals for tool choice, response interpretation, and task completion.