matt8p

Hi HN,

I was experimenting with writing writing unit tests for MCP servers. I find unit testing incredibly useful for writing assertions for every MCP primitive: tools, prompts, and resources.

This helps with ensuring that future changes aren't breaking the server's behavior. To do this, I used a simple set up with Jest and MCPClientManager, a custom MCP client I wrote for testing.

Hope you find the article an interesting read!

Matt

Neat!!

Thanks for sharing this!!

Hi folks, we've been working on a CLI tool to programatically test and eval MCP servers. Looking to get some initial feedback on the project.

Let's say you're testing PayPal MCP. You can write a test case prompt "Create a refund order for order 412". The test will run the prompt and check if the right PayPal tool was called.

The CLI helps with: 1. Test different prompts and observe how LLMs interact with your MCP server. The CLI shows a trace of the conversation. 2. Examine your server's tool name / description quality. See where LLMs are hallucinating using your server. 3. Analyze your MCP server's performance, like token consumption, and performance with different models. 4. Benchmarking your MCP server's performance to catch future regressions.

The nice thing about CLI is that you can run these tests iteratively! Please give it a try, and would really appreciate your feedback.

Congrats on the launch!! How is privacy handled? I would like to try it but I have emails I don't feel comfortable being consumed by third party. Thx!

For context, I've been working on the MCPJam inspector. It's an open source dev tool to test and debug MCP servers. We did an entire rebuild from Vite + Express to Next.js two weeks ago. We did this out of personal preference - we've built stuff in Next.js before and like its routing system and built in backend.

Switching to Next was a mistake for our use case. We didn't consider that our users are starting MCPJam with npx. Our npm package size exploded to 280MB. Next.js was too heavyweight for a locally ran web app. Switching back to Vite + Hono brought our package size to 9MB, much more manageable.

This post isn't to bash Next.js. It's just to remind you that tech stack does matter. We didn't think about the consequence of switching to Next and didn't consider our users' use of npx. If MCPJam was a hosted webapp, it would probably matter less. Remember to think about your stack's tradeoffs before you commit to building!