cameronmpalmer

I'm an AI implementation engineer consulting with real business implementing LLMs. In my most recent post, I investigate how prompting with an expert persona ("act like an expert") actually affects factual accuracy of language models when answering multiple-choice questions, and whether the added cost is worth it.

I’m an independent consultant building agent systems. I recently went down a rabbit hole finding exposed shared Claude conversations. I found over 6000 of them, and it led me to a detailed case study of how LLMs can have extremely negative effects. Who do you think should take responsibility when LLMs cause harm, the labs or the user?

Agreed. In a development environment where you don't want to walk your agent how to do the same thing over and over I use either an MCP or a CLI tool. This is SO important in production agents too because there isn't a dev working directly with the LLM all the time. What kinds of tools have you personally built to automate stuff?

Exactly! This is a huge thing I preach in development workflows, if you're going to need to do something more than once (or have your agent do something more than once), make it repeatable and reliable. What kinds of things have you seen given to LLMs that should have been code?

I’m an independent consultant building agent systems. I kept running into the same problem where my clients and I tried to use LLMs when regular code was the better tool. The last straw for me was a prospecting agent that ruined 10-20% of the records it inserted into my CRM, and I had to review all 800+ entries by hand. I thought I’d write up what I’ve learned from this experience - I landed on a six-question framework that has helped me greatly when deciding if a solution could benefit from an LLM. I’d be interested to hear what boundary questions others use.