kierangill

Joined 8 years ago

Karma
418
Submissions
70
Member since
Apr 2
I work for Blueberry Pediatrics.

https://www.blueberrypediatrics.com/

Reach me a <first name>.<last name>@blueberrymed.com

https://engineering.blueberrypediatrics.blog/

View on Hacker News

Recent submissions

Comment · 1 month ago
Mine as well. Thanks for the callout. It has been added.
Comment · 1 month ago
Comment · 2 months ago
Comment · 2 months ago
Instead of compaction, has anyone seen a successful implementation of pruning? That is, the agent looks at the conversation history and removes any low-value messages.

For example, sometimes context will be taken up by a side tangent, tool call outputs, or low-value codebase exploration.

Much of the time, I prefer to preserve the history of my conversation instead of summarizing it. I find summarized conversations lead to more frustrating future chats because the LLM misses intent and or context. (Or, the presence of paragraphs and paragraphs of LLM output makes the next token predictor dumber? Unsure.)

Comment · 4 months ago
I work at a telemedicine company. We’ve benchmarked a few frontier LLMs on public medical imaging datasets. One test included high-quality and high-consensus otoscopic images. We didn’t anticipate the models to do well on something so niche, but what concerned us was how poorly calibrated the models were.

I know you can’t trust an LLM’s self-assessed “confidence” of a prediction, but I’ve found that confidence can at least be directionally correct for some tasks. For our benchmarks, however, confidence was poorly correlated. What’s worse is that binary classification models (“Do you see $diagnosis in this photo?”) highly influenced the LLM to confidently predict $diagnosis.

I’m concerned for those using LLMs for diagnostics, and getting confidently led to the wrong conclusion.

Comment · 8 months ago
Comment · 10 months ago
Comment · 10 months ago
Agreed here. A key theme, which isn’t terribly explicit in this post, is that your codebase is your context.

I’ve found that when my agent flies off the rails, it’s due to an underlying weakness in the construction of my program. The organization of the codebase doesn’t implicitly encode the “map”. Writing a prompt library helps to overcome this weakness, but I’ve found that the most enduring guidance comes from updating the codebase itself to be more discoverable.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton