joshuaisaact
Joined 2 years ago
Karma
46
Submissions
22
Member since
Nov 5
Recent submissions
Comment · 7 months ago
Comment · 8 months ago
ASML is European and is arguably the most strategically important company in the entire semiconductor supply chain.
Comment · 8 months ago
Have you heard of a little company called Arm Holdings?
It was a travesty that the UK government let it be sold, admittedly.
Comment · 8 months ago
I've been exploring Petri nets as a formalism for AI agent safety, specifically, proving properties like termination and human-gate enforcement exhaustively across every reachable state, rather than testing them on sample inputs. This post benchmarks the approach against n8n and ReAct on the same agent. Tomorrow I'm open-sourcing the engine as a declarative rules DSL.
Comment · 8 months ago
I don't think you need two separate models for this - I get similarly good results re-prompting with Claude. Well, not re-prompting, I just have a skill that wipes the context then gets Claude to review the current PR and make improvements before I review it.
Comment · 9 months ago
Couldn't disagree with this article more. I think the future of software engineering is more T-shaped.
Look at the 'Product Engineer' roles we are seeing spreading in forward-thinking startups and scaleups.
That's the future of SWE I think. SWEs take on more PM and design responsibilities as part of the existing role.
Comment · 9 months ago
Fair push back. The distinction I'm drawing is between:
A. Using a role prompt to configure a single function's scope ("you are a code reviewer, focus on X") - totally reasonable, leverages training
B. Building an elaborate multi-agent orchestration layer with hand-offs, coordination protocols, and framework abstractions on top of that
I'm not arguing against A. I'm arguing that B often adds complexity without proportional benefit, especially as models get better at long-context reasoning.
Fairly recent research (arXiv May 2025: "Single-agent or Multi-agent Systems?" - https://arxiv.org/abs/2505.18286) found that MAS benefits over single-agent diminish as LLM capabilities improve. The constraints that motivated swarm architectures are being outpaced by model improvements. I admit the field is moving fast, but the direction of travel appears to be that the better the models get, the simpler your abstractions need to be.
So yes, use roles. But maybe don't reach for a framework to orchestrate a PM handing off to an Engineer handing off to QA when a single context with scoped instructions would do.
Comment · 9 months ago
Fair point on the date - the paper was updated October 2024 with Llama-3 and Qwen2.5 (up to 72B), same findings. The v1 to v3 revision is interesting. They initially found personas helped, then reversed their conclusion after expanding to more models.
"Comprehensively disproven" was too strong - should have said "evidence suggests the effect is largely random." There's also Gupta et al. 2024 (arxiv.org/abs/2408.08631) with similar findings if you want more data points.
Comment · 9 months ago
This has been pretty comprehensively disproven:
https://arxiv.org/abs/2311.10054
Key findings:
-Tested 162 personas across 6 types of interpersonal relationships and 8 domains of expertise, with 4 LLM families and 2,410 factual questions
-Adding personas in system prompts does not improve model performance compared to the control setting where no persona is added
-Automatically identifying the best persona is challenging, with predictions often performing no better than random selection
-While adding a persona may lead to performance gains in certain settings, the effect of each persona can be largely random
Fun piece of trivia - the paper was originally designed to prove the opposite result (that personas make LLMs better). They revised it when they saw the data completely disproved their original hypothesis.
Maybe I'll try that myself, because it feels like it could be a great source of improvements. It would be really useful to see adaptive per-token sampling as an additional decode-only baseline.