manmal
Joined 15 years ago
Joined 15 years ago
Given the user is logged in
And the XYZ endpoint is offline
And the dashboard is visible
When the user clicks on the endpoint status
Then an overview of XYZ endpoint errors is shown
If you have 200k context, but only 200t/s prefill, then you’ll wait multiple minutes when continuing an average session from yesterday. Or compaction - might take 15 minutes.
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
It’s less nice when your web app launches that other service only when the other Tailscale node is up, because the agent interpreted your prompt as a hard access rule.
Living near Vienna, Austria.
meet.hn/city/at-Vienna