jlamberts
Joined 9 years ago
Joined 9 years ago
A good engineer will acknowledge this tradeoff between robustness and cost and behave accordingly. For example, if you're working on safety critical or very foundational systems like OSes, medical tech, etc you should bias very heavily in favor of robustness. If you're not, this can easily be an act of overengineering. The engineer's job is to find the right spot along the cost-correctness curve for the thing they are building.
This has always been true, and LLMs just change certain parts of the equation. For example, code writing is far less of a bottleneck than before, so "we can just try with a throwaway impl and see if this works" is suddenly economically viable. It also turns out that many things, in practice, don't need to be as correct as some of us may have believed.
We can still enjoy making quality things, but doing so is often an act of artisanship rather than engineering.
> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.
The full essay is worth a read, it's pretty short http://www.incompleteideas.net/IncIdeas/BitterLesson.html
I agree that a randomized control is extremely valuable, but more as a way to collect unbiased data than a way to validate that you're outperforming random: it's pretty difficult to do worse than random in most recommendation problems. A more palatable way to introduce some randomness is by showing a random item in a specific position with some probability, rather than showing totally random items for a given user/session. This has the advantage of not ruining the experience for an unlucky user when they get a page of things totally unrelated to their interests.
Also, a dramatic anime intro (complete with cheesy AI generated theme song and video) starring our foster kittens. It's been interesting to learn about some of the techniques needed for consistency, how to storyboard, etc.
K-means could potentially be helpful to group together common users/items if e.g. you're memory constrained and don't want to give each user a fully unique embedding entry so that's also possible.