mrfox321
Joined 7 years ago
Karma
396
Submissions
193
Member since
Jul 26
Recent submissions
Comment · 3 months ago
No. Distillation trains on a teach model's logits or output tokens.
Comment · 11 months ago
... the toppings contains potassium benzoate
Comment · 1 year ago
I know right?
They have always been behind. Why would this time be any different?
Comment · 1 year ago
Read the paper.
They control for the data being in-distribution
Their dataset also has examples of the problem being solved.
Comment · 1 year ago
Sounds like excuses
Comment · 1 year ago
They didn't show this, they just increased the length where accuracy breaks down.
Comment · 1 year ago
What's your leg circumference at, now?
Mine is also smaller, due to patella tendinopathy.
Comment · 1 year ago
Who do you think the shareholders are?
What demographic are they?
Comment · 1 year ago
It's because they originally built their recommendation system to recommend friends and their content. Here, the social graph makes complete sense as the foundation for their simple search algorithm.
But as they expanded their recommendation capabilities, the features stuck around. It's the same reason why tech debt accumulates. Data sticks around in the same way code does. But data is even higher friction, since it's a superset of the code.
Comment · 1 year ago
I mean, it's their serving infra, but not their features. See my reply in another parent comment.
Comment · 1 year ago
It's simpler intuition but more complex from a data / ml perspective.
Their algorithm is really built around their features. Specifically, temporal representations of user interest:
https://ieeexplore.ieee.org/document/9458799/
The features used by their algorithm tells you what a user is interested, historically.
Contrast this to Meta, which uses the social graph as their features. Imagine features like the number of times a user likes another author's / cluster's content.
Tiktok will serve you $TOPIC because you have $INTERACTED with $TOPIC historically.
Meta will serve you $TOPIC because you have $INTERACTED with $PEOPLE who post $TOPIC, historically.
Meta only coincidentally gives you what you like.
Tiktok knows what you like.
This is the difference. This is why IG is losing.
Comment · 1 year ago
IG users and Tiktok users were / are quite similar. Especially when Tiktok wasn't yet eating IGs lunch.
Comment · 1 year ago
IG has plenty of data (I did ML at IG). Don't be naive.