patresh
Joined 9 years ago
Joined 9 years ago
I occasionally read articles on Nate Silver's substack but I'm still missing the breadth of 538conbined with the trademark data-driven analysis.
Those are not contradictory: a company's inference costs can increase due to deploying more models (Sora), deploying larger models, doing more reasoning, and an increase in demand.
However, if we look purely at how much it costs to run inference on a fixed amount of requests for a fixed model quality, I am quite convinced that the inference costs are decreasing dramatically. Here's a model from late 2025 (see Model performance section) [1] with benchmarks comparing a 72B parameter model (Qwen2.5) from early 2025 to the late 2025 8B Qwen3 model.
The 9x smaller model outperforms the larger one from earlier the same year on 27 of the 40 benchmarks they were evaluated on, which is just astounding.
For standard questions I feel like it doesn't matter too much what you use. When it comes to multi-step searching + reasoning flows like look for alternatives, fetch pricing, feature lists, compare etc, the differences are larger because of the engineering glue and prompting around the pure LLM inference which makes the tools more or less powerful.
Moderation for sure helps, would there be ways to make it scalable with less manual supervision? Or a system that would organize people with certain rule-sets to distribute them into suitable sized groups?
I do agree with your statement that "Good discussions evolve naturally and also randomly", let's say now your platform becomes popular. It will attract players that will want to exploit that either to sway opinions for their own gain, and I believe that this is becoming increasingly cheaper to game and simulate whole crowds. So the limits are mostly with this in mind.
Indeed perhaps the term social platform is vague and different "optimal rules" could be different for social platforms that is a mega-forum, a network for friends, or just generic post sharing.
I'm wondering if there is some sort of taxonomy of these rulesets or levers that exist? Or a review paper on what has been tried and what effects they had? There are so many possible ways to structure online social interactions.
I wonder if it would be possible to simulate this to understand what behaviors will emerge if you set certain types of rules. It is certainly difficult to create coherent personalities with LLMs that act in realistic ways but I wonder if one could get an approximation.
Perhaps what I have in mind is also not best described as "pleasant", but also something that is net-positive for society, where as a whole society is better off having that than not. This is arguably the case for HN but not necessarily for some of the bigger ones out there.
Typically LLMs don't produce usable embeddings for clustering or retrieval and embedding models trained with contrastive learning are used instead, but there seems to be no mention of any other models than LLMs.
I'm also curious about what type of clustering is used here.