mtokarski

Does it miss 443 port config on ngnix?

Interesting work, but I think the interpretation may be a bit overstated. The authors claim that injecting too much factual "knowledge" during pretraining causes models to collapse — performance drops below the baseline once knowledge frequency crosses a threshold.

The problem is how they inject it. Their “knowledge” isn’t natural language; it’s templated Wikidata triples like "X is the capital of Y." That’s a super low-entropy, highly repetitive distribution. When you cram enough of that into a fixed token budget, you’re not really teaching the model more facts — you’re just destroying linguistic diversity and skewing the token statistics.

In real pretraining or domain adaptation scenarios, “knowledge” tends to appear in richer, more varied contexts. The practical takeaway isn’t "don’t add too much domain data," but rather "don’t overrepresent any single format or narrow syntactic pattern" The issue seems more about representation homogeneity than about factual density itself.

I don't see trial download link there, or I missed something?

When I worked in VoIP dpmnt. of some IM provider users were reporting issues like this. Sometimes caller and callee were connected to other party in a random manner. It turned out this strange behavior was caused by bug in PBX software- TDM channels were linked incorrectly under heavy load on E1 links. AFAIK this bug is known for years and it is still not fixed. But... I guess it's not related to your case, since vendor of this PBX is Polish.