> Oof - it's not like goodfire - we have quantitative results and a clearly defined method!
Goodfire is a discovery from research, an ahaha moment.
"concept elements" would have been a better descriptor in both cases. The V for the cats and the cars, and for your idea.
> 16,068 LLM ... The Coherence Model is a 42.8M-parameter transformer trained
My question is this: was this the smallest model you could train on the 16k papers? Isn't there an optimal small size to identify everything that is common?
Wouldn't it make more sense to use such a model to strip OUT the commonality? Get rid of everything where the "existing distribution of expertise and collaboration networks" are already optimal - and what's left is the novelty?
Ultimately I would be curious to see if your existing "atoms" happen to follow Zipfs law...
Goodfire is a discovery from research, an ahaha moment.
"concept elements" would have been a better descriptor in both cases. The V for the cats and the cars, and for your idea.
> 16,068 LLM ... The Coherence Model is a 42.8M-parameter transformer trained
My question is this: was this the smallest model you could train on the 16k papers? Isn't there an optimal small size to identify everything that is common?
Wouldn't it make more sense to use such a model to strip OUT the commonality? Get rid of everything where the "existing distribution of expertise and collaboration networks" are already optimal - and what's left is the novelty?
Ultimately I would be curious to see if your existing "atoms" happen to follow Zipfs law...