nestorD
Joined 8 years ago
Joined 8 years ago
One in particular, about sharing secrets, that had not crossed my mind (I have found that virtually no one ticks the "do not train on my data" box...):
> Similarly, it seems that mathematicians already concerned and guarded about what they write in research statements and research proposals. After all, an administrator or a reviewer may upload those documents into an LLM to get help in writing a report, thus potentially compromising the confidentiality of unpublished research.
I went through a book similar to the one above, with no internet connection. The first few weeks were rough: I did not quite know what a type was and the compiler error messages were unforgiving and hard to understand without that context. But, once I grokked the core ideas (a proper idea of what could be done with recursion took much longer), things went surprisingly smooth. I definitely credit it with making me a better programmer.
I... disagree. The joy I get from learning a programming language does not come from knowing that it is useful. Rather, it comes from engaging with its abstractions and figuring out the associated mental puzzles.
On a related note, I recently realized that I am barely writing code these days (instead I am reviewing a lot of LLM-written code and nudging it in the right direction) which will make doing the advent of code (which I usually use to play with a language I have never used) without any LLM help even more enjoyable: it will be both the fun of figuring out those puzzles in a new language, and stretching my code-writing muscles.
Plus it looks like they have support for dynamic shapes: https://tensor4all.org/tenferro-rs/design/dynamic-symbolic-s...
But, in practice, when asking a model to pick the best answer they see a single question / answers pair and focus on determining what they think is best.
In short: start with a dataset of question and answer pairs, where each question has been answered by two different LLMs. Ask the model you want to evaluate to choose the better answer for each pair. Then measure how consistently it selects winners. Does it reliably favor some models over the questions, or does it behave close to randomly? This consistency is a strong proxy for the model’s intelligence.
It is not subject to dataset leaks, lets you measure intelligence in many fields where you might not have golden answers, and converges pretty fast making it really cheap to measure.
Specialized in artificial intelligence, high-performance computing, and floating point arithmetic.
Nestor Demeure (https://nestordemeure.github.io/about/)