MichaelNolan
Joined 3 years ago
Joined 3 years ago
I don’t remember the source but here is the first page of Harry Potter with only 75% coverage:
> 75% coverage
_______ ONE
THE BOY WHO _____
Mr . and Mrs . Dursley , of ______ four , ______ _____ , were _____ to say that they were _________ ______ , _____ you very much . They were the last people you ' d ______ to be ________ in anything _______ or __________ , because they just didn ' t ____ with ____ ________ .
Mr . Dursley was the ________ of a ____ called _________ , which made ______ . He was a big , _____ man with ______ any ____ , ________ he did have a very large ________ . Mrs . Dursley was ____ and ______ and had nearly _____ the _____ ______ of ____ , which came in very ______ as she _____ so much of her time _______ over ______ ______ , ______ on the _________ . The Dursleys had a small ___ called Dudley and in their _______ there was no _____ boy ________ .
Here is a similar source https://thepatrologist.com/what-98-comprehensibility-looks-l...
In 2010 Hinkley C was the right choice. But if starting today, that £48 billion could buy you a whole lot of wind, solar, and batteries.
Thankfully I left all that behind a long time ago. Though I just checked and they still don’t have AC.
There is a fairly famous quote: “The degree of civilization in a society can be judged by entering its prisons” - Fyodor Dostoevsky
While a lot of metrics make intuitive sense, we don’t have that much hard evidence to prove or disprove their value. Part of it is the whole “if a metric becomes a target, it ceases to be a good metric” thing. Adding the checks to a large existing project probably has negative value. But I think it’s worth doing for greenfield projects.
For humans, these should just be advisory. But for LLMs I’m happy enough to make it a blocking check.
I keep thinking of doing an experiment where I give the same LLM the same problem, and only change which metric is enforced. And then see if any of them have a noticeable effect on correctness/maintainability.
> any examples of how people get this into an actual report / CI test / benchmark / whatever ?
Yeah they have examples of adding it to CI, or local checks, generate html reports, etc in their docs.
We will hit 1TW per year of new solar soon, but to get to 100% electricity by the end of 2033 I think we would need closer to 3TW per year.