cheekygeeky

I had one other thought: The LABELs given to the models assume that the models behind the name never change (GPT 6.1 Sol, for instance). I doubt that is true. I just don't think that they update the number behind the name until they have a feature or benchmark that they can report is significantly better. Thus, I think it would useful to reset the model scores to the starting score each day and TRACK/CHART the ELOs of the models over consecutive days. That would test my theory, if nothing else. Does a model "get dumber" as some people feel/report sometimes? Does a model get incrementally "smarter"?

Thanks for the reply to my earlier question! If I'm understanding your comment, you started them off at different arbitrary scores? Wouldn't it have been fairer to start them off at the same scores (assuming nothing about relative strength) and let the results move the scores up or down from that?

Daniel, GREAT job. This is fascinating. Your leaderboard is based upon ELO, correct? I can't find anywhere that you describe the formula for calculating the ELO score. It seems like we would need to verify that is indeed an accurate way to judge the Model Leaderboard. Care to share? Also, if you gave them DIFFERENT arbitrary ratings to start, that would definitely influence their calculated ELO's later, wouldn't it? Why not start them off with even ELOs (like all starting at 1500)?

U.S.-centric reply here. Word mean things. So I think it would we wise to separate the terms "school" and "education". It has long been accepted that school is where one receives (or obtains) an education. However, schools have also introduced roadblocks to education. For example, students are often reticent to raise their hand to participate or ask questions because they are afraid it will make them look dumb or uncool ("brown-noser", etc.). I think we would all agree that both vocalization and question-asking are necessary to accelerate learning/education. So schools have LONG been non-optimal for real education and this is just one of the reasons that more and more parents have opted for homeschooling. And that is before we get into other issues like teacher to class size ratios, safety, bullying, sexual harassment and other things that come with today's schools.

The biggest problem with homeschooling has been unqualified teachers/parents, both in their ability to provide proper oversight of learning and lacking the qualifications of knowledge in the areas they are supposed to be teaching.

If you believe the maxim that "Education is not about filling a bucket, but rather about starting a fire" for learning, then self-directed learning with Ai could be the biggest boon for education POSSIBLE. All one must do is inculcate curiosity in the student and teach them how to think critically and evaluate sources and then set them loose on answering the questions that naturally come from that curiosity (or are supplemented by more questions to lead them farther down the road to knowledge or expertise on that subject).

An argument could be made that we are entering a post-school period of education. Except that the Public School system in the U.S. has become the biggest taxpayer-funded daycare center possible for school-aged children.

And that's why few really care about what happens in a school. They are a necessary evil and there is a belief that they promote social skills and a chance for athletics, if nothing else.

Suggestion: Save the article as an .md file. Upload it to Fable 5 with the prompt: "Agree or disagree. Be verbose." I learned a lot: Where he's right. Where he's wrong. A 'delicious bug in his own example" code (command injection vulnerability - in the code sample used to demonstrate why you don't need sandbox.git.clone). I also learned that, according to Fable 5: "A Typescript SDK is the strongest anti-hallucination device we currently have." And a lot more.

Didn't Ai make cloning ANYTHING easier?

They do. See: Claude Code leak.