This one is crazy. I made up a silly topology problem which I guessed wouldn't be in a textbook (given X create a shape with Euler characteristic X) and set it to work. Its first effort was a program that randomly generated shapes, calculated X and hoped it was right. I went and figured out a solution and gave it a clue.
Watching it "think" through the answer is surreal and something I haven't felt since watching GPT-2 write punchlines for jokes.
Here's the lightbulb moment it "realizes" what the clue meant:
> For example, if you have a covering space of degree n, then the Euler characteristic of the covering space is n times the Euler characteristic of the base space.
> But I'm not sure how that relates here.
> Alternatively, perhaps it's about considering the Euler characteristic of a quotient space.
> But again, that might be too advanced for this problem.
> Maybe I'm overcomplicating things.
> Let me go back to the beginning.
> The problem says: "Consider the Euler characteristic of the tetrahemihexahedron and of a genus g torus. Then consider the additive property of the Euler characteristic. Then try again."
> So, perhaps it's simply about adding their Euler characteristics together.
I'm under no illusions about how LLMs are built and operate, but this really feels somehow more "real" and person-y than previous generations, even when you coax them into an answer.
I'm going to go and try having GPT-4o roleplay a student solving the problem and see if it's genuinely different. I've been getting impressive answers from o1, but just coldly getting the result is much more robot than human.
Watching it "think" through the answer is surreal and something I haven't felt since watching GPT-2 write punchlines for jokes.
Here's the lightbulb moment it "realizes" what the clue meant:
> For example, if you have a covering space of degree n, then the Euler characteristic of the covering space is n times the Euler characteristic of the base space.
> But I'm not sure how that relates here.
> Alternatively, perhaps it's about considering the Euler characteristic of a quotient space.
> But again, that might be too advanced for this problem.
> Maybe I'm overcomplicating things.
> Let me go back to the beginning.
> The problem says: "Consider the Euler characteristic of the tetrahemihexahedron and of a genus g torus. Then consider the additive property of the Euler characteristic. Then try again."
> So, perhaps it's simply about adding their Euler characteristics together.
I'm under no illusions about how LLMs are built and operate, but this really feels somehow more "real" and person-y than previous generations, even when you coax them into an answer.
I'm going to go and try having GPT-4o roleplay a student solving the problem and see if it's genuinely different. I've been getting impressive answers from o1, but just coldly getting the result is much more robot than human.