ianbicking

I looked at Khanmigo fairly early on. I tried it out and thought: this is ChatGPT with a system prompt. And not a good system prompt either. In addition to what felt like rote math it had some had some experiences, like you could "talk" to Plato or some other figure. Each of these was also pretty awful; simplistic prompts that spent all their time on guardrails and none on the experience itself. (What it the student thinks they are ACTUALLY talking to Plato?! Avoiding this seemed very important to the makers.)

In the more core experience they were super focused on not giving kids answers. I suppose built on a fear of subverting the teachers or classroom environment. But again it became a fixation that took all attention away from teaching itself.

But I figure you put something out in the world and see what happens. Yet I came back to it a year later and it was exactly the same. Nothing had changed. It baffled me then, and still baffles me... I had done enough to know the whole experience was easy to manipulate and yet I couldn't see any evidence of effort. It felt very much like "we tried nothing and it didn't work". Maybe I was missing something, I don't know.

I think there's a few different problems here:

1. I think the author is correct to highlight that underlying motivation, which is so important to teaching. We can focus on a high-minded Constructivist ideal of intrinsic motivation, but motivating students is also what grades are for, and classrooms, and learning with peers, having someone pay attention to your work, and so on. I do think that the AI chatbot is not capable of providing that. But the AI can still ENGAGE with that motivation. I have a feeling Khanmigo never did. I suspect some of their privacy controls also kept it from ever creating a good model of the learner.

2. The entire experience was very focused on answers, on solving problems. And yet it focused on that so reluctantly. The obsession with not giving away the answers had the unfortunate result in not engaging with anything but answers. It felt like it was always teasing at something it would withhold. But questions are cheap, especially rote questions, they shouldn't be treated as precious.

3. This may be unfair, but I infer the makers of Khanmigo did not build it with love and craft, which is unfortunate. I don't know enough to say why. Though there's a weird confidence to the product and to all of Khan Academy that I feel does not bring the necessary humility.

4. The chat experience was asymmetric in the wrong direction. In each turn the AI can and will spew out paragraphs of text, and the student types a couple words. Chatbots DO NOT have to work like this, they operate wonderfully with large chunks of user input. But you have to have the modality to receive it, the openness of experience to elicit that response, and the respect to make use of that response. Speech can be good, but it's a real challenge in a classroom environment (though if you are investing in Khanmigo, you can also invest in the other hardware to make it possible).

5. In some fairness, individual tutors don't represent any pedagogy at all. It's usually just someone making it up as they go. Motivation is a huge aspect, studying with someone sitting next to you, watching you study, will do wonders for keeping you focused! But also a tutor will have a natural theory of mind, hopefully the ability to see a student get the wrong answer and get inside the student's head to model their understanding and find a path to a correct understanding. That's a very challenging task, not beyond GPT-4 entirely but still quite challenging, especially if you are optimizing cost or time to response.

6. A good experience will do a TON of prework. I don't know if Khanmigo does that, if they've reshaped and expanded each educational module into a thorough playbook for the AI. That's where you have a chance to model things like wrong answers (there are actually SEVERAL books enumerating wrong answers in math), and use that to plan out responses.

If Khanmigo is not successful, I think that's on Khanmigo. Still teaching is very challenging. And maybe all of Khan Academy fails in this way: it does not seem to acknowledge the challenge of what they're trying to attempt. They seem far too confident in what they do, and not nearly creative enough.

Looking at the OpenAI/Hugging Face incident and the difference in what "persistent" models do, it seems reasonable. Like: is this a solvable problem? How much work does the model think is intended to solve this problem? Each input raises the expectation.

And then finally both model output and human input become one world frame for the model, and the human adding a "you can do it!" isn't just input but a frame that colors not just the next step for the model, but also all previous steps (since at each step the model is viewing the totality of the transcript).

That this makes sense only makes it all the more absurd

I remember in San José and maybe other Costa Rican cities with a street number system where even/odd were used for the cardinal directions. Calles (streets) run N/S, Avenidas (avenues) run E/W, and odd is east or north. So Calle 8 Avenida 7 is in the northwest. I also remember finding it helpful in theory but not very helpful in practice since the rest of the addressing wasn't normalized. I also recall more than one Latin American city where they ran the numbers up one-by-one, and this didn't scale over the decades or centuries, so they renumbered... just meaning that every house had two numbers, and people used both.

Maintaining human understanding in the face of fast progress is a big unsolved problem. If you are worried about there being nothing left when LLMs tear their way through a topic then the presence of a big unsolved problem should be comforting!

And it seems legitimately like an important problem for mathematics (and other areas), one which is deeply academic in the best sense: mathematicians spend much more time transmitting understanding than doing novel math, because they mostly exist in academia and that’s the job.

When I was a kid in the 80s I noticed a lot of hackers of that era that typed like that. I thought it was strange at the time, but not at all uncommon

I've been doing something similar to this in a personal claude code frontend, though not particularly "magical".

I'm mostly using my system to make comments on long AI-generated documents (especially design documents). I find it works well to have the AI generate something, and then I read through it, making comments along the way.

You can get pretty far just repeating the things you see... "I'm reading [heading] and [comments]". But I do find some use in selecting content and saying "I don't agree with this" or whatever else.

The result is just an augmented message. It looks like:

    <transcript>
      Let's see what we've got here.
      <selection doc="proposal.md" location="paragraph 3">
        The system already...
      </selection>
      No, I don't like how this is approaching the problem, ...
    </transcript>

Then I just send this as a user message. Claude Code (and I'm guessing any of the agentic systems) picks up on the markup very easily. It also helps to label it as a transcript, as it can understand there may be errors, and things like spelling and punctuation are inferred not deliberate. (Some additional instruction is necessary to help it understand, for example, that it should look for homophones that might make more sense in context.)

It makes reviewing feel pretty relaxed and natural. I've played around with similar note taking systems, which I think could be great for studying in school, but haven't had the focus on that particular problem to take it very far.

But I think the best thing really is giving the agent a richer understanding of what the user is experiencing and doing and just creating a rich representation of that. The keywords can be useful, but almost only as checkpoints: a keyword can identify the moment to take the transcript and package it up and deliver it.

One difference perhaps in design motivation: I have really embraced long latency interactions. I use ChatGPT with extended thinking by default, and just suck it up when the answer didn't really require thinking. I deliver 10 points of feedback at once instead of little by little. (Often halfway through I explicitly contradict myself, because I'm thinking out loud and my ideas are developing.) I just don't stress out about latency or feedback, and so low-latency but lower-intelligence interactions don't do it for me (such as ChatGPT's advanced voice mode, or probably Thinking Machine's work). I think this focus is in part a value statement: I'm trying to do higher quality work, not faster work.

In a sense they do use their own language; they program in tokenized source, not ASCII source. And maybe that's just a form of syntactic sugar, like replacing >= with ≥ but x100. Or... maybe it's more than that? The tokenization and the models coevolve, from my understanding.

If we do enough passes of synthetic or goal-based training of source code generation, where the models are trained to successfully implement things instead of imitating success, then we may see new programming paradigms emerge that were not present in any training data. The "new language" would probably not be a programming language (because we train on generating source FOR a language, not giving it the freedom to generate languages), but could be new patterns within languages.

The knowledge machine question is fascinating ("Imagine you had access to a machine embodying all the collective knowledge of your ancestors. What would you ask it?") – it truly does not know about computers, has no concept of its own substrate. But a knowledge machine is still comprehensible to it.

It makes me think of the Book Of Ember, the possibility of chopping things out very deliberately. Maybe creating something that could wonder at its own existence, discovering well beyond what it could know. And then of course forgetting it immediately, which is also a well-worn trope in speculative fiction.

I haven't really kept up with what Midjourney has been doing the past year or two. While I liked the stylistic aspects of Midjourney, being able to use image examples to maintain stylistic consistency and character consistency is SO useful for creating any meaningful output. Have they done anything in that respect?

That is, it's nice to make a pretty stand-alone image, but without tools to maintain consistency and place them in context you can't make a project that is more than just one image, or one video, or a scattered and disconnected sequence of pieces.

I've been working on a LLM fiction writing workflow and associated tools. It's built on agentic coding tools with lots of structure, guidance, prompting, and critique. Almost all of the flow is on the filesystem and using a custom command-line tool, making it accessible to agentic programming tools. (No MCP though; it seems superfluous?)

I was fairly neutral about the tool for a while, but lately I've been going all-in on Claude Code, using things like rules and subagents.

It's also built to "rerender" the story, for instance rewriting it (slightly) for voice, translate it, or target different reading levels or background. I'm interested in translating stories for language learners in addition to simply translating into other native languages.

I'm also hoping to create some stories that stretch the medium. Perhaps CYOA (though I'm struggling with understanding what a CYOA is good at), though also other multi-perspective stories with reader autonomy in how to read through the story. LLMs make it easier to overproduce content, so you can give the reader flexibility without feeling regret that much of the content will be skipped, or rewrite passages for readers who jump into stories part way through.

Producing quality content is hard, and frankly kind of expensive, which is why I'm focused on finished products instead of interactive experiences. Though I do look forward to some future opportunity to take these rich characters that are grounded in full stories and find other things to do with them.

I've come around to feeling that if I'm going to make an experimental development tool, I need to make it in service of building something specific. Maybe something playful... if I'm building something "important" then it can put unwanted conservative pressure on the tool. But something, and if I do that then at least I have something interesting regardless of the fate of the development tool. Because yeah, there's a good chance no one else is going to be excited about the tool, so I have to build for my own sense of excitement, be my own most enthusiastic user.

It does however work just fine if you ask it for grammar help or whatever, then apply those edits. And for pretty much the rest of the content too: if you have the AI generate feedback, ideas, edits, etc., and then apply them yourself to the text, the result avoids these pitfalls and the author is doing the work that the reader expects and deserves.

I think there might be a pattern across education with a strong ideology (Montessori, Waldorf, Classical education, etc) that they aren't very good at recognizing when the ideology is failing a kid. The relatively weak and mushy educational philosophy of a normal public school is also a somewhat reasonable way to run a school that has to take kids wherever they are at and wherever they came from.