jmatthews
Joined 16 years ago
Joined 16 years ago
https://huggingface.co/spaces/dreddnafious/mini-agi-replicat...
At the bottom of my write up I added some ideas to pin down the actual dominant parameters. I also added a pr to fix an issue in the codebase:
PR: https://github.com/volotat/mini-AGI/pull/20
What it fixes: a crash in upstream's GradSNR meter (issue #19), a diagnostic that tracks how much of the gradient is signal versus noise.
It's a weird two-front war where people tend to answer the social question and ignore the economic one, but the economic question has not done any favors to the EU or the US for that matter
The hubris to think a group of people could precipitate the rapture is just awe inspiring and the hubris to reduce someone's relationship with the creator to some Wikipedia excerpts is a bit less awe inspiring.
I appreciate the former and am trying to filter the latter.
I know how to do it and it's all internalized. Even if I've never needed to do it.
that's the toolbox I'm trying to develop in ML. For example. I've studies LSTM and implemented one. What I didn't know was if gating was turing complete and essentially unbound. I didn't know if gates could be arbitrarily complex. Importantly I had no idea how to translate "I need a switch here" to a gate, or if a switch was even possible given the need to be differentiable for backprop.
If you think you can generate this artifact with a prompt then show me. This was 2 days of exploration and research.
It's unclear to me if you think the resource has no value or if it bother you that I wrote it using a coding agent.
I wrote the syllabus and worked through each section. Where my understanding was weak I explored the space, pulled in research, referred the model to other sources, and just generally tried to ground the topic in something I understood.
What resulted was something that helped a lot of subjects click together for me. Especially when to reach for a particular activation function and the section on gating.
This enter survey was motivated by an ML expirent I ran with assosicative memories that just failed horribly. So rather than post mortem that I set about understanding why it failed.
Anyhow, thank you for the feedback. I submitted this in good faith that it may help others.
This was a two day exploration where I provided the syllabus and ran through it with Claude Code, asking questions, trying to anchor it to stuff I understand well. I feel like the artifact has value.