jmatthews

Joined 16 years ago

Karma
331
Submissions
174
Member since
Sep 16
software/data engineer, python, golang, c#, java, c++, in that order. Domain expert with data: hygiene, acquisition, warehousing. Generalist.
View on Hacker News

Recent submissions

Comment · 1 month ago
I built a smaller scale replication. Most of your claims prove out but the lever is essentially Biderman et als "learn less forget less". My first pass actually made this mistake initially and the findings didn't replicate. After controlling for step density I replicated your findings. Here is a write up:

https://huggingface.co/spaces/dreddnafious/mini-agi-replicat...

At the bottom of my write up I added some ideas to pin down the actual dominant parameters. I also added a pr to fix an issue in the codebase:

PR: https://github.com/volotat/mini-AGI/pull/20

What it fixes: a crash in upstream's GradSNR meter (issue #19), a diagnostic that tracks how much of the gradient is signal versus noise.

Comment · 1 month ago
This is an interesting approach. First of all, thanks for sharing your work. I've done. I want to say similar work in that I have trained continuous learning models and I have also offloaded parametric knowledge to hard drive people underestimate how difficult that is to do in a functional model. I look forward to digging in deeper.
Comment · 1 month ago
Without commenting on the EU, particularly, I would be concerned if I were a Canadian about Carney's commitment to globalism when the degenerate cases have become so common.

It's a weird two-front war where people tend to answer the social question and ignore the economic one, but the economic question has not done any favors to the EU or the US for that matter

Comment · 4 months ago
It's always weird hearing yourself described and recognizing approximately nothing, right?

The hubris to think a group of people could precipitate the rapture is just awe inspiring and the hubris to reduce someone's relationship with the creator to some Wikipedia excerpts is a bit less awe inspiring.

Comment · 5 months ago
That's great compensation. I'm glad justice was served but couldn't have happened to a bigger piece of crap.
Comment · 6 months ago
Well done
Comment · 7 months ago
I learned some of this the hard way. I did the thinking and the distillation, but I had AI write the prose and that's all a lot of people saw, the AI tell-tales.
Comment · 7 months ago
it's a worthwhile lesson. thank you. There was a great deal of effort on my part, but not in the prose. You've taught me something and I appreciate it.
Comment · 7 months ago
it's not that you're teaching the AI, it's that you're framing the conversation on a reference material and having a conversation around it. Exploring a problem with referential framing, like a white paper or a dense blog post is a useful cognitive hack. You just have to be careful to pin extraordinary claims to extraordinary evidence.
Comment · 7 months ago
I'm trying to untangle the "this content isn't valuable" signal from the luddite "Anything with AI is low effort slop"

I appreciate the former and am trying to filter the latter.

Comment · 7 months ago
I'll give a software example because I'm just better at it, but if you describe Trello and tell me to decompose it on a white board I start thinking in terms of queues and write|edit|save tuples. I don't have to invent queues in my head and I have to wonder if it is possible to proactively assign a series of tranforms or schedules on an fresh input.

I know how to do it and it's all internalized. Even if I've never needed to do it.

that's the toolbox I'm trying to develop in ML. For example. I've studies LSTM and implemented one. What I didn't know was if gating was turing complete and essentially unbound. I didn't know if gates could be arbitrarily complex. Importantly I had no idea how to translate "I need a switch here" to a gate, or if a switch was even possible given the need to be differentiable for backprop.

Comment · 7 months ago
It's not a trick bud. The github page shows my user name and Claude. The content is intended to be read by an AI agent and explored through a text interface. That is explicit in the readme and the primer itself.

If you think you can generate this artifact with a prompt then show me. This was 2 days of exploration and research.

Comment · 7 months ago
my apologies it wasn't up to your standards. In fairness to me, that line is exactly what my effort was. I wasn't trying to "learn ML" I am trying to build a mental model that let's me decompose real problem into ML primitives.

It's unclear to me if you think the resource has no value or if it bother you that I wrote it using a coding agent.

I wrote the syllabus and worked through each section. Where my understanding was weak I explored the space, pulled in research, referred the model to other sources, and just generally tried to ground the topic in something I understood.

What resulted was something that helped a lot of subjects click together for me. Especially when to reach for a particular activation function and the section on gating.

This enter survey was motivated by an ML expirent I ran with assosicative memories that just failed horribly. So rather than post mortem that I set about understanding why it failed.

Anyhow, thank you for the feedback. I submitted this in good faith that it may help others.

Comment · 7 months ago
I've bounced off of many good textbooks. Even Karpathy's YouTube series was too dense for me. I'm trying to come in at a more palatable level.

This was a two day exploration where I provided the syllabus and ran through it with Claude Code, asking questions, trying to anchor it to stuff I understand well. I feel like the artifact has value.

Comment · 7 months ago

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton