Did I use AI to help write this article? Absolutely. I use AI 10-14 hours day, seven days a week to do all sorts of things, including trying to get a code generation and verification app off the ground. It would be unnatural for me not to use AI to research and help think through this article.
Do I know how to use Pangram, too? Absolutely. I got my Pangram merit badge months ago.
So, why didn't I make sure I got a 100% human-written score on Pangram before putting this out there?
I spent weeks reading what others had to say on these topics--Lilian Weng, Addy Osmani, Han Lee, Simon Willison, and a number of arxiv papers. I felt I had some things to add to the conversation. A slight twist here and there, maybe some more detailed information about how to implement these ideas than I'd seen elsewhere. For example, I see people writing about the model eating the harness, but very few people countering with "but AI failure modes".
I feel like there are a lot of holes and gaps in what I've been able to piece together from others' writings, largely because each article is only covering a portion of the entire process. The reader is left to fill in the blanks and connect the processes.
I have firsthand experience filling those gaps and holes I wanted to share. I felt this needed to be a single article, against the advice of multiple AI writing partners, because that's where the devil lives, in the details. Without a single piece covering the process from end-to-end, the reader is left to fill in the gaps themselves by triangulating multiple excellent articles. I've done that work. I wanted to share what I've found with others to try to start meaningful conversations about best practices for code generation, code verification, and learning loops that can optimize the entire process.
So, call it slop because Pangram told you you could. But I spent ~20 hours just researching and writing the article, not including the 100's of hours I spent writing code generation and verification pipelines.
I remain eager to engage with anyone on the contents of the article.
Brad
Do I know how to use Pangram, too? Absolutely. I got my Pangram merit badge months ago.
So, why didn't I make sure I got a 100% human-written score on Pangram before putting this out there?
I spent weeks reading what others had to say on these topics--Lilian Weng, Addy Osmani, Han Lee, Simon Willison, and a number of arxiv papers. I felt I had some things to add to the conversation. A slight twist here and there, maybe some more detailed information about how to implement these ideas than I'd seen elsewhere. For example, I see people writing about the model eating the harness, but very few people countering with "but AI failure modes".
I feel like there are a lot of holes and gaps in what I've been able to piece together from others' writings, largely because each article is only covering a portion of the entire process. The reader is left to fill in the blanks and connect the processes.
I have firsthand experience filling those gaps and holes I wanted to share. I felt this needed to be a single article, against the advice of multiple AI writing partners, because that's where the devil lives, in the details. Without a single piece covering the process from end-to-end, the reader is left to fill in the gaps themselves by triangulating multiple excellent articles. I've done that work. I wanted to share what I've found with others to try to start meaningful conversations about best practices for code generation, code verification, and learning loops that can optimize the entire process.
So, call it slop because Pangram told you you could. But I spent ~20 hours just researching and writing the article, not including the 100's of hours I spent writing code generation and verification pipelines.
I remain eager to engage with anyone on the contents of the article.
Brad