gnunez
Joined 12 years ago
Karma
52
Submissions
44
Member since
Jun 29
Recent submissions
Comment · 2 months ago
Comment · 2 months ago
I could, but then I would miss the chance to interact with such lovely people as yourself.
Comment · 2 months ago
I totally missed the memo on batching. That changes everything. Thank you for the info.
Comment · 2 months ago
Ok. I see. I guess people are not referring to the raw models themselves when they say non-deterministic, but are also including the harness used in conjunction with the model. Then, in that case, for the exact same input you could get a non-deterministic output. But the model itself and all the mathematical machinery around the model is still very much deterministic.
I guess if we really needed to, we could construct a deterministic agent harness. But in most use cases we probably want some chaotic behavior to increase our chances of stumbling on the desired results.
Thank you for the clarification
Comment · 2 months ago
I don’t understand why people are calling these transformers models non-deterministic? Are you referring to the temperature parameter? I haven’t played with transformer internals in a while but my understanding is that if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something?
Comment · 4 months ago
I remember those days. They released smaller versions of the model that were interesting but pretty useless. The wild part is, about 5 years later I was able to train the same model from scratch, using a couple of gpus in my home office, following Kaparthy’s online lectures, for a fraction of the cost. Long Live Open Source!
Comment · 6 months ago
I agree. I like using Antigravity for some of my frontend work, and I find it does a better job than Claude Code - Opus 4.6. I’ve also found the Gemini Flash models to be good at legal defense research—I use them to help New Yorkers fight parking tickets (https://nyceasyparking.com). That said, the Claude models are still amazing at agentic work.
Comment · 1 year ago
I’m working on yet another cloud based coding agent https://seniordev.io/ that connects to an existing GitHub repo, spins up a feature branch, commits incremental changes, and opens a PR. You can jump into an embedded VS Code server to review and tweak the code before merging—no local setup needed. Any feedback is greatly appreciated Thanks!
Comment · 1 year ago
Great work! I love your videos; they've taught me so much. Any plans for a Mixture of Experts (MoE) video? My understanding is that starting from GPT4 most advance models use MoE to some extent. For example, can I take the model from your GPT2 video and just change the feed forward layer to an MoE layer like the one found here (1)? I guess I can just try it myself but I enjoy the expert guidance you provide in your videos. Please don't stop! great content!
1. https://github.com/mistralai/mistral-inference/blob/main/src...
Comment · 2 years ago
The 555 timer is considered one of the most successful chip designs in electronics. It has a simple, well understood, design. The timing on the 555 can be configured to generate a periodic signal using just a few resistors and capacitors. The 555 is often used as a timer in electronic circuits. The Linux scheduler is usually configured to respond to interrupts generated by a cpu. To replace those cpu interrupts by a simple 555 timer is impressive due to the unconventional nature of the setup.
Comment · 2 years ago
Thanks for your helpful comment! I was also struggling with the bad circuits voltage divider.
Comment · 3 years ago
Hi apologies for any confusion. What i meant to say is that OpenWrt can be run on the Raspberry Pi. I did this because I had a Raspberry Pi laying around and did not want to invest in additional hardware just for experimenting with OpenWrt within my internal network. You can also run OpenWrt in a virtual environment like QEMU -- this is useful if you're aiming to connect multiple VMs to OpenWrt, effectively setting up a virtual network with OpenWrt at its core. Please see links below for more info:
https://firmware-selector.openwrt.org/?version=21.02.3&targe... https://openwrt.org/toh/raspberry_pi_foundation/raspberry_pi https://openwrt.org/docs/guide-user/virtualization/qemu
Comment · 3 years ago
it also works really nice with the raspberry pi 4 and QEMU.
Comment · 3 years ago
It could be that the corpus ChatGPT was trained on is full of ‘confidently wrong’ answers from these ‘experts’. One solution could be to train these LLM on a higher quality corpus from real experts instead of random text from the internet. But would that just bring us back to the days of expert systems?
Comment · 4 years ago
Wow this looks amazing! I can’t wait to get my copy. I wonder with all the available resources to learn electronics can someone become an electronics engineer without going to college (assuming enough mathematical maturity to understand Electrodynamics, etc)?
But if we are seeing problems with agents replacing knowledge workers at the coding layer, why assume the executive layer is uniquely immune?
There was an interesting article in the Harvard Business Review about this exact problem. Researchers found that when LLMs are asked for strategic business advice, they tend to converge toward what is popular and trendy rather than necessarily what is best for the particular situation. They called this "trendslop," and described the response as,
"On strategy, LLMs might be more akin to a freshly minted MBA or junior consultant, parroting what's popular rather than what's right for a particular situation."
I really hope this idea works but just thought I'd share some of the research out there. https://archive.is/OPObK/again?url=https://hbr.org/2026/03/r...