mikehollinger
Joined 11 years ago
Joined 11 years ago
Yes. There's a difference between scrapping a session and starting over, or going back and branching something, or using sub-agents to see five outcomes, vs arguing with a system in a long drawn out chat.
Like - I know that if a model starts doing something silly, instead of correcting it - I can probably go back and edit two steps prior to add an extra guardrail, or extra data, or whatever.
I'd add the Zones of Thought series by Verner Vinge. [1]
My naive view was that there was some sort of “normalization” or “pattern matching” that was happening. Like - you can look at a trend line that generally has some shape, and notice when something changes or there’s a discontinuity. That’s a very simplistic view - but - I assumed that stuff was trying to do regressions and notice when something was out of a statistical norm like k-means analysis. Which works, sort of, but is difficult to generalize.
For example some former colleagues timeseries foundation model (Granite TS) which was doing pretty well when we were experimenting with it. [1]
An aha moment for me was realizing that the way you can think of anomaly models working is that they’re effectively forecasting the next N steps, and then noticing when the actual measured values are “different enough” from the expected. This is simple to draw on a whiteboard for one signal but when it’s multi variate, pretty neat that it works.
[1] https://huggingface.co/ibm-granite/granite-timeseries-ttm-r1
and
> a key thing with LLMs is that their ability to help, as a tool, changes vastly based on your communication ability.
I still hold that the innovations we've seen as an industry with text transfer to the data from other domains. And there's an odd misbehavior with people that I've now seen play out twice -- back in 2017 with vision models (please don't shove a picture of a spectrogram into an object detector), and today. People are trying to coerce text models to do stuff with data series, or (again!) pictures of charts, rather than paying attention to timeseries foundation models which directly can work on the data.[1]
Further, the tricks we're seeing with encoder / decoder pipelines should work for other domains. And we're not yet recognizing that as an industry. For example, whisper or the emerging video models are getting there, but think about multi-spectral satellite data, fraud detection (a type graph problem).
There's lots of value to unlock from coding models. They're just text models. So what if you were to shove an abstract syntax tree in as the data representation, or the intermediate code from LLVM or a JVM or whatever runtime and interact with that?
[1] https://huggingface.co/ibm-granite/granite-timeseries-ttm-r1 - shout-out to some former colleagues!
I've told it before, but when we were doing some clean sheet work a while ago I decided to use the C4 model and drew out the obligatory "Context" diagram with "user" "phone" "laptop" "app" sort of stuff.
I found them silly and (honestly) I still find that if I see one "in the wild" with no further elaboration I become suspect.
However two hours later, because of that silly context diagram, I realized that we had both an online and a semi-disconnected mobile app that could be offline for hours, and that certain things -had- to use a queue and expect an arbitrary amount of time for a task to run, and it completely changed how we thought about the core of how we implemented something pretty important.
Sold. :-)
But if you want to talk about REAL complex systems talk to a microprocessor logic owner or architect trying to shoot a bug.
A while ago we found a bug that could crash a system (fixed in a new RIT of the chip) if we did X then Y in state … we didn’t know.
Listening to the various leads for the sub-units on a phone call trying to reason about what was happening I found myself visualizing this increasingly complicated steam powered machine, with parts sprawling, tiny gears whirring, and bits zipping about whenever X happened.
It was humbling.
The ultimate compression is to reduce the video clip to a latent space vector representation to be rendered on device. :)
Just give us a few more revs of Moore’s law for that to be reasonable.
edit: found a patent… https://patents.google.com/patent/US11388416B2/en
Looking at it 16 years later, and… what is this nonsense? It’s so customizable that it’s loaded with footguns.
There are quite a few memorable words you can spell using 32 or 64 bits—like BA5EBA11. This is the story of me -not- choosing one of those.
These bit-pattern words are handy because they’re easy to recognize, especially in a random memory dump.
On my first “real” assignment, I was writing real-time embedded C code for a 16-bit processor that communicated with a host microprocessor on a server. We needed to run periodic assurance tests across a bus to ensure reliable communication with the host since we weren't constantly using the bus.*
We were given an unused register address on the host processor and told to write whatever we wanted to it. The idea was to periodically write a value, read it back, and if we encountered any write errors, incorrect reads, or failures, we’d declare a comm error and degrade the system in a controlled manner.
Instead of using zeros or something like 0xDEADBEEF, I decided to write 0x4D494B45 - "MIKE" in ASCII. It was unique, unlikely to be tampered with, it worked, and no one argued with me. The code shipped, the product shipped, and all was well. We even detected legitimate hardware errors, which I thought was pretty cool.
Fast forward two generations of systems, and long after I’d moved on from that team, the code had been ported around but that assurance test remained unchanged. Everything was fine until they brought up a new generation of systems, flipped on the firmware for that device, and 10 seconds later, my assurance test clobbered an important register. The entire system promptly checkstopped and crashed. It took the team days to figure out what was wrong, and I had to explain myself when they found "MIKE" staring back at them from the memory dump.
That was a fun project. ;-)
* Note: It would've been bad if our device went out to lunch because we were responsible for energy management of the server. If the power budget was exceeded and we couldn't downclock and downvolt the processor, something might have crashed or been damaged.
Lesson learned: we built a 42-node several petabyte storage cluster. Worked fine as we were starting small, and got bigger, and bigger. When we “had it” and disconnected from our internal network and moved to an isolated network we discovered quickly that the timebase on all the systems drifted very quickly to be seconds and then minutes out of sync after several days. This was because they couldn’t access NTP.
Set up an ntp server and “resolved” it, except now we had an entire cluster of hardware that was (in lock step) drifting out of sync with the rest of the world. Fixed that. Moved on.
Another example: we bought a bunch of really cheap LED ropes to make our LGBT+ pride float’s backlighting for a pride parade several years ago. I wired it. The ropes were all independent and we planned to have them rotate colors. The little controller boxes had a mode that would smoothly transition through presets. It turned out that simply turning on the power would cause the lights to start in sync, and then quickly drift out of sync in a really mesmerizing way. They’d occasionally come back together but frankly it was better than anything we could’ve programmed.
We also learned we could control the rate of the “smear” by over or under volting the LEDs a bit.
Even if it’s digital, everything is eventually analog. And analog is weird.
Find me at https://www.mikehollinger.com