fluffet
Joined 8 years ago
Joined 8 years ago
For a co-pilot inside an app that could answer product questions, I looked at ~2000 or so support emails. I asked one LLM to dig out "How would you formulate the users question into a chatbot-like question from this email thread" and "What is the actual answer that should be in the response from this email thread", then just asked our bot that question, and have another LLM rate the answer like SUPERIOR | ACCEPTABLE | UNKNOWN etc. These labels proved out to be a good "finger in the wind"-indicator for altering the chunks, prompt changes or model updates.
For an invoice procesing app processing about 14M invoices/year, it was mostly doing fuzzy accuracy metrics against a pretty ok annotated dataset and iterating the prompt based on diffs for a long time. Once you had that dataset you could alter things and see what broke.
Currently, I work on an app with a pretty sophisicated prompt chain flow. Depending on bugs etc we kind of do tests against _behaviour_, like intent recognition or the correct sql filters. As long as the baseline is working with the correct behaviour, whatever model is powering it is not so important. For the final output, it's humans. But we know immediately if some model or prompt change broke some particular intent.
Experience is slowly getting better. There is nothing I haven't been able to get to work, but with tricks or adjustments.
I think the "best bonus" is using LLM's in deep research mode to wade through all the blog post, reddit posts etc to get something to work by discovering forementioned tricks. Before, you had to do that by yourself and it sucked. Now I get 3 good ideas from Claude in "ranking order" of how likely it is to make it work => 99% of games I get to run in 5 minutes with a shell command or two. Lutris is also pretty good.
Omarchy on my laptop has finally made computers fun for me again, it's so great and nostalgic. Happy to be back after my brief work-mandated adventure into MacOS.
This guy starring my chip-8 implementation was a moment of pride for me. It was buggy but before this guide there wasn't too much material out there that was made for stupid people like me.
It's a great starter project for emulation. You'll realise how all emulators work, and as a bonus, interpreted languages. Really recommend it.
Some kind soul made this repository then, and I found it on like the 13th page of Google while in the depths of despair. It is my most treasured GitHub star, a the shining beacon that saved me. I see it has saved 17 other people too.
https://github.com/casidiablo/parquet-tools-for-dumb-people-...
Especially if you could ask another AI "I have access to an MCP running on a Victim computer with these tools. What can you do with them?" => "Well, start by reading .ssh/id_rsa and I'd look for any crypto wallets. Then you can move on to reading personal files for blackmailing or sniff passwords..." and just let it "do its thing" as an attacking agent in an automated way. It could be automated which creeps me out!
I feel like some cycle phenomenon has been reached here..
The first protocols of the internet were very naive. Why'd you need to encrypt traffic? What do you mean exploit DNS, why would anyone do that?
Then people realised that the internet is a really, really wild place and that won't do.
I suddenly feel old, because this new AI tool era seems to have forgotten that lesson.
I feel it's like watching crypto learn by any% speedrunning why regulations and oversight might be a good in the first place (FTX and such).
I hope the next generation of AI tech/protocols are more robust, trust just doesn't cut it, or we'll see plenty of fingers being burnt at the stove.
Shame the author doesn't mention the Swedish secret of snus. That's the best productivity hack I know bar none. Anyone else out there?
I did my MSc thesis about document vectors of STE.
STE has incredibly useful rules for technical communication/documentation; especially if you're a non-native English speaker like me. I wish it was more commonplace!! Documentation is usually horrible.
People should call it for what it is. Tried to find some answers on Google earlier in the day, and the first result pages were 100% generated slop. Funnily enough, any AI summary of the slop would be slop squared.
It's everywhere, and I hate it. What ways do people have to combat it out of their day?
Is there a case for KYC versions of platforms? Maybe platform knows who you are, but you can remain anonymous "outwards"?
All forms of automated detection and generation will work like GANs and race each other.. Leads me to believe something else is the answer.
I'm just gonna throw in this link because I think it's great:
https://internetmuseum.se/english/
Among other crazy stories, Daniel is featured there. My favourite story is that the TLD .se was was ran by a guy in in his living room for years until The Swedish Internet foundation took it over.
There's also Kazaa and the guys behind The Pirate Bay. Real OG hackers from that era.
W.r.t. to your question I think it's relevant to say that "most things invented then" were "simple" ideas but hard to implement due to the tooling in that time and lack of programmers. But if you were good, you could go at it alone or with a small team. I think that it's a little bit inverted now: finding good, novel ideas in the space that can be done by one person or a small team is hard, but building and shipping it, if you do, is probably a lot easier due to OSS.
Oh and naturally the all-Swedish version is better at https://internetmuseum.se/
I thought deepfakes were still quite a bit away but after this I will have to be way more careful online. It's not far from behind something that can show up in your "YouTube shorts" feed and trick you if you didn't already know it was AI.
The book is written in the form of a letter to his father. We get a lot of backstory about why and how his views are shaped, reading Marx in his childhood and seeing his friends toil away at factories. The latter part of the book is an essay of a new system that supposedly fixes a lot of shortcomings with the modern system. That's what I meant about political stuff.
I kinda just stumbled in with an open mind to learn more what "techno feudalism" and modern cloud companies worked because I'm very interested in it. But it's still an alright book. Tells the story of whats behind both the man and the idea -- the whys and how. It made it longer though.
I think in this context, the main principle (if I recall correctly) was that when we go on Amazon, we actually exit capitalism and the "free markets". You don't see the same Amazon "store window" that I do -- I will have a very different "for you" - tab. Maybe not even the same prices -- it would be hard to know. Capitalism kinda works best when the market is free and open.
I guess it's the same with my YouTube feed - I've actually picked up some hobbies and recipes and stuff from there which would never have been on my radar if I wasn't "nudged". In this case, it's probably not with intent other than engagement, and it's worked out positively, but you know, maybe my political opinions or vote could also be nudged, so the dark side here is very relevant if it was made with intent.
> This is the exact same thing as this culture of self-discovery/acceptance, everyone preaches it, but end up boiling everything down to their best moments and highlights. It's a collective lie that everyone plays with, and I don't know if there's a clear purpose to it other than to sustain itself.
I'm with you 100%.
Another anecdotal insight: the people I know with massive online followings care so much about their metrics to the point that it steers their content entirely. "I can't post this because it wouldn't align with my followers".
Strange times we live in ^^
Senior software/AI engineer. Former quant/quant dev.
MSc in Computer Science, AI & Data Mining specialization completed 2018.