josefcub
Joined 4 years ago
Joined 4 years ago
20-40 minutes is 2 hours if you're like me and want to make sure you're actually thorough answering questions. Aisa (the agent) was great at asking pertinent questions and keeping the conversation somehow both on-topic and on-going. It was not nearly as laborious as I thought it would be, despite how long it took.
I scored 89. I'm still in the top 30 on the leaderboard. I have no idea how I scored that with as little experience at work as I have with AI. At the time AISA first debuted here, we hadn't even gotten access to agentic models at my employer, just chatbots. I learned literally everything at home with local models.
I'm not going to argue a high score, but that feeling of disbelief is still real. I think this is a great product for evaluating your personal skills according to a fairly broad but well-defined rubric. The fact that you get to read the analysis itself and why you were graded as you were is great for self-improvement.
If I had an ask, it would be to actually download that analysis and the original conversation log. It was comprehensive enough that I'd love to turn it into actionable steps for improvement without having to copy-paste so extensively.
Thanks for making an evaluation system that works so well!
Windows has vendor support by the sheer mass of install base; vendors write their own drivers for Windows.
If you created a new OS tomorrow, where do the drivers come from? If you support too little, then nobody will use the OS.
On top of all that is application support. You need a reason for anyone to want to switch to a different OS, and application developers already need to be there.
Linux (and the unrelated BSDs) has several decades of volunteers writing the apps, subsystems, and drivers based on their own interests. This is one of the reason it works so well in the places it's most used: someone's already done the work for themselves and gave it to us for our use too.
An OS startup has none of those.
/I pine for OS/2 //and BeOS
Another thing to consider is that most people would likely rent it for the unified memory and AI usage. Speaking from experience, your M3 Ultra will run _hot_ while performing; you'll need to plan a good, cool environment that the machine lives in, and set the fans to always run via something like MacsFanControl or some other utility, since it won't engage the fan at a higher rate just because your GPU is pegged and generating all the heat.
The third thing is knowing how to balance out its memory usage. SSDs have a finite lifetime in terms of writes. Macs are _very_ good at not wearing out SSDs, but if your renter botches something in a config and you spend days or weeks swapping in and out of virtual memory, your machine will wear faster and harder.
When I repeated the experiment with a MUD that I'd built by hand (A small American town) for the LLM's own limitations (Descriptions referenced things that I made sure existed, more common verbs existed for it to use on things, there was a map facility, and at least me to interact with on a second connection), I found the agent much more likely to take its time exploring, making up its own goals, and spending time traveling in the space just communicating with me in a roleplaying context.
It was an interesting time; I wasn't sure what I was expecting it to do after the first experiment, but it seemed to really jump into the second one and kept playing until I terminated the experiment.
If I were going to do it a third time, I'd probably create objects and give a modern agent fetch quests and other goals, and see how well it independently can handle that.
I'd really like to understand why you'd use "Answer now" though; what's your use-case?
Don't rethink your entire company, offer your customers choice instead. Some may prefer OpenClaw and others might be better served with Hermes Agent or even a third type. If you can find the best use-cases for each, you can help customers decide what they need when they buy your service and get all of the customers instead of a subset.
There's one CLI text editor left with WordStar keybindings used natively: Joe's Own Editor. If you're like me and enjoy WordStar's key combinations and the editing paradigm, JOE is a good text editor to use on the command line.
If you have a beefy enough device, then yes this can be done on-device.
Lower quants like that affect its output, making it less capable overall and letting it easily forget things.
Here's what I'd do with 96GB of RAM: Run Qwen 3.6 35b-a3b at Q8 for coding/agentic tasks. You'll get around 70tokens generated per second, the prefill is lightning fast in comparison, and you'll get a lot of work done. Qwen 3.6 27b is out now too, and I'm getting 17tok/sec token generation with a slower prefill.
The upshot is that you'll still have 20-40GB of RAM left for your workstation and development loads. Running Qwen 3.6 35b or 27b at Q8 quantization, the model at 128k context uses about 40GB of RAM; my OS and application load uses 20-30GB most of the time, for a total of 60-70. That's plenty of room in memory for you to work _and_ run inference.
You _may_ end up getting Deepseek 4 Flash running, but it'll be a lower quantization like Q2 or Q3, making it kind of dumb in comparison. And you may not have enough memory left over for any appreciable amount of context. Working with today's reasoning models needs context for it to generate and give out good answers. Doubly so for agentic/coding tasks.
{ "$schema": "https://charm.land/crush.json", "options": { "disable_provider_auto_update": true, "disable_metrics": true }, "providers": { "ollama": { "name": "Local Models", "base_url": "http://localhost:11434/v1", "api_key": "nunya", "type": "openai-compat", "models": [ { "name": "Qwen 3.5 Local", "id": "qwen-3.5-35b-planning", "cost_per_1m_in": 0.01, "cost_per_1m_out": 0.01, "context_window": 131072, "think": true, "default_max_tokens": 5120, "supports_attachments": true } ] } } }
...or not, thanks to formatting. I can't even search for help formatting this text box, because of HN's nature haha
https://github.com/charmbracelet/crush
Crush is pretty new, but getting better all the time. It's written in Go, so no node hijinks to get it working. It works fine with my ollama or llama-server localhost endpoints, and I've used it to make up a couple of internal projects without any issues.
It does have internal telemetry and such (including updating its list of external models it can use) that can be turned off in the crush.json configuration file.
If you're on a Mac, you can install via homebrew or use the more traditional route via Github.
I've only had it a couple of months, but so far it's proving its worth in the quality of LLM output, even quantized.
I generally run Qwen3-vl at 235b, at a Q4_K_M quantization level so that it fits, and it leaves me plenty of RAM for workstation tasks while delivering tokens at around 30tok/s
The smaller Qwen3 models (like qwen3-coder) I use in tandem, of course they run much faster and I tend to run them at higher quants up to Q8 for quality purposes.
The gigantic RAM's biggest boon, I've found, is letting me run the models with full context allocated, which lets me hand them larger and more complicated things than I could before. This alone makes the money I spent worth it, IMO.
I did manage to get glm-4.7 (a 358b model) running at a Q3 quantization level; it's delivery is adequate quality-wise, although it delivers at 15tok/s, though I did have to cut down to only 128k context to leave me enough room for the desktop.
If you get something this big, it's a powerhouse, but not nearly as much of a powerhouse as a dedicated nVidia GPU rig. The point is to be able to run them _adequately_, not at production speeds, to get your work done. I found price/performance/energy usage to be compelling at this level and I am very satisfied.
If you can find some public domain literature from the 20th century, it would be a much harder game.
2.3L 4-cylinder. Good enough for the light truck tasks I usually end up having. It's the last year of the first generation Rangers. I had a 1988 model with a 4 cylinder when I was young, and -that- one trucked along through all the youthful abuse I threw at it. I'm really looking forward to enjoying this one too.