dystnitem4r3

cosmopolitan libc, by Justine Tunney (jart here on HN.) How have you missed this?

As someone who has been running llama.cpp for 2-3 years now (I started with RWKV v3 on python, one of the previous most accessible models due to both cpu and gpu support and the ability to run on older small GPUs, even Kepler era 2GB cards!), I felt the need to point out that only needing llama.cpp binaries and only being 5MB is ONLY true for cpu inference using pre-converted/quantized models. If you are getting a raw trained model from Meta, RWKV, THUD, Bytedance, Microsoft, Alibaba, or any of the big companies releasing open weight (but generally not open source) models to the public, they WILL require python, torch, and dozens to hundreds of prerequisite python modules in order to run the convert.py script to produce an output model.

Should you wish to convert a model yourself, make sure you use BF16 (exceptions apply for natively trained models in FP32, FP16, and 1/1.58 bit native formats) for the majority of the models you convert if you have enough disk space then run llama-quantize on that model to create any quantized models to minimize conversion losses and allow the accuracy vs. performance vs. space considerations that make the most sense for you.

As far as models go, Mistral-2-Large, GLM-4 variants, Mistral-Nemo-8B are my current non-multimodal favorites. llama.cpp doesn't currently support multimodal models unless you use one of the various forks using it as the inference backend due to issues embedding the image tokens in the llama-server implementation. The three models listed have most recently given the most personality when asked to play Colossus (M2L), the best translation between multiple languages while maintaining consistency between translations (GLM-4), and the most obscure code knowledge and annotation capabilities (Mistral-Nemo-8B with CodeGeeX-4-9B, a GLM-4 finetune as a close second). The last two models both were able to answer questions on 16 bit DOS C programming, near and far pointers, and even give assembly examples, although you have to specify very carefully to only emit 8086 or pre-80386 assembly mnemonics to avoid them using e?x variants of ?x registers.

May this comment prove illuminating for one searching for light.

Even more critically, dual-license commercial and AGPLv3 to avoid the AWS loophole (hosted services or source modifications without licensing it commercial.)

AGPLv3 is perfect for kernel and userspace tools because, while it doesn't guarantee licensing revenues, as soon as they do anything to modify it they have to release those changes to ANY users of the system, whether downstream distributors or simply users of the network services they provide. Since they have to provide any IP/patents they used to do it under copyleft terms, QNX would get publicly visible benefits, albeit with changes that might not be useful under the commercial license without negotiating a CLA with the modifying party, but under the commercial license you could negotiate terms/a discount to trade their new code for a reduction in their cost of licensing. It's a win/win from either an open source PR or a behind the scenes commercial negotiation point. Obviously your lawyers, bean counters, and C-suite have to get onboard, but if you are serious about growing QNX's marketshare now that Linux is gaining real-time kernel support in mainline, you had better point out to your bosses they need to make the decision quick and stick to it, or there won't be a next time.

Edit: Adjustments made to clarify my position and suggestion.

As someone who actually didn't participate in the facebook generation (I was a straight edge Millenial who started college at the tail of the Gen X generation), I do not envy anyone trying to live in modern western society without their generation's social media of choice. The few of us 'counterflow' cannot win against the tides as long as we remain part of the larger society. At best we eke out livings generally disadvantaged compared to our brethren with social media presences and all the drama that goes with it. A few of us may get opportunities from those rare outliers in positions to make a call or introduce you to a friend of a friend. But make of the rest of us simply become one of the unspoken masses, just like say the people in Slab City, or those old rock hounds who used to live in Quartzite, AZ (now some weird mass of RV park and bedroom community for Phoenix, as I understand it.)

My point, winding as it may seem, is that this generations kids are bound to their social mediums just like the radio and then television generations were to theirs, for mainstream culture, and like the Beatniks, Hippies, Progressives(I'm not sure of the proper term here, but the non-internet groups of the 80s-00s, LGBTQA movement, the BDSM movement, etc) for the outliers. There are plenty of other subcultures out there that have waxed and waned as well, some of them crossing other boundaries, like the religious or politcal gaps in this country.

But for many of us that leaves us as the odd person out. Not being into the right hobbies or social activities or just having the wrong values and you soon find yourself distanced from those around you. The internet can give that back to you or help take it away, but in the long term the dossiers on each of us that being online produce is far less damaging than the lack of in-person connections many of us(not I) gain from social networks even as we give up our privacy and our opportunities for future dissent against the status quo, something that Eastern and Western societies alike are rapidly barreling towards an ultimatum on.

Assuming you're not YOLOing it, what will you give up for your life now, versus the lives you want to leave you to your descendants, or if you're not planning on your own and not a selfish jerk, for other people's descendants?

Footnote: This comment was written from an American point of view, although much of it still applies to our Canadian cousins and European/Australian brethren.