jodoherty
Joined 15 years ago
Joined 15 years ago
I've been trying to find something to use for enriching my own self-hosted LLMs and agentic tools with information I find useful. Metasearch tools like SearXNG make it less likely you'll get blocked by bot detection tools when finding information, but usually it's something I've already found, read, or seen that I want to incorporate into my tooling.
I came to the conclusion that a self-hosted content storage system with a search engine and a browser extension that can extract and save web page content and metadata was the ideal setup for me. Preferably with some sort of federated content sharing ability and the ability to import creative commons content like Wikipedia and Gutenberg.
This looks almost exactly like what I wanted.
It'll be a few weeks before I have time to audit the code and deploy it, but I'm really looking forward to trying it out.
If I get some time to circle back, I'll be sure to incorporate these into some new tests and address them.
I want to set up a qemu-system emulator based testing approach so I can incorporate things like AppArmor and SELinux into end to end tests that include different environment configurations.
Part of that will be setting up software defined networking so I can have things like DNS and wireguard VPN servers in a box and then test and evaluate the wg-wrap behavior at the packet level.
I find it useful.
This side project highlights a similar approach to how I scope and tackle projects at work now:
https://git.theodohertyfamily.com/wg-wrap.git/tree/README.md
https://git.theodohertyfamily.com/wg-wrap.git/tree/CASE_STUD...
You have to apply a lot of careful architecture and TDD to your approach. Eliminate technical risk by tackling hard things early and wrapping them up in a simple, easy to use interface.
I find I can get some projects done 2-3 times faster than if I wrote them by hand. It can also save about 5-10x time on mundane or broadly scoped projects by helping me consolidate and try out ideas very quickly.
Setup-wise, I switch between vLLM using nvidia/Gemma-4-31B-IT-NVFP4 and llama.cpp using unsloth/gemma-4-31B-it-qat-GGUF with MTP. I throttle the GPU power usage to 400W.
My current llama.cpp setup gets token generation rates between 60-150 t/s depending on MTP draft acceptance rates. Prefill is between 1500-4000 t/s depending on context length/depth.
WinApps needs a Windows RDP server to work. Most of the functionality doesn't care where that Windows RDP server is actually running as long as its FreeRDP client can connect to it. The container or libvirt VM options are just ways to accomplish that via virtualization.
I imagine the container part makes it easier to automate QEMU virtualization using bash scripting without worrying about distribution specific differences in the environment. These kinds of scripts become fairly ossified to their environment. Making them run consistently on different Linux distributions is its own adventure unrelated to installing and running Windows, so the containerization eliminates the need for a lot more bash scripting to account for those differences.
The container's bash scripts download the Windows installer ISOs and run them in an unattended mode inside a QEMU VM. The unattended installation is configured to skip activation:
- https://github.com/dockur/windows/blob/c7aac1edcf37a69ff730d...
- https://github.com/dockur/windows/blob/c7aac1edcf37a69ff730d...
- https://github.com/dockur/windows/blob/c7aac1edcf37a69ff730d...
Once the container is running, WinApps configures RDP via some scripts and registry settings exposed into the container via a volume so the container's scripts can copy and run them in the Windows VM:
- https://github.com/winapps-org/winapps/blob/b4766336903d0cbe...
- https://github.com/winapps-org/winapps/blob/main/oem/RDPApps...
You can do it all yourself too with your own libvirt VM, but it's just more involved:
- https://github.com/winapps-org/winapps/blob/main/docs/libvir...
I haven't seen any of this before, but I think it's a pretty clever use of scripting and containers on top of some fairly mature but hard to use pieces of software.
However, people can also adapt pretty quickly.
Those grocery shopping "optimization" skills are making a big come back (and have been since Covid). There are plenty of YouTube and TikTok videos popularizing how to get more out of their grocery hauls.
Lots of people are also learning how to budget, how to invest, etc. and sharing their excitement about it too. For some folks, they finally learn this stuff in their 40s and 50s, but there are also a lot of young adults learning these skills thanks to the Internet.
So I also have hope.
I just don't think the lack of basic math and budgeting skills displayed by average consumers are a problem so much as a symptom.
Probably a better example is figuring out the cost of a loan. Just multiply the amortized monthly payment by the term and compare that to the loan amount. If the difference makes you balk, then go ahead and walk.
How many people even realize that loan interest is a significant cost and would bother to do that? Or know how to do that? Most people just try to minimize monthly payments to something they can bear and sign the paperwork.
Math gives you the ability to leverage the very structure and relationships of pure abstraction. It's quite the super power.
None of the specific things you learn studying math will be nearly as useful as the ability to think mathematically.
A lot of editing tools and processes automatically converting `--` to `—`, so folks editing markdown or using a Word processor might get the emdash automatically. Similar things are often done for matching double quotes. I wouldn't be surprised if quite a few WYSIWYG CMS systems do this conversion too.
There are also a lot of input methods that make it trivial to write special characters.
Apple famously uses the Alt/Option key to make inputting a lot of special characters simple. Look at any place that does a lot of writing and publishing, and you're bound to see a lot of Macs.
On Windows, you can memorize and input code points pretty easily as well if you have a number pad. Just hold Alt and punch in the 4-digit character code.
I hop platforms a lot, so I commonly use digraphs with Ctrl+K in VIM, or TeX input in Emacs to insert unicode characters. I'll also use `Ctrl+x 8 <RET>` to insert characters by name in Emacs when I need to search for something specific.
My $200 1500VA/1000W CyperPower UPS could handle a short one or two hour storm, but storms can last for 10+ hours here during the wet season. One long power outage could cost me a lot more than $1000 in basement damages (my basement is finished).
A comparable all-in-one product to this pila battery would also be around $1000: https://us.ecoflow.com/products/delta-2-max-portable-power-s...
You can easily cut that in half shopping for an inverter and battery separately:
$240 for an inverter: https://www.homedepot.com/p/VEVOR-2500-Watt-3-4-HP-Sump-Pump...
$200 for a battery: https://www.homedepot.com/p/Basement-Watchdog-Maintenance-Fr...
But a lead acid battery might last 5 years where the pila battery is designed for 10+ years, so you're looking at $400 total on batteries. That quickly gets close to the $1000 for one pila battery.
I want to believe $100 is all you need, but to me the math with the numbers that I've seen suggests $1000 is not unreasonable for something like this.
https://docs.python.org/dev/howto/sockets.html#socket-howto
It gets overlooked because most folks only check the Tutorial and the Standard Library sections.
Now I have to worry about my laptop randomly overheating itself in my backpack and even catching fire.
See page 9 of the datasheet here:
https://ftdichip.com/wp-content/uploads/2020/07/DS_FT232H.pd...
The async and sync bitbang columns denote the GPIO pins as D0-D7 and show them assigned to the 8 ADBUS pins that are provided on the breakout board.
Assuming the udev rules are set up in Linux, you can simply install pyftdi, open the device, and start using the ADBUS pins as GPIO pins:
https://eblot.github.io/pyftdi/gpio.html#setting-gpio-pin-st...
If you're using libftdi, you want to call ftdi_set_bitmode with the BITMODE_BITBANG enum value for the mode:
https://www.intra2net.com/en/developer/libftdi/documentation...
Then the ftdi_read_data and ftdi_write_data functions can be used to read or write to the ADBUS pins:
https://www.intra2net.com/en/developer/libftdi/documentation...
https://www.intra2net.com/en/developer/libftdi/documentation...
You can then build a nice, simple high level GPIO interface over that if you want.