tbocek
Joined 3 years ago
Karma
25
Submissions
12
Member since
Jun 8
Recent submissions
Comment · 4 months ago
Comment · 10 months ago
That's a valid concern. I took a more constrained approach for web searches for exactly this reason. Instead of giving the LLM full browser control, I built a Firefox extension that only handles web search client-side.
When my local LLM (llama.cpp) needs to search, it opens DuckDuckGo in a new window, loads the result pages in tabs, extracts content with Readability.js, and feeds it back. You stay in the loop - can see what's loading, solve captchas if needed. Less autonomous than Comet/Playwright, with a narrower use-case, but also less risk.
Its still a prototype though: https://github.com/tbocek/llm-local-web-search
Comment · 1 year ago
Thx, I cannot change my previous message, the QH is set to private, as me and my students are working on it. It will be made public next month. The qh:// protocol, we have just started, and we only focused on simplifying the header as an alternative to HPACK / QPACK. Currently we are collecting headers for identifying what should go into our static table for lookup, and for benchmarking. For simplicity, we decided against dynamic tables or huffman-style encoding/decoding. Just a table with often used header/values, the rest is binary encoded with no further compression.
Comment · 1 year ago
UDP-based protocols are well suited for P2P, since hole punching is straightforward if you have predictable port mapping, you cannot disallow it. In that spirit, we are currently exploring this with:
https://github.com/tbocek/qotp and https://github.com/qh-project/qh
The main idea is to have a simple encryption (ed25519/chacha20+poly1305) for encryption in the transport layer, on top of that then qh, where certs are use for signing content.
With out of band key exchange, you can establish a connection after you successfully punched a hole.
However, its not QUIC compatible in any way (https://xkcd.com/927)
Comment · 1 year ago
This is probably due to this: https://github.com/ggml-org/llama.cpp/issues/12637. This GitHub issue is about interleaved sliding window attention (iSWA) not available in llama.cpp for Gemma 3. This could reduce the memory requirements a lot. They mentioned for a certain scenario, going from 62GB to 10GB.
Comment · 1 year ago
Here is mine, I have a docker compose file locally, and this deploy.sh script deploy to my remote machine. That also means that my remote machine is building the image. And I have not found a good solution for secrets/env files yet:
#!/usr/bin/env bash
export DOCKER_HOST="ssh://username@host:port"
docker compose up -d --buildComment · 1 year ago
Just did a test with https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-32..., with the question "what happened at Tiananmen Square?", and here are parts of the thinking process:
...I also remember something about the "Tank Man" image, where a lone protester stood in front of a line of tanks. That image became iconic, symbolizing resistance against oppression. But I'm not sure what happened to that person or if they survived.
After the crackdown, the government censored information about the event. So, within China, it's not openly discussed, and younger people might not know much about it because it's not taught in schools. But outside of China, it's a significant event in modern history, highlighting the conflict between authoritarian rule and the desire for democracy...Comment · 2 years ago
There is a difference between chat and code completion. While with chat, you can use localhost with llama.cpp, but code completion you cannot do that: https://github.com/zed-industries/zed/issues/12519.
The config for chat, you can do:
"language_models": {
"openai": {
"version": "1",
"api_url": "http://localhost:8080",
"low_speed_timeout_in_seconds": 120,
"available_models": [
{
"provider": "openai",
"name": "Qwen2.5-Coder-7B-Instruct-Q8_0.gguf",
"display_name": "llama.cpp",
"max_tokens": 131072
}
]
}
},
Comment · 3 years ago
I had the same issue, so I implemented the following workaround: https://github.com/tbocek/dvorak. This remaps on Linux CTRL-I back to CTRL-C when using a Dvorak keyboard layout.
I tried working through it with Claude, but after a few failed attempts I gave up. I'd like to use podman, but the docker compose + buildx compatibility gaps made it more trouble than it was worth for now. I'm definitely going to try it again.