wombat23

UPDATE: I did more experiments - this time with llama-server and pi harness. i'll leave the results here for posteriority:

  ./llama.cpp/build/bin/llama-server \
    -c "$context_size" \
    -ctk q4_0 \
    -ctv q4_0 \
    --no-models-autoload \
    --models-dir ~/bonsai \
    --reasoning-preserve \
    -ngl 99

the max context size i could serve is 64K on GPU only (the -ngl 99 setting). tested with pi harness and it is very fast. the /thinking level always gets reset to off though and it is not very smart like this. haven't figure out a way to fix that.

I don't see any significant speed up. also with defaults according to --help

  -b,    --batch-size N                   logical maximum batch size (default: 2048)
  -ub,   --ubatch-size N                  physical maximum batch size (default: 512)

I also tried double the default. that also means that they can be left to default settings, apparently.

I managed to run it with RTX 3070 (8GB VRAM) following the "Quickstart" on HF model card with minor modifications (modify the architecture 86 for your own hardware):

  git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
  cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 && cmake --build build -j

Then downloaded & verified `Ternary-Bonsai-2-27B-PTQ1_0.gguf` from HF and ran

  ./llama.cpp/build/bin/llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -fa on -c 32768 -b 256 -ub 64 --temp 1.0 --top-p 0.95 --top-k 20 -ctk q8_0 -ctv q8_0 -p "hello world in x86 assembler" -n 256

the parameters were suggested by gpt-5.6-luna to reduce memory footprint, as the defaults ran OOM on my gpu. result looks good:

  [ Prompt: 165.6 t/s | Generation: 40.6 t/s ]

would be nice if they upstreamed their changes so that it runs with the original llama.cpp

Super interesting.

Last week-end I was exploring the current possibilities of automated Ghidra analysis with Codex. My first attempt derailed quickly, but after giving it the pyghidra documentation, it reliably wrote Python scripts that would alter data types etc. exactly how I wanted, but based on fixed rules.

My next goal would be to incorporate LLM decisions into the process, e.g. let the LLM come up with a guess at a meaningful function name to make it easier to read, stuff like that. I made a skill for this functionality and let Codex plough through in agentic mode. I stopped it after a while as I was not sure what it was doing, and I didn't have more time to work on it since. I would need to do some sanity checks on the ones it has already renamed.

Would be curious what workflows others have already devised? Is MCP the way to go?

Is there a place where people discuss these things?

I've been wondering the same. Apparently, where I live (HCOL) people don't care about ever paying off the full amount, as long as the monthly installment is low. Regulations only require that you paid off 33% of the house value after 15 years.

You do need to start at 20% capitalization though, which at current prices means they need to own assets already (market risk), or have bonkers amounts of currency sitting around (inflation risk). I understand the FOMO some people must have.

I wonder if ultimately there is a cascade happening, where increased valuations lead previous owners to either take new mortgages, or sell their old house, and pay more for a new one. Which creates a cycle of ever increasing prices? I've never read anything though whether such an effect exists.

> Even just running correlation analysis on these various assets over the last 20 years showed that crypto was super underpriced

care to elaborate?

When I read stories like this about cool technology from the past that people can still enjoy today, I think it's a pity that the source code is locked away and lost as people and companies move on to new things.

IMO, there should be some kind of archive that conserves and publishes them after some time has passed, so that they could be ported to new hardware and kept accessible. and somehow documented for future historians.

Looks like a different approach. Intel's chip is based on digital circuits. They try an analog approach.

Well it kinda is.

Empirically, you could have bought a share of the SPX at any point in time, and sold it with profit later. The real problem is what happened in the time between, and whether you were able to hold on.

Nice reminder of some general principles. I wonder how well it applies to young scientists given today's academic career prospects, though.

> If you want to make progress in any area, you need to be willing to give up your best ideas from time to time. [...] Medawar notes that he twice spent two whole years trying to corroborate groundless hypotheses.

Unfortunately, being willing to scrap your idea is only one part of the equation. Securing funding after 2 "failed" post-docs is an entirely different matter.

slightly OT, but does anyone know if there have been any attempts at creating an open source version of the original CS? Kinda like OpenRA or similar?

Are there more sources for technical details about the new infrastructure? The interview linked above left me with more questions than answers.