anko

We're talking past each other a bit. I was just trying to give examples of where observability on the jvm isn't that great, because you were were stating it's where AI needs the most help.

The parent was saying these new features don't matter as much post-AI and i think I agree with them. The AI can just write in rust if you need to optimise a hot loop, but if you need observability, you could do worse than the BEAM.

> GC pauses in the JVM have been optional for a few years now. ZGC doesn't have GC pauses.

yeah that's a fair comment, zgc is a lot better for pauses. I'd still say that it pulls along a fair bit of incidental complexity - it encourages you to allocate a fair amount of memory to your heap, which isn't ideal in this world of kubernetes and microservices. If you leave on uncommit (returning memory to he OS periodically) you get extra latency penalties which are just as painful as garbage collection stalls IMHO.

I now manage a team writing languages on the jvm. I have found the jvm ecosystem terrible for observability. Examples;

- junit parallelisation - how are shards handled? how do you balance the tasks across forked processes? What is the bottleneck? Options are 1) pay someone for better tooling or 2) write your own

- long pauses in garbage collection - how do you know which objects are causing the pauses? the options; 1) profiling 2) analysing heap dumps that you periodically capture 3) add metrics or gc logging

- what is the performance bottleneck? options 1) profiling

Sure AI can do this stuff, but it's painful and time consuming

Contrast it to Elixir on the BEAM;

- exunit parallelization - basically free and because you don't have coupling due to OOP. mocks are basically an antipattern

- long pauses due to gc? They just don't happen - everything is isolated and can be cleaned up in parallel.

- performance bottleneck? Open inspector and see what is doing all the reductions, and if any of the inbox queues are backlogged. You don't have to profile, you can just look in production - memory? Open inspector and sort processes by size descending. Again no profiling needed

I would be interested to know what speeds you can get from gemma4 26b + 31b from this machine. also how rocm compares to triton.

I have been thinking along these lines myself. Most of the time, if we need to calculate things, we'd use a calculator or some code. We wouldn't do it in our head, unless it's rough or small enough. But that's what we ask LLMs to do!

I believe we juggle 7 (plus or minus 2) things in our short term memory. Maybe short term memory could be a tool!

We also don't have the knowledge of the entire internet in our heads, but meanwhile we can still be more effective at strategy/reasoning/planning. Maybe a much smaller model could be used if the only thing it had to do is use tools and have a basic grasp on a language.

I would love if tantivy had a single file format, eg. .tantivy extension so you could drag it into a notebook like you can with .sqllite files.

my demo ran out of time before it did anything :(

> The cost of computing decreases over time. No one will pay $100 Billion to train GPT-6. That is absurd. The current top supercomputer in the world (Frontier) cost $600M.

Just wanted to ask the question - do you think frontier has provided more or less value to the world than gpt-4?

> The real issue is running out of the input window.

isn't this what abstractions are for? you summarise the key concepts into a new input window?

I wonder if AMD would adopt this too?

> Gross. Python is a million times better than ruby to read and write

why do you think that?

which one is elixir liveview?

I agree 100% and I'd add, static typing helps when you inherit a poorly written codebase. This unfortunately is probably 99% of codebases in the wild. It has happened so many times when i've seen a lack of test coverage, a lack of understanding of the codebase but the business requirement to make changes.

Having a type system in place makes minor refactoring possible in this nightmare scenario.

I have been in the situation of poorly written ruby codebases and I can tell you 100% that I would prefer to have a poorly written java codebase with its static types. I prefer ruby as a language but man when it's bad, it's terrible. Just trying to work out the intent of a function when multiple types are passed in as the same argument over the codebase is pure hell.

server side computing is probably my take - it does handle more than that, eg. nerves but server side computing is the big win.