nickreese

This is 100% my experience. I rarely reach for Astra as we speak.

Fun fact I learned in Spain recently. They block Cloudflare Edge IPs during soccer games. Took me a long time to figure out why I was having intermittent outages.

I had the same experience. Moving back to Sol for actual implementation.

I think people undervalue skills on the cross repo boundaries and how systems interact with other systems.

For instance… how to deploy a service or new service’a docker container. Get secrets in value blind, manage secrets value blind. Those sorts of things have been wildly valuable. Also due to the nature of skills and how they are pulled in by your harness they can really prime the context in a way that is really useful to agent autonomy if that is your thing.

This is 100% smoke and mirrors. Prove the bucket is empty and nothing was transferred out and I'll believe they deleted it.

Some people use hooks for that. I just avoid CC and use Codex.

Where would one get the dataset for this. This would be awesome to build in realtime.

This is 80% of the reason I have an apple watch. I whisper to it for reminders, timers, calendar stuff all day.

It isn't open... but drop me an email and I can send you it. Basically just tracks a list of known lmstudios on the network, queries their models every 15 seconds and routes to the ones who have the requested models loaded in a FIFO queue tracking the number of tokens/model (my servers are uniform... m4 max 128gb studios but could also track the server) and routes to the one that has just finished. I used to have it queue one just as it was expected to finish but was facing timeout issues due to an edgecase.

Exactly. Using lmstudio you can get structured output like with openai... I have everything in a zod schema then wrote an adapter to both openai and lmstudio instance.

I run lmstudio for ease of use on several mac studios that are fronted by a small token aware router that estimates resource usage on the mac studios.

Lots of optimization left there, but the systems are pinned most of the time so not focused on that at the moment as the gpus are the issue not the queuing.

I'm just now moving my main workflows off openai over to local models and I'm starting to find that these smaller models main failure mode is that they will accept edgecases with the goal of being helpful.

Especially in extraction tasks. This appears as inventing data or rationalizing around clear roadblocks.

My biggest hack so far is giving them an out named "edge_case" and telling them it is REALLY helpful if they identify edgecases. Simply renaming "fail_closed" or "dead_end" options to "edge_case" with helpful wording causes qwen models to adhere to their prompting more.

It feels like there are 100s of these small hacks that people have to have discovered... why isn't there a centralized place where people are recording these learnings?

Wow, weird to see Kuppy on this platform. Thought it was a mistake haha.