tssge

I've been using something similar with ZFS, 15min frequency with autopruning (Sanoid) and the snapdir mounted for the agent. Has worked well and a big plus is being able to tell the agent to just solve it's mistake via restore from the snapdir.

The reason speculative decoding shows diminishing returns in batched workloads is because the principle of both is the same.

Speculative decoding predicts a group of tokens and verifies this group using the main model in one pass instead of decoding each token separately. Eg. for this group, the weights are loaded from RAM per group instead of per token: roughly the same computation is performed but not the same memory movement (and other overhead like kernel launches).

Batching utilizes the same mechanism, so speculative decoding is essentially an attempt to batch a single stream using prediction. An attempt, because the verification may reject some tokens if the prediction was inaccurate.

>LLMs are still bad at low-level hardware optimizations, but really good at high-level composition.

I disagree. While yes they don't have all the architectural quirks of every GPU memorized, they are able to extract such optimizations from ISA docs and online guides. Now with 1M context available on frontier models, they can even fit the whole ISA definition in context (RDNA 3.5 here specifically) and spit out swathes of optimizations to try. The rest is just bruteforcing a single goal which they are extremely good at.

Or that's how simple it'll look until you have subtle bugs to solve somewhere deep in your stack.

Anyways, low-level hardware optimized GPU kernels has been an exceptionally good use case for agents in my opinion. They have far more trouble in other domains like doing GUI.

For me writing manpages as a spec worked well. While the format is unintuitive and tedious (at least to me), I used AI to format the manpages from my descriptions and rendered to PDF to review. Though it's been mainly useful for CLI utilities.

Once I got the manpages ready and the way I wanted, I just stored those in an empty project folder and told the agent to implement as specified in a language of my choosing.

Maybe for other development scenarios like GUI there is some "native" way to spec the project in a common format like this.

The GPU has INT4, INT8, BF16 and FP16. Notably no FP8 or FP4.The official GPTQ-Int4 release from Qwen is a great quant for this but custom kernels are still rare for this hardware.

I'm not arguing it's right. Frankly, I think it's stupid the way things are. But I can understand why some people make such choice.

I guess my argument mainly is that people who take their luggage are not stupid, instead their behavior may be highly rational, however we have the means to change it with by making such choice irrational and I wish we will.

In general the airlines just ask you to leave your luggage. If they were legally obliged to replace all your items, they would inform you of such.

On international flights, an airline is liable for up to $1700 per the Montreal Convention. This might cover say half of one's laptop, which no matter how stupid it sounds, makes taking your luggage with you the only financially sensible choice in a crash (unless you have insurance). Now obviously such an event has other priorities than just financial ones, but it's no surprise if people choose to take their luggage with them.

On US domestic flights the amount is somewhat higher, $4700. However even this might not be enough for some. On EU domestic flights it's 1800€.

Airlines however are free to pay any amount they want, but they are not legally required to pay more than the limits set by law. So it is possible you will be reimbursed in full, but you wouldn't know that beforehand.

Sarcasm aside, while still not acceptable, some people might not have the means to buy new items to replace what they lost in a crash. So it is understandable for some people to make the choice of taking their luggage with them in such an event, as they might not have the wealth and/or insurance necessary to replace those items afterwards.

Of course the solution would be to make airlines liable to replace passengers' luggage in the event of a crash and inform the passengers that they will do so, but that's not how the world works currently.

[deleted]

Which of course is different than the market price of an asset. You can't say sell something at intrinsic value if the market thinks differently.

>Genuine question, in what scenario is the self hosting setup and maintenance worth it?

Maybe if you're a huge org with a dedicated security team and so on, which could easily handle managing such service. I guess at a certain point it would bring cost savings at a scale in comparison to using Bitwarden, where it costs per team member or seat. Inhouse team has fixed costs in comparison.

Of course for smaller orgs or individuals there is little sense in hosting security software yourself. No way you're going to have enough time to manage the service and keep it secure, which is where almost all of such software's value is derived from.

>If you can't distinguish bad from good with LLMs, you might as well be throwing crap at the wall hoping it will stick.

This is why I think LLMs are more of a tool for the expert rather than for the novice.

They give more speedup the more experience one has on the subject in question. An experienced dev can usually spot bad advice with little effort, while a junior dev might believe almost any advice due to the lack of experience to question things. The same goes for asking the right questions.

What value? What kind of logistics value could they possible provide if they don't have logistics in the first place?

It of course is problem of the consumer, as consumer is the one returning the item and the one who ultimately has to deal with a Chinese company that may not even speak English, not to mention the time the package takes to get there for a refund.

While it's the seller's responsibility in theory, yes, they make it very clear that you'll be dealing with a Chinese seller in their terms. Yes, it is your job to read the terms before you buy, however if you do read them, why wouldn't you just go buy from AliExpress after that? Absolutely no extra value buying from the dropshipper once identified.

I think you have misunderstood what dropshipping means.

In dropshipping the business won't have any foreign made inventory and neither will they have any in their country. Instead they will hook you up with a Chinese outlet without telling you.

The Chinese outlet will handle shipping, returns, etc with the dropshipping business just taking a cut for essentially spending money on Facebook Ads. This is the part where you'll deal with a Chinese outlet regardless of your preferences, because the dropshipping company basically did this to you without your knowledge.

They don't have a local stock to replace items under warranty. They don't have a local warehouse to return your item to if you're unsatisfied. They might've never even seen the product themselves and thus cannot even provide support regarding it's use.

So that's why I think it's important to point out dropshipping. It's a completely different thing than keeping a foreign-made inventory. There isn't really any good reason to buy from a dropshipper, because you'll deal with the Chinese outlet behind it regardless.

#1 is the website ships globally with no restrictions. A lot of Chinese businesses ship to any country. A local business with their own stock generally won't handle shipping to every single country around the globe. Sure there are exceptions to this rule, but in general it works well as your first sign.

Read the terms and conditions, privacy policy and other more "obscure" information like that. Dropshipping items always ship from China and returns are received to a warehouse in China.

In their Terms of Sale, Sonic Brush mentions that it is rebranding the following product "We are selling the following brand mark : W-White.". So this would be the dropshipping product they buy from AliExpress.