hellohello2

Touchscreens are so passé, the future is talking to your computer and having it do things for you.

I though 4 bit (mixed) quant is pretty standard, at least for local models that basically what everyone runs? Unless I'm mistaken it can really be a negligible difference in perf.

By (morbid) curiosity did you program as a hobby before? Sorry for being a jerk, just actually wondering how different people are taking it.

IDK, I thought programming was great fun before and am having a blast making fun stuff with AI.

Yes it is, and imagination is how we got to the moon, or got rockets that land, or why I can talk to my computer now.

Why use apps that you dislike, just ask Claude to click through them for you. And no I'm not being ironic, this is the way things are headed, you just abstract over all the crap by having an agent do it for you.

"Slop is its own cure"

Social roles get created at the frontier of what isn't technologically feasible without a human, as culture develops around some form of human activity. Role A does B, role X does Y; I don't know, maybe its just time for new roles that didn't exist before.

Publicly-accessible AI started working 1 year ago, 2 years tops. What did you expect? It to solve war driving prices up?

The main difficulty with fast (low-latency) inference, is not actually computation but loading parameters from memory. The problem with generating 1 token at a time isn't that that its expensive computationally (it is, but so is training), but that you need to stream your entire model from memory for every single token (and also the KV cache but that's besides the point). So the strategy usually taken is to process big batches of user requests, letting you share the memory loads across users. This means individual answers aren't that fast, but you can do lots at once. This is why local inference isn't cost-effective; its because the model weren't designed for it in the first place.

My hypothesis for Jev is that they simply generate many answers independently in parallel from your prompt, and then discard the duplicates (or train to avoid duplication with attention between the ). In that way the entire batch is 1 user's prompt.

I was thinking about this some more, and perhaps the best solution for subscription plans would be to charge more for real privacy. In which case breaking that privacy would be committing fraud. Just a thought.

Option 2 feel reassuring enough, but is out of reach of particulars.

Option 1 is not. In part because it is opt-out (will it turn back on on its own like my Facebook privacy settings?), and not always respected (sending feedback can mean your chat is used?). Also because disabling "Improve model for everyone" is very vague.

There simply needs to be a setting like "my data is confidential", in which case there clear guarantees like there are for ZDR.

As an example, I've seen people speculate that while input prompts and output tokens are discarded, thinking traces are retained for training, which could leak information. I doubt this is true, but it shows that the policy is not unambiguous and reassuring enough to remove all doubt.

Thanks for asking.

You guys should seriously offering a clear way of working with (semi-)confidential data for particulars. Regardless of what is actually done internally, toggling off an opt-in isn't reassuring enough, which is why people are having these worries.

Generative models copy training data verbatim and also generalize, the two are not mutually exclusive.

You can have a look at the literature on exact copying in image models if it interests you, but just online we often see online examples of agents outputting code that already exists, even if its not the common case.

I very much doubt OpenAI points the model towards a specific conversation, but these trillion parameter models can very much "remember" their training data. For instance I can ask GPT to summarize my papers from their title alone, without looking them up, and it works decently.

Of previous, not concurrent work. Science is friendly competition, and spying on others is unfriendly.