josalhor

1. I know, but that is somewhat irrelevant to my question 2. I am indeed a CS PhD student (well, I am finishing now)

> If OAI is using chat data from researchers to make discoveries, essentially every researcher who chats with an LLM can get scooped

But they make very clear that they do train on this if you do not disable the setting. We can comment on the fact that this is opt out instead of opt in, but this discourse of OAI sniping the solution out of some researchers hands seems to be running on the best case speculation of the researchers having perfectly handled all their chats and discussions with other researchers and the worse case of OAI not having full pipeline control and I think that is an unfair assumption.

Have the other researchers opted out? On all their accounts? Through the entire time? And did they discuss this with anyone else? And did those people ask ChatGPT stuff? And did they disable it? If I was OpenAI, I would be very careful about my wording here when making claims of "we have never trained on any of their ideas directly or indirectly".

But why is this news? This is clearly described in their ToS and in the Settings to improve their models.

I think this drama was blown up a bit out of proportion. The entire discourse I am seeing online seems to revolve around this:

> While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models

I mean... yeah? What do you expect? What else can they say? How could you prove a negative in this case? I do not want to comment on specific OAI employee chat messages, but on the actual OAI discovery here.

I want the right to not be identified. I also want the right to SELF identify. I think the toxic culture of the internet has happened in part because anonymity leads to lack of accountability. I want to self identify and interact with other people who also self identify. The cost of running LLM botnets is going down. I cannot imagine how the next cycle of elections is going to be like on all countries. I understand the right to remain anonymous, that should remain, but that state should not be the default on these massively big platforms. Our brains and political systems are not designed for this. And no, "get people to get rid of their smartphone voluntarily" like I see on the comments is not going to happen.

> Well… it doesn’t exist FROM APPLE or MICROSOFT or GOOGLE at their shipped OS

What I am saying does not exist period. What I am saying is that there isn't a proper abstraction that helps the ecosystem build upon it.

> But I think the parents post is suggesting YOU CAN BUILD a prototype of what you want, how it should work, on Linux

I mean, yes. But me saying "this does not exist" and someone saying "but you can build it" does not take away from the fact that... Yeah, it doesn't exit :).

And also, no, I cannot build it, at least not alone. Because I want apps to eventually build upon my abstractions. This would require a good set of millions, of which the technical development would be a small part. The coordination, contracts, API definitions, even marketing, etc would be the majority.

I am saying something that Google, Telefonica, Microsoft etc could do.

> Chat completions is just a network call like anything else

But what if Chat completion was resolved locally with hardware? Or what if I want my OS to coordinate Chat completions locally and, if my hardware is overwhelmed, send some to network?

You do have a valid point, yes, that what I am saying, without support for local hardware could be done with a sort of Open Router equivalent.

> they don’t need to pay separately (they can use the same account), they just pass their API key over the network to the completions server

That I would be conformable putting what I am saying on my parents phone. I do no trust my parents to manage API keys. What I am saying is an ecosystem thing, not only a low level thing

Querying a CLI is also querying an API. I never said API endpoint. An API can be a Java Interface, a CLI, an endpoint etc.

No, what I am saying simply does not exist yet.

I am saying I want my OS to expose APIs like it does for the disk or the network for AI. And I want my apps to be able to use those APIs.

I want my backend LLMs to be able to change on a whim. Imagine an Android app consuming from these LLMs. Maybe I am outside and it is making queries to Gemini. And maybe I get home and now it makes queries to my local llm, almost like connecting to local Wifi.

What I am saying does not exist on many levels:

- Agreed upon APIs for this don't think exist (in text maybe, but not in image/sound/video).

- OSs do not expose this (I am not talking manually configured user space stuff here).

- I see a world where your Network provider bundles "calls + data plan + AI tokens". But not only are the offerings for these not standardized, in order to even reach that point we would need to standardize the offerings. How do you compare intelligence among models? How do you compare cost?

- The apps need to start adopting this model

The tech is here, the ecosystem is not.

We need LLM query routing at the OS level like Mobile data. I know it will sound crazy but hear me out. I think about this AI inference as infrastructure. I do not want to pay for it on every app I use it on. I do not think "I have to pay the mobile data of youtube, and the mobile data of whatsapp etc.". I pay Mobile data infrastructure and let my device route it appropiately. In fact, if we ever go the local llm route, you could have LLM capabilities without having access to the internet (or local LAN), and your OS/computer is the only one capable of doing that routing for you.

I want to add that Obscurity is ambiguous. Is changing the port of SSH "obscurity"? Some may say yes, because you could find it by bruteforce. But a password with infinite attempts can also be bruteforced. Here, the defining factor of security is maximum number of attempts (either on ports, username or whatever).

I haven't had time to see the whole thing yet, but I'm quite surprised this yielded good results. If this works I would have expected CPU implementations to do some optimization around this by default given the memory latency bottleneck of the last 1.5 decades. What am I missing here?

People use their phone today to: Manage 100k+,1M+ bank accounts, 2FA, secret messaging, sensitive media, medication, credentials and more. This privacy feature makes a lot of sense. Give it a couple of iterations and I think this will be the standard in business. It never made sense to me the trust that we put on no one looking at the contents of a display at the same time as us.

While this is great, I expected faster CPython to eventually culminate into what YJIT for Ruby is. I'm not sure the current approaches they are trying will get the ecosystem there.