declaredapple
Joined 2 years ago
Joined 2 years ago
These smaller models are a relatively cheap to run, especially in high batches.
I'm sure there will also be aggressive rate limits.
Also because of their unified memory design, they actually have insane bandwidth which is incredibly useful for LLMs. IMO they may have a head-start in that respect for on-device inference of large models (e.g. 1B+ params).
However much of this social impairment may or may not be a real problem. "special interests" and "obsessing over one topic" is an impairment in social scenerios, but can be extremely beneficial for tasks related to that special interest.
The irony in this is magical.
I highly suggest you remarket as a collection of some type of link a la product hunt. Tech startups, cool apps, anything.
"backlink service" is literally just "seo spam" on it's own.
From what I could tell from Nvidia's recent presentation, Nvidia works directly with OpenAI to test their next gen hardware. IIRC they had some slides showing the throughput comparisons with Hopper and Blackwell, suggesting they used OpenAI's workload for testing.
H100's have been generally available (not a long waitlist) for only several months, but all the big players had them already 1 year ago.
I agree with you, but I think you might be 1 generation behind.
> OpenAI used H100’s predecessor — NVIDIA A100 GPUs — to train and run ChatGPT, an AI system optimized for dialogue, which has been used by hundreds of millions of people worldwide in record time. OpenAI will be using H100 on its Azure supercomputer to power its continuing AI research.
March 21, 2023 https://nvidianews.nvidia.com/news/nvidia-hopper-gpus-expand...
Are you asking if the framework automatically quantizes/prunes the model on the fly?
Or are you suggesting the LLM itself should realize it's too big to run, and prune/quantize itself? Your references to "intelligent" almost leads me to the conclusion that you think the LLM should prune itself. Not only is this a chicken and egg problem, but LLMs are statistical models, they aren't inherently self bootstraping.
Claude3 opus (and often Sonnet) actually fills in the full dependency paths, actually makes tests, and just overall seems to "know what I want from it".
FWIW asking LLMs about their training data is generally HEAVILY prone to inaccurate responses. They aren't generally told exactly what they were trained on, so their response is completely made up, as they're predicting the next token based on their training data, without knowing what they data was - if that makes any sense.
Let's say it was only trained on the book 1984. It's response will be based on what text would most likely be next from the book 1984 - and if that book doesn't contain "This text is a fictional book called 1984", instead it's just the story - then the LLM would be completing text as if we were still in that book.
tl;dr - LLMs complete text based on what they're trained with, they don't have actual selfawareness and don't know what they were trained with, so they'll happily makeup something.
EDIT: Just to further elaborate - the "innocent" purpose of this could simply be to prevent the model from confidently making up answers about it's training data, since it doesn't know what it's training data was.
> Changing the license of PostgreSQL would shatter all of the goodwill the project has built up through the past (nearly) 30 years.
I love Postgres and I have no reason to think they would change their license. But IMO any type of "it would be a disservice to users" is useless. This blog post itself is in response to Redis changing their license after 15 years. Surely this would be a disservice to their users?
Businesses have proven time and time again - They will absolutely cause a "disservice to users" or "harm the product" once money is involved.
Again I recognize the way PostgresQL is structured gives me hope this wouldn't happen, but a simple "we wouldn't break user trust" is meaningless.
I know people likely want to join NASA for more than the salary. But that's brutal.