appenz

IANAL and this is not legal advice, but to explain the issue further in a number of countries training is considered fair use. If someone training the model can find one country where this is the case (and there are no edge case issues from things like copying the data for training) training is legal. So far, the US hasn’t restricted importing models (which specifically for open source might not be possible due to free speech) so using that model in the US is legal as well.

And understood that there are strong opinions on both sides of the argument if this is good or bad.

I tested it against Flux Pro Kontext (also image editing) and while it's a very different style and approach I overall like Flux better. More focus on image consistency, adjusts the lighting correctly, fixes contradictions in the image.

No deeper reason. I think there just a lot in LLMs happening right now which skewed it towards them. We would love to do something in the SD ecosystem.

We 100% agree!

Chat history may work, it depends on how long it is and the business model.

I don't quite understand how general summarization would work. If you use an LLM to simply to summarize in order to feed it into a prompt, the summarization needs to be specific to the query. i.e. "summarize what this text says about topic X". You can't summarize long text in a generic way without losing information. Or do I misunderstand the comment?

If you have a perfect table of context (or better, an index by topic) you may not need semantic search. But for the typical use case we are seeing you have unstructured data without an index (e.g. tech support knowledge db entries, company reports, emails). For that, semantic search work quite well.

For the sizes, the observation is that the data that people want to search over (e.g. your email, a wiki, JIRA, a knowledge base) is far larger than the context length. You are correct that we assume that inference cost and speed won't decrease sufficiently quickly in the near future. Why is a longer topic, but in a nutshell GPU speed increase is ~2.5x gen/gen and other than overtraining vs. Chinchilla we don't see immediate model gains. But that is speculative, we don't know what's in store.

To some degree we are just reacting to user adoption in the market. We don't build these systems, but if we see enough of them eventually we recognize the pattern. And while I am optimistic, we could be wrong. AI is major revolution and we are all students.

edit: disclaimer, I work for a16z.

Summarization is much more expensive than vector db's. Assume you have 1m tokens of context. You could run all through GPT-4 and summarize the information, but it would cost $60 (based on current prices) and take 10's of minutes of GPU time to do the inference.

Disclaimer: I work for a16z and on the infra team, so consider me biassed.

Wrong school? Our local schools (Silicon Valley, CA) encourage kids to use Wikipedia. Your school may just be a little behind the times.

The discussion now has moved to NLP models. GPT-3 models at this point can generate extremely high quality answers to complex questions. Is there still a point in asking a student to write a few paragraph on the definition and effect of acid rain if you can get that from OpenAI within seconds?