qxfys

Thanks! But, I can't find any details on how you "intelligently adjust quantization for every possible layer" from that page. I assume this is a secret?

I am wondering about the possibility that different use cases might require different "intelligent quantization", i.e., quantization for LLM for financial analysis might be different from LLM for code generation. I am currently doing a postdoc in this. Interested in doing research together?

How do you pick which one should be 2, which one should be 4, etc. Is this secret sauce? or, something open?

Wondering how this kind of thing can be automatically discovered by an LLM. Anyone have any experience?

Noob question: do they need to re-certify for each new model release?

random question to a popular thread:

do any of you use LLM for code vulnerability detection? I see some big SAST players are shifting towards this (sonar is the most obvious one). Is it really better than the current SAST?

It sounds like my random Raspberry Pi sitting somewhere in my server room that has to be restarted every <x> weeks.

Tried it. It would save me a lot of time, I would say!

One suggestion: The back-and-forth chat in the beginning could be improved with a more extensive interaction. So, the final prompt could be more fine-grained into a specific area/context/anything one would aim for.

he did actually.

I can see two different interpretations in the comment section. just like our world. people believe what they want to believe.

I guess it's somewhere in the EU?

+1

flew this area countless times. cannot recall when I had a flight without turbulence.

I did. But unfortunately, work does not allow me to optimize this solution.

So, what do Ilya and Jan know that we don't know?