hdkrgr

came here to make exactly the same comment

As a corporate customer, the main point for me in this is Microsoft now retaining (non-exclusive) rights to models and products after OpenAI decides to declare AGI.

The question "Can we build our stuff on top of Azure OpenAI? What if SamA pulls a marketing stunt tomorrow, declares AGI and cuts Microsoft off?" just became a lot easier. (At least until 2032.)

>> Always keep in mind that sometimes the only difference between yourself and the person writing the book/blog/article is that they actually wrote it.

Well said!

When you write code, do you also ignore edge cases to keep things simple?

I agree, that seems reasonable!

I was referring more to the users of such a system (what the AI Act would call a 'deployer'). They may have significantly less expertise but could still be required to track real-time energy use. Of course, simply referring to the 'energy label' by the provider could be a viable solution.

I agree and share this concern in principle.

But... have you seen the state of GDPR enforcement? Anyone who made an honest effort is fine. I don't know of any GDPR enforcement action where the indicted company wasn't blatantly and willfully violating or ignoring the law.

FWIW, everything I've seen from regulators and the legislators involved in the nitty-gritty of the act seems to suggest that most of them are really smart people who know what they don't know. They know that AI is quickly evolving and the draft of the law goes out of its way to _not_ be too specific about _how_ to comply. E.g., I would not expect the EU (or national regulators) to bring down 'one right way' to report energy consumption.

The fact that they COULD still bothers me.

The AI Act EXPLICITLY enumerates all use cases that will be considered 'high-risk' (Annex III). If your use case is not on the list (or on the 'prohibited' list), then you're good to go. There's no mechanism where someone opposed to your model can argue you should be high-risk because of supposed harms perceived or dreamt-up by some political group. (Caveat: The list of high-risk use cases will probably be able to be amended by the Commission unilaterally after the regulation is enacted.)

Since there's some confusion about this:

- The AI Act regulates both 'high-risk AI systems' and 'foundation models' and applies different requirements for them.

- 'foundation models' are essentially defined in the act as "very large scale and expensive generative ai models that will probably only be offered via API" (my words). The reason the act wants to regulate them is so that USERs of foundation models have a chance to make their downstream use case complaint if that use case is high-risk. For example, if I'm a health insurance provider and I'm using a chatbot enabled by GPT4 in my health insurance sign-up flow, then my system may be high-risk and needs to be compliant. I need access to some information aobut GPT4 (e.g. expected error modes, potential biases etc) to do that.

- The wording of the act makes a point of highlighting that your run-off-the mill open source generative AI project will not constitute a 'foundation model'. The exact scale at which a project will become a regulated 'foundation model' is not yet clear, but it can be assumed that it will be at least tens of millions of dollars. If you can spend that much on compute an researchers, I think you can spend a few k on becoming compliant.

- The technomancers article confuses requirements for High-risk systems with those for foundation models. (It also gets some of the high-risk requirements completely wrong, but that's another discussion.)

- The stanford HAL website does a great job with the facts! I really value seeing thoughtful contributions to the discussion like theirs. (Especially from an American institution!)

Sure... but maybe the GPU is sitting idle 40% of the time while still consuming 200W. Should I have to break this idle energy consumption down onto actual use (assuming the server/gpu is only used for this one model)? I guess it would make sense, but... WHO should do this and then continually update the model documentation when idle rates or the hardware changes?

The full marks for Google in the 'member-states' section made me laugh out loud. -- Bard is not available in the EU. ;)

I do think this will be a useful metric, and it seems obvious that the hyperscalers will have a feature helping you keep track of energy use and emissions of the resources you rented. But why demand this on the level of an individual model/product? For these foundation models, I think it's reasonable to assume they will all be trained on hyperscaler-provided gpu-clusters, so there'll likely be an off-the-shelf funcitonality by AWS/Azure/GCP to report this number, but the draft of the EU AI Act also demands tracking energy use for other 'high-risk' AI systems which companies may plausibly train and/or deploy on-prem. Good luck tracking the per-token energy use of your model that's running on some on-prem server on last-gen GPUs.

I had the chance to talk to a staffer of one of the MEPs leading the political negotiations in EU parlament committee a few weeks ago. His take was that the pro-tech/pro-business parties conceded the 'AI users must track their energy use'-point to the Greens in the latest draft (which is the parliament's counterproposal to earlier drafts by the commission --think EU executive-- and council --think governments of the member states--) because it's so unrealistic in practice that it's likely to be stricken out of the law again during the final negotiation round between parliament and council negotiators.

I really hope that'll be the case. FWIW, I believe companies _should_ be required to keep tabs on their (and their supply chain's) emissions, but demanding that this be done at model/system level by data scientists is just ridiculous.

edit: grammar