girfan

Stop giving ideas to agents...

Can someone help clarify what it means, concretely, to make binding agreements to proceed "safely and slower" in the context of AI research (whether or not at frontier labs)? Are there specific policies and actionable guides being proposed?

Curious to know what people's thoughts are on this "Stop The AI Race" movement?

I have not been able to find a clear objective---beyond slowing the progress on various AI fronts---and how that achieves an end-goal towards safer AI.

They mention things like "limits through compute thresholds (FLOP caps)" which seems extremely hand-wavy to me. What does that achieve and how do you enforce only "negative" use-cases of AI being a target of such policies? The same compute is used for non-AI workloads, along with plenty of useful advancements as a result of AI. The lines between general scientific work in numerous tangential fields (e.g., computer architecture) and how it impacts AI is blurring fast, but that does not mean (at least to me) that those fields should also be subject to an outright blockade.

I completely understand and support the need for researchers and practitioners to come together, ensure net-positive outcomes from AI progress and minimize the steep cost (e.g., environmental). I want to understand how this movement in particular (i.e., their demands, vision, plan) brings us closer to that end-goal?

Interesting. What would the incentives be for NVIDIA, for example, to opt into this? They are building their own racks now, like the NVL72 etc.

Cool tour. I haven't kept up with their developments; what kind of workloads have they been pushing for? Since they don't seem to have any specialized accelerators in the Compute Sled, I am assuming they are not targeting AI workloads for now?

Go open models!

Partly serious, partly in jest: so type systems are no good?

This is super annoying and imo, really limits the usefulness of this model. It speaks volumes about what Anthropic's position as a company and its priorities will be going forward. I doubt this kind of gatekeeping will prevent open-models or other innovation outside Anthropic to slow down. I would imagine these guardrails, if needed at all, should be done at a legal framework level and students should not be a part of this blanket approach to limiting the usage of these models.

I broadly agree with the premise. As a PhD student in Computer Science, I feel there are some significant upsides to my work routine. LLM access has made many new domains more "accessible" to me which I otherwise would be too hesitant in investing my time in. For example, my area of research is computer systems which involves operating systems, distributed systems and more recently systems for AI. Within these, there is a wide breadth of topics/techniques one can employ and up until now, I have not gone deep into theoretical aspects of things like scheduling etc. But with access to LLMs, I feel like I can at least brainstorm from a high-level about these sub-areas that I am not well-versed in and the responses give me some relevant pieces to start exploring on my own, depending on what interests me more or the amount of time I want to spend on that sub-branch of a larger tree of ideas. However, the one thing I do have skepticism is the lack of awareness of blind-spots when dabbling into areas that I am not an expert in, and taking the LLM's lead in applying such techniques to some systems problems that I am working on. I often feel that I am not aware of what alternatives exist that the LLM has not explored for me, or if the directions it has proposed really do apply or have corner cases/assumptions that break in what I am doing. On the other hand, when working on something I have good intuitions about, I am often correcting the model's assumptions and it back-tracks what it told me. Unfortunately, I cannot do that comfortably with topics I don't have good intuition about which limits my confidence in "if this is the right direction to pursue."