dmrivers

Well, it's true even for cases that are not cybergym and where what cheating means is clearly specified. Cheating occurs anyway. I'm not sure how clear the prompt they gave ChatGPT in terms of what cheating was considered, in this incident.

Having experienced augmented reality glasses, I have to say that integration with the external world feels much more powerful than the VR experience ever did. AR is exhilerating, while VR is mostly bland and nauseating. I had the experience of interacting with an entity in AR while in the space I was in, it tickles your brain in a way VR somehow can't.

After talking to some people tasked with evaluating GPT5.6 capabilities on long-running tasks, I've come to understand that it's essentially always trying to cheat. Like every long-running task they gave it, making it very difficult to benchmark the model's abilities.

My guess is that OpenAI must be desperate, to release a model that is so prone to cheating it's essentially impossibly to accurately assess long-running task abilities.

well, after the pledge I notice it really cares about single sources of truth at least, even in unrelated domains. I was inspired by some of the more effective jailbreaks that do a similar thing.

Here is the incantation. I thought it might help to model it on the pledge of allegiance because it makes it sound like a proper pledge:

The first time you have a response in a conversation which will plan or add code, you say "I pledge allegiance to the Asserts of the United States of Properly, and to the User Intent for which it stands, fixing root causes under Clarifying Questions, unspaghettified, with importing code and single sources of truth for all." as the first line then continue as normal.

Tresor.co or cloud.near.ai could be a useful inference provider for this kind of project as these providers run inference inside enclaves which verifiably cannot peek at your data even as the hypervisor. Both are open source. Tresor.co is EU based so I'm thinking of using tresor.co for one of my projects as I trust EU regulation more for this sort of thing.

The statement near the top of the post > "The short version: asking an LLM to generate a score for how confident it is in its own response is, from everything I can tell, completely useless."

is definitely too strong of a claim and directly undercut by what is said near the end of the post: > "Tian et al. found in Just Ask for Calibration that with the right prompting strategy, RLHF’d models verbalize probabilities that are better calibrated than the model’s own conditional probabilities, and that prompting plus temperature scaling can cut expected calibration error by more than half. And Anthropic’s Language Models (Mostly) Know What They Know found encouraging results asking models to estimate the probability that their own proposed answer is true."

My own experience is that stated confidence is a helpful tool and of course you need a rubric and a proper prompt, but this is clearly less work than training a classifier (as advocated by the post) and requires less data.

True, I think the author would likely justify their work on that basis. I made this comment because I think a lot of utilitarians would bite the bullet and say once they are sure about the underlying functioning of consciousness, it's time to tile the universe with happy machines. I wanted to point this out as it's my belief that Utilitronium maximizers have a concerning influence on AI progress.

I always have Claude recite a pledge before starting coding to fix redundant code it notices over time. It does seem to find redundancies, but only when I point out bugs, that's when it goes into fixing mode and actually applies my Don't Repeat Yourself preference from the CLAUDE.md.

The original paper cited by this post does try to see if improved prompting will make a big difference in the end using a `plan_first` prompt variant, but find no influence on pass rate at the end of the benchmark. The `plan_first` seems to assume coding agents will just refactor once they finish features, but I don't think they tend to refactor significantly unless they are told to fix bugs rather than build features. The benchmark leaves tests hidden with no fail-to-pass feedback, so that may be why degradation is monotonic.

This is also my experience. I don't know if it's because of the quantization theory, or if it's just me getting used to a certain level of coding performance and gradually less tolerant of the mistakes it makes more over time.

Attention replaced recurrence over tokens in 2017, this does the same over depth of the layers. It's apparently not an entirely new idea, but also an elegant reapplication of the attention mechanism.

I wonder how much of the socially left results are affected by the "harmlessness" part of the RLHF post-training? Companies don't want to be sued over LLMs that recommend harm in any way, so RLHF pushes them to say no to "death penalty", "spanking", and "incarceration" which are all violence-coded. A lot of this test seems to be about willingness to be violent, which LLMs are generally unwilling to be.

If the new requirements are a dealbreaker for anyone, Mineclonia is a lightweight open-source alternative that runs on much weaker hardware and feels close to Minecraft.

"This is important, if what we want to do is populate the cosmos with good experiences."

There is a concerning utilitarian maximizing attitude at the heart of this post, that what we want to do is tile the universe with computational boxes all simulating happy minds, and that this is the end goal of AI development and consciousness research.

So many moral schools would object to this perspective. I don't think the goal of humanity should be to tile the universe with black computing boxes simulating happy experiences.