redgreenblueyel

I agree that a formalization of coherence (if it exists) probably would be more complex than a few constraints on a simple graph representation. However, this sadly gives no clue about whether it exists or not. The adoption argument is a little bit missaligned in my opinion. Coherence should make an abstract statement about how easily new changes can be integrated, not necessarily how long it takes humanity to do so. While there may be a statistical argument here, a single example might also be influenced by other effects, like trends, etc. Also, just because humans needed a lot of time to end up in a somewhat coherent state of futures, does not imply that there cannot be an abstract pattern here. I think the real question is whether coherence is context dependent or not. Or in other words, can the context influence how entangled the graph looks like.

I have to disagree. If there exists an underlying abstract, general pattern of coherence, you can optimize against it without the need of any current or next real world state. The next real world state is then resolved during inference by a system that produces high coherence. But the training can be independent. I am not claiming that there exists such learnable signal - i have no clue either, but wanted to note that the article somewhat suggests this in its plots.

This argument is true as long as coherence can only be derived from the state (or context) of the outside world. As soon as coherence becomes an inherent and measurable property of systems themselves, AI can be theoretically trained inside a bubble to achieve high amounts of coherence. Adapting the systems to the real world is then part of the inference procedure, but no longer the training procedure. Since the article argument is based on a plotting of coherence inside graphs, it basically suggest that coherence may be a system property on its own together with clear, abstract rules to derive it.