Carlos Ramirez's Shortform

The thing is though, there isn't a dichotomy between agents and processes. Everything physical (except maybe the elementary particles) is a process in the final analysis, as Heraclitus claimed. Even actual individual persons, the paradigmatic examples of an agent, are also processes: the activity of the brain and body only ever stop at death. The appearance of people as monadic agents is just that, an appearance, and not actually real.

This might have sounded too much philosophical woo-woo, but it does have pragmatic consequences, which is that since agents are a facade over what is actually a process, the question becomes how do you actually tell which processes do or do not have goals? It's not obvious that only processes that can pass as agents are the only ones that have goals.

EDIT: Think about it like this: when a river floods and kills hundreds or thousands, was it misaligned? Talk of agents and alignment only makes sense in certain contexts, and only as a heuristic! And I think AI X-risk is a context in which talking in terms of agents and alignment obfuscates enough critical features of the subject that the discourse starts excluding genuine understanding.

EDIT 2: The above edit being mostly a different way of agreeing with you. I guess my original point is "The scientific research process is dangerous, and for the same reasons rogue superintelligence would be: opacity, constantly increasing capabilities, and difficulty of guaranteeing alignment". I still disagree with you (and with my own example of the river actually) that non-agents (processes) can't be misaligned. All natural forces can be fairly characterized as misaligned, as they are indifferent to our values, and this does not make them agentic (that would be animism). In fact, I would say "a dangerous non-agent is not misaligned" is false, and "a dangerous non-agent is misaligned" is a tautology in most contexts (depends on to whom it is dangerous).

Civilization collapsing is blatantly better than rogue superintelligence, as it's plausibly a recoverable disaster, so yes, that is my honest belief. I don't consider non-organics to be moral entities, since I also believe they're not sentient. Yeah, I'm aware those views are contested, but then, what the hell isn't when it comes to philosophy. There are philosophers who argue for post-intentionalism, the view that our words, language and thoughts aren't actually about anything, for crying out loud.