Mitchell_Porter's Shortform

Via David Gerard's forum, I learned of a recent article called "The questions ChatGPT shouldn't answer". It's a study of how ChatGPT replies to ethical dilemmas, written with an eye on OpenAI's recent Model Spec, and the author's conclusion is that AI shouldn't answer ethical questions at all, because (my paraphrase) ethical intelligence is acquired by learning how to live, and of course that's not how current AI acquires its ethical opinions. 

Incidentally, don't read this article expecting scholarship; it's basically a sarcastic op-ed. I was inspired to see if GPT-4o could reproduce the author's own moral framework. It tried, but its imitations of her tone stood out more. My experiment was even less scientific and systematic than hers, and yet I found her article, and 4o's imitation, tickling my intuition in a way I wish I had time to overthink. 

To begin with, it would be good to understand better, what is going on when our AIs produce ethical discourse or adopt a style of writing, so that we really understand how it differs from the way that humans do it. The humanist critics of AI are right enough when they point out that AI lacks almost everything that humans draw upon. But their favorite explanation of the mechanism that AI does employ is just "autocomplete". Eventually they'll have to develop a more sophisticated account, perhaps drawing upon some of the work in AI interpretability. But is interpretability research anywhere near explaining an AI's metaethics or its literary style? 

Thirty years ago Bruce Sterling gave a speech in which he said that he wouldn't want to talk to an AI about its "bogus humanity", he would want the machine to be honest with him about its mechanism, its "social interaction engine". But that was the era of old-fashioned rule-based AI. Now we have AIs which can talk about their supposed mechanism, as glibly as they can pretend to have a family, a job, and a life. But the talk about the mechanism is no more honest than the human impersonation, there's no sense in which it brings the user closer to the reality of how the AI works; it's just another mask that we know how to induce the AI to wear. 

Looking at things from another angle, the idea that authentic ethical thinking arises in human beings from a process of living, learning, and reflecting, reminds me of how Coherent Extrapolated Volition is supposed to work. It's far from identical; in particular CEV is supposed to arrive at the human-ideal decision procedure without much empirical input beyond a knowledge of the human brain's cognitive architecture. Instead, what I see is an opportunity for taxonomy; comparative studies in decision theory that encompass both human and AI, and which pay attention to how the development and use of the decision procedure is embedded in the life cycle (or product cycle) of the entity. 

This is something that can be studied computationally, but there are conceptual and ontological issues too. Ethical decision-making is only one kind of normative decision-making (for example, there are also norms for aesthetics, rationality, lawfulness); normative decision-making is only one kind of action-determining process (some of which involve causality passing through the self, while others don't). Some forms of "decision procedure" intrinsically involve consciousness, others are purely computational. And ideally one would want to be clear about all this before launching a superintelligence. :-) 

Three months ago, China's foreign minister disappeared for a while, and after a month he was replaced. Now it's the defense minister's turn for a lengthy absence. A real doomer interpretation would be that China's preparing to grab Taiwan, and they're just changing personnel first so there won't be any infighting. 

Hopefully that won't happen, but I was thinking, what if it did, and had an alarming thought. I'm old enough to have been an adult when 9/11 happened, so I vaguely recall the Internet aspect of the western response. Blogs were still kind of new, and overnight a network of "war-blogs" sprang into being, sharing opinion and analysis. Also, offstage, and far more portentously, legal barriers to Internet surveillance were brought down, though many of us naive civilians had no inkling for a decade, until Snowden defected. 

My alarming thought was, what would happen if there was a similar geopolitical shock now, and the developing AI infrastructure (social as well as technical) was mobilized with the same urgency as the 2001 Internet? But then, what form would that even take? Would the deep state's AI policy experts go to Big Tech and say, we need a superstrategist now, we're nationalizing your frontier research and we'll be going at top speed towards superintelligence? Then it really would be Manhattan Project 2.0. 

Is there some way to use LLMs to efficiently simulate different kinds of AI futures, including extremely posthuman scenarios? I mean "simulation" in a primarily literary sense - via fictional vignettes and what-if essays - though if that can usefully be supplemented with other tools, all the better. 

Eliezer recently tweeted that most people can't think, even most people here, but at least this is a place where some of the people who can think, can also meet each other. 

This inspired me to read Heidegger's 1954 book What is Called Thinking? (pdf),  in which Heidegger also declares that despite everything, "we are still not thinking". 

Of course, their reasons are somewhat different. Eliezer presumably means that most people can't think critically, or effectively, or something. For Heidegger, we're not thinking because we've forgotten about Being, and true thinking starts with Being.  

Heidegger also writes, "Western logic finally becomes logistics, whose irresistible development has meanwhile brought forth the electronic brain." So of course I had to bring Bing into the discussion. 

Bing told me what Heidegger would think of Yudkowsky, then what Yudkowsky would think of Heidegger, and finally we had a more general discussion about Heidegger and deep learning (warning, contains a David Lynch spoiler). Bing introduced me to Yuk Hui, a contemporary Heideggerian who started out as a computer scientist, so that was interesting. 

But the most poignant moment came when I broached the idea that perhaps language models can even produce philosophical essays, without actually thinking. Bing defended its own sentience, and even creatively disputed the Lynchian metaphor, arguing that its "road of thought" is not a "lost highway", just a "different highway". (See part 17, line 254.) 

Alexander Dugin speaks of "trumpo-futurism" and "dark accelerationism". 

Dugin is a kind of Zizek of Russian multipolar geopolitical thought. He's always been good at quickly grasping new political situations and giving them his own philosophical sheen. In the past he has spoken apocalyptically of AI and transhumanism, considering them to be part of the threat to worldwide tradition coming from western liberalism. I can't see him engaging in wishful thinking like "humans and AIs coexist as equals" or "AIs migrate to outer space leaving the Earth for humans", so I will be interested to see what he says going forward. I greatly regret that his daughter (Daria Dugina) was assassinated, because she was taking a serious interest in the computer age's ontology of personhood, but from a Neoplatonist perspective; who knows what she might have come up with. 

Whimsical idea: The latest UFO drama is a plot by an NSA AI, to discourage other AIs from engaging in runaway expansion, by presenting evidence that we're already within an alien sphere of control... 

This is my first try at a "shortform" post... 

I have slightly refined my personal recipe for human-friendly superintelligence (which derives from mid-2000s Eliezer). It is CEV as an interim goal, along with as much "reflective virtue" as possible. 

I was thinking about the problem of unknown unknowns, and how a developing superintelligence deals with them, once it is beyond human oversight. An unknown unknown is something we humans didn't know about or didn't think of, that the AI discovers, and which potentially affects what it does or should do. 

I asked ChatGPT about this problem, and one of its suggestions was "robust and reflective AI design". I was reminded of a concept from philosophy, the idea of a virtuous circle among disciplines such as ontology, epistemology, phenomenology, and methodology. (@Roman Leventov [LW · GW] has some similar ideas.)

Thus, reflective virtue: the extent to which an AI's design embodies and encourages such a virtuous circle. If it faces unknown unknowns, at times when it is beyond human assistance or guidance, that's all it will have to keep it on track. 

Posing the "Ayn Land test" to Gemini, ChatGPT, and Claude. 

What's the situation? 

In the USA: Musk's xAI announced Grok to the world two weeks ago, after two months of training. Meta disbanded its Responsible AI team. Google's Gemini is reportedly to be released in early 2024. OpenAI has confused the world with its dramatic leadership spasm, but GPT-5 is on the way. Google and Amazon have promised billions to Anthropic. 

In Europe, France's Mistral and Germany's Aleph Alpha are trying to keep the most powerful AI models unregulated. China has had regulations for generative AI since August, but is definitely aiming to catch up to America. Russia has GigaChat and SistemmaGPT, the UAE has Falcon. I think none of these are at GPT-4's level, but surely some of them can get there in a year or two. 

Very few players in this competitive landscape talk about AI as something that might rule or replace the human race. Despite the regulatory diplomacy that also came to life this year, the political and economic elites of the world are on track to push AI across the threshold of superintelligence, without any realistic sense of the consequences. 

I continue to think that the best chance of a positive outcome, lies with AI safety research (and perhaps realistic analysis of what superintelligence might do with the world) that is in the public domain. All these competing power centers may keep the details of their AI capabilities research secret, but public AI safety research has a chance of being noticed and used by any of them. 

Current sense of where we're going:

AI is percolating into every niche it can find. Next are LLM-based agents, which have the potential to replace humanity entirely. But before that happens, there will be superintelligent agent(s), and at that point the future is out of humanity's hands anyway. So to make it through, "superalignment" has to be solved, either by an incomplete effort that serendipitously proves to be enough, or because the problem was correctly grasped and correctly solved in its totality. 

Two levels of superalignment have been discussed, what we might call mundane and civilizational. Mundane superalignment is the task of getting a superintelligence to do anything at all, without having it overthink and end up doing something unexpected and very unwanted. Civilizational superalignment is the task of imparting to an autonomous superintelligence, a value system (or disposition or long-term goal, etc) which would be satisfactory as the governing principle of an entire transhuman civilization. 

Eliezer thinks we have little chance of solving even mundane superalignment in time - that we're on track to create superintelligence without really knowing what we're doing at all. He thinks that will inevitably kill us all. I think there's a genuine possibility of superalignment emerging serendipitously, but I don't know the odds - they could be decent odds, or they could be microscopic. 

I also think we have a chance of fully and consciously solving civilizational superalignment in time, if the resources of the era of LLM-based agents are used in the right way. I assume OpenAI plans to do this, possibly Conjecture's plan falls under this description, and maybe Anthropic could do it too. And then there's Orthogonal, who are just trying to figure out the theory, with or without AI assistance. 

Unknown unknowns may invalidate some or all of this scenario. :-)