Live Theory Part 0: Taking Intelligence Seriously

post by Sahil · 2024-06-26T21:37:10.479Z · LW · GW · 3 comments

Contents

      Acknowledgements
      Disclaimers
  Taking Intelligence Seriously
  Failing to Take Intelligence Seriously
  Opportunities from Moderate Intelligence at Scale
  Live Interfaces
  Live Theory (preface)
    Navigation
  Live Theory (teaser)
    Possibility
        ^
        ^
        ^
        ^
        ^
        ^
        ^
        ^
        ^
        ^
        ^
        ^
        ^
None
3 comments

Acknowledgements

The vision here was midwifed originally in the wild and gentle radiance that is Abram [LW · GW]'s company (though essentially none of the content is explicitly his). 

The PIBBSS [? · GW]-spirit has been infused in this work from before it began (may it infuse us all), as have meetings with the Agent Foundations team at MIRI over the past ~2 years. 

More recently, everyone who has been loving the High Actuation project into form (very often spontaneously and without being encumbered by self-consciousness of this fact):[1] individuals include Steve Petersen, Mateusz Baginski, Aditya Prasad, Harmony, TJ, Chris Lakin; the AISC 2024 team, Murray Buchanan, Matt Farr [LW · GW], Arpan Agrawal, Adam, Ryan, Quinn; various people from Topos, ALIFE, MAPLE, MATS, EA Bangalore. Published while at CEEALAR.

Disclaimers

 

 

 

 

Taking Intelligence Seriously

Sahil:

I gave a talk recently, at an EA event just two days ago, where I made some quick slides (on the day of the talk, so not nearly as tidy as I’d like) and attempted to walk through this so-called “live theory”. (Alternative terms include “adaptive theory” or “fluid theory”; where the theories themselves are imbued with some intelligence.)

Maybe I can give you that talk. I'm not sure how much of what I was saying there will be present now, but I can try. What do you think? I think it'll take about 15 minutes. Yeah?

Steve: 

Cool. 

Sahil:

Okay, let me give you a version of this talk that's very abbreviated.

So, the title I’m sure already makes sense to you, Steve. I don't know if this is something that you know, but I prefer the word “adaptivity” over intelligence. I'm fine with using “intelligence” for this talk, but really, when I'm thinking of AI and LLMs and “live” (as you’ll see later), I'm thinking, in part, of adaptive. And I think that connotes much more of the relevant phenomena, and much less controversially. 

It’s also less distractingly “foundational”, in the sense of endless questions on “what intelligence means”.

 

 

Failing to Take Intelligence Seriously

Right. So, I want to say there are two ways to fail to take intelligence, or adaptivity, seriously.

One is, you know, the classic case, of people ignoring existential risk from artificial intelligence. The old “well, it's just a computer, just software. What's the big deal? We can turn it off.” We all know the story there. In many ways, this particular failure-of-imagination is much less pronounced today.

But, I say, a dual failure-of-imagination is true today even among the “cognoscenti”, where we ignore intelligence by ignoring opportunities from moderately capable mindlike entities at scale. I'll go over this sentence slower in the next slide.

For now: there are two ways to not meet reality. 

On the left of the slide is “nothing will change”. The same “classic” case of “yeah, what's the big deal? It's just software.”

On the right, it's the total singularity, of extreme unknowable super-intelligence. In fact, the phrase “technological singularity”, IIRC, was coined by Vernor Vinge to mark the point that we can't predict beyond. So, it's also a way to be mind-killed. Even with whatever in-the-limit proxies we have for this, we make various simplifications that are not “approximately” useful; they don’t decay gracefully. (Indeed, this is how the “high-actuation spaces” project document starts.)

All of the richness of reality: that’s in the middle.

Steve:

I think that makes sense. I like how there are both ways to avoid looking at the kind of intelligence in between, given slow takeoff.

Sahil:

Yeah, cool. In the event of a “slow takeoff”, sure.[2]

 

 

Opportunities from Moderate Intelligence at Scale

Okay, so, going over the sentence slowly:

  1. I mean opportunities, so the focus for today is less about risks. We're talking about reframing risks tomorrow anyway, in the series of conversations we're having. However, conversation about risk will end up here as well, because it is an inextricable aspect of the opportunity frame. Indeed, “live theory” is about (research methodological) opportunities meeting risks (of infrastructural insensitivity towards beings).[3]
  2. “Moderate”, as in: intelligence that is not super intelligent or singularity-level intelligent, but kind of intelligent, the way LLMs seem kind of intelligent, and somewhat beyond.[2]
  3. “Intelligence” meaning, really, adaptivity again.
  4. And at scale, meaning: everywhere, cheap, abundant, etc. The analogy I tend to use: attempts to talk about video calls, or even, say, remote teaching when all we currently have to extrapolate from is the telegraph/telegram. With the technology of the telegraph, people can tell that being able to send messages over large distances seems exciting. But they're not really thinking “oh, in the future, some jobs will entirely be dependent on being able to take video-calling tech for granted. Schoolkids might do the entirety of their schooling using this tech.” They just didn't think of that, for the most part. Even if some people had ideas, few were working towards orienting to that future in a way that would enable better equilibria. They failed to take it seriously.

    Video calling is in a way the same type of technology as telegraph: the ability to send messages. But with incredibly reduced cost, latency plus increased adoption, bandwidth, fidelity. This allowed for remote work and video calls and the whole shebang that’s allowing us to have this conversation right now.

    And so the question is: what happens when we have much lower latency and moderately more intelligence, much lower costs, with adaptivity tightly integrated in our infrastructure?[4] Just like we have Wi-Fi in everything now.

And notice that this is not extrapolation that goes only moderately far. That is, just because I'm talking about “moderate” intelligence does not mean this extrapolation is not about the real crazy future up ahead. Only, this is AI-infrastructural extrapolation, not AI-singleton extrapolation (or even what’s connoted by “multipolar”, usually, IME). It’s neglected because it is usually harder to think about, than a relatively centralized thing we can hold in attention one at a time. 

This frame also naturally engages more with the warning carried in “attitudes to tech overestimate in the short run, underestimate in the long run.”

So to repeat, and this is important for sensemaking this: I am doing extrapolation that will venture far. What follows is simultaneously very obvious and very weird. In fact, that combination is what makes it work at all, as you’ll see.

But that’s also a double-edged sword. Instead of it being sensible (obvious) and exciting (weird), the perspective here might seem redundant or boring (too obvious) and irrelevant or scary (too weird). 

Hopefully, I will:
a) avoid its obviousness being rounded off to “ah right, live theory is another word for autoformalization” and
b) bring its weirdness closer to digestibility. To quote Bostrom in Deep Utopia, “if it isn’t radical, it isn’t realistic.”

So even though some might classify this series as being about "automating[5] alignment research", it tastes nothing like unfleshed mundane trendlines[6] or spectacular terror that are mixed together, for example, in Leopold Aschenbrenner’s “AGI from automated research by 2027”. 

Again, this isn't to say that there aren't some serious risks, only that they might look very different (which view will be elaborated in an upcoming post). [7]

 



Live Interfaces

This slide was ‘Live UI’ (don't bother trying to read the image): what happens to infrastructure and interfaces, generally, when you can “do things that don’t scale”, at scale. People don’t seem to update hard enough, based on their reactions at Sora etc, on what the future will take for granted. 

What is possible, when all this is fast, cheap, abundant, reliable, sensitive? Live UI seeks to chart this out. The six pillars, without much explanation for now, are:

 

Steve:

Am I supposed to be following all of that?

Sahil:

Definitely not, it's just a trailer of sorts. I've included only a relatively accessible example each (it gets way weirder), but there are volumes to say for the pillars, especially to gesture at how it all works together. (Btw: reach out if you're interested to know more or work together!)

A bit more, before we move on though. 

Nearly all of the above is about sensitivity and integration[9] as we gain increasing choice in adaptive constructability at scale. 

The above could, instead, already sound nauseating, atomizing, and terrifying. A good time, then, to meet the key concern of the High Actuation agenda: the applied metaphysics of cultivating wise and lasting friendship in a reality full of constructs. 

High Actuation is the research context (for both Live UI & Live Theory, among several other tracks), where the physical world becomes more and more mindlike through increased and ubiquitous adaptivity, in the process challenging a lot of (dualistic[10]) assumptions about mind and matter, and how to go about reasoning about them.[11]

But yeah, don't worry about this interface-6-pillars stuff above. I'm going to talk about what I’ll be focusing on building tools (and hiring!) for, in the coming months: intelligent science infrastructure.

 

 

Live Theory (preface)

So the boring way to think about intelligent science infrastructure is to say “AI will do the science and math automatically.”

(What does it really mean, to say “automatically”? We’ll get to that.)

First, a succinct unrolling of the whole vision. A series of one-liners follow, with increasing resolution on the part of this dream that matters. The italics signify the emphasis in each stage of clarification.

Live theory is...

(Here "theories" are only one kind of distributable artefact in research to think about.)

But more importantly, the vision is...

(Here "artefacts" includes papers, theories, explainers, code, interfaces, laws[12], norms etc.)

But more importantly, the vision is...

(Here "protocols" don’t need to be a fixed/formal protocol or platform either!)

But really, the vision is...

(Here “lending spirit” being the crucial activity that allows for spacious resolution towards better equilibria.)

 

Some navigation before the longer conversation around this: there are four gates that we'll have to pass through, as you see on the slide below, to come to terms with the proposal.

Each of the four gates has been invariably challenging to convey and invite people through (although it is satisfying to witness people's fluency after a few runs through them):

  1. Possible (what Live Theory is and whether it is even conceivable/tractable)
  2. Bearable (how, paradoxically, Live Theory has always been true, and whether we can stand it being true already)
  3. Desirable (what new modes open up when we incorporate Live Theory-like adaptivity)
  4. Necessary (Live Theory's suggestions for threat models from extremely adaptive AI, like in Deep Deceptiveness [LW · GW] and Robust Agent Agnostic Processes [LW · GW]. [13])

I'm going to walk through these gates, and conclude, and that will be the talk.

This decomposition into gates should hopefully make it easier to critique the whole thing—eg. “X aspect is undesirable because…” vs “X aspect is impossible because…”. The two obviously deserve very different kinds of responses.

I’m offering the talk this way for many reasons, but to say a bit more. Most of the work is in a) noticing the circumrational water of mathematics as it is today (which can be too obvious for mathematicians and too unbearable for math-groupies respectively) and b) connecting it to mathematics as it might become in the near future (which can seem too bizarre or undesirable, if you don’t notice its importance in mathematics as it is today). When new paradigms start being Steam [LW · GW]’d, they often have to pull off a similar straddling of the familiar and the unfamiliar. Not too different from the ordeal of hitting the edges of one’s developmental stage… but at a sociotechnical/civilizational level.

If making it easy for you to respond and critique were the only goal of the gates, they would have been set out in a tidier decomposition. However, in tandem, I'm using the gates-structure to construct a “natural walk”, a narrative arc, through the details of Live Theory. This polytely has some tradeoffs (such as the Bearable and Desirable gates not quite disentangling), but I think it works! Let me know.

The next post will cover the first two gates. A teaser slide for the first one follows.

Live Theory (teaser)

Possibility

 

And a teaser of two questions I will start the next post with, but also include now to give you time to actually think:

 

 

 

 

 

 

  1. ^

    Challenging and supporting, especially through the frustration of freezing this written artefact before I can avail the spaciousness of the fluidic era.

  2. ^

    This will be elaborated in the upcoming post on risks. “Slow takeoff” is an approximation that will have to do for now.

  3. ^

    Generally, it is a bit suspect to me, for an upcoming AI safety org (in this time of AI safety org explosion) to have, say, a 10 year timeline premise without anticipating and incorporating possibilities of AI transforming your (research) methodology 3-5 years from now. If you expect things to move quickly, why are you ignoring that things will move quickly? If you expect more wish-fulfilling devices to populate the world (in the meanwhile, before catastrophe), why aren't you wishing more, and prudently? An “opportunity model” is as indispensable as a threat model.

    (In fact, "research methodological IDA" is not a bad summary of live theory, if you brush under the rug all the ontological shifts involved.)

  4. ^

    TJ comments:

    I'd guess you'd want to be more explicit here that what you mean is that "human cognition, thought, and intellectual enterprise is itself going to  go through radical transitions, and will reshape the human experience." It feels like you're implying this here, but is left as a silent gesture for the reader.

  5. ^

    The word “automation” does not distinguish numb-scripted-dissociated from responsive-intelligent-flowing. It usually brings to mind the former. So I avoid using it. More on this in the next post, but for now, a word might be “competence”. When you’re really in flow while dancing, you're not thinking. But it is the opposite of “automation”. If anything, you're more attuned, more sensitive. 

    This isn't “high-level” abstraction either. The high-low dichotomy is the good-old-fashioned way of looking at generalization, and does not apply with expanded sensitivity, as we'll see in this sequence.

    The relevance of the choice "live" will hopefully also become clearer as we go. Meant to connote biological rather than machine metaphors, dialectics rather than deliverables, intentional stance, though not necessarily patienthood.

  6. ^

    It's not that people predict "no fundamentally new ways of looking at the world will occur" and therefore decide to exclude them in their stories. I think people exclude ontological shifts because it's very hard to anticipate an ontological shift. 

    (If you disagree, I'd ask the fundamental question of implied invisible work: what have you rejected?)

  7. ^

    Matt adds:

    It might be worth stating more about these [risks] from the outset. Just to prevent anyone thinking you're some kind of e/acc full steamer techbro who isn't interested in safety

    Indeed. The Necessary gate will cover much relevance to threat models. Apart from that, expect much much more on "infrastructural insensitivity" to articulate further vectors of risk.

  8. ^

    Abram helpfully adds:

    I find the meeting example [see next quote block] much more compelling, where people get dynamically crafted versions of the meeting which can catch them up on the most important things they missed as a result of coming in late -- someone uses the phrase "lightbulb moment" in a context-dependent way, so the live interop unpacks the context for those who missed it. 

    I think the reason I like this better is that meetings invoke a more utilitarian mindset where we evaluate the usefulness of tools. Movies invoke a warm friendly mindset where we are more prone to horror over the seeming non-togetherness of the described scenario. If you want to shock people with the idea of such severely mediated reality, fine. But I think the meeting example will provoke less feeling of shock and undesirability, if you prefer that.

    And so here is my meeting example pasted raw, which might be more palatable or more triggering:

    So imagine you join a meeting on Zoom, a little late. AI summarizers can "catch you up". Instead of immediately chatting, you peruse the transcript/read the summary.

    Now that's great, but a little boring. Ideally you'd have "immersive summary", a mini-meeting that you get to see live, a generated video summary. You watch that, and then join the real meeting.

    Then you seamlessly transfer to the "real" meeting... except when Alice references a detail that wasn't given enough attention in the summary-virtual-meeting you initially watched to catch up. So before you get confused as Alice starts using the details assuming you were in the meeting, the AI interop again pulls you back into a simulation where generated Alice says "so, just to recall, some details I want to add" before dropping you back into reality.

    And in fact, your replica will already have joined the meeting even though you were 10 minutes late. And you watch it, and might say "oh, no, that's not what I would have said"... and so, once you say that, everyone else will be put in a simulated meeting, where your replica says "I know I said earlier that X, but no, [I take that back/actually meant/wanted to say] ..." so everyone is again caught up.

    Rather than being blocky or jarring, imagine a perfect stabilizer, like in a drone or in video-correction. It's uncanny, that despite all the noises, rapid responses can correct destabilizations quite magically and beautifully into a coherence.

    This again blurs "construction" and "reality", doing such a fantastically better job that you will only occasionally need to rebel against this matrix. The real-constructed dichotomy makes less and less sense.

  9. ^

    NB. this is not brute, ham-fisted merging, for those worried. See also: Unity and diversity.

  10. ^

    also: centralization-heavy, preformationist, foundationalist, control-oriented

  11. ^

    This is a very expensive delivery mechanism for a koan/pointing-out instruction, to let go of the static-machine mythos, but there you go.

  12. ^

    Live Governance post coming up soon.

  13. ^

    See also this proposal for the commonalities in the two posts and limitations of existing approaches.

     

3 comments

Comments sorted by top scores.

comment by Viliam · 2024-06-27T14:09:20.106Z · LW(p) · GW(p)

I agree that many people focus on "superhuman intelligence, sooner or later or never" and ignore the possibility of how a human-level or even below-human-level intelligence can change the world if it becomes cheap enough.

I heard a rumor about someone who made lots of money by telling an AI to generate thousands of e-books and selling them on Amazon. Previously people did a similar thing by scraping Wikipedia pages and selling them as books, but this is different -- the books are not obviously scam, they are probably just mediocre; and as the AIs get better, the new books will get better, until enough people realize they do not need to buy them, because they can ask an AI directly.

I imagine similar "shovelware" everywhere. There are probably already many people accepting programming or artistic tasks online, giving them to an AI, and getting paid for the result. If you want to make money this way, do it quickly, because at some moment people will realize they can ask an AI directly.

This may be a convenient time to establish fake artistic credentials. Create a page on Patreon, and ask an AI to make deliberately bad pictures... gradually improving, so that people can see your artistic progress. Later, when the backlash against AI art comes, people will stop trusting art from random strangers online, but you will have established credentials as an actual human artist, so you can charge extra for "hand made" art.

At some moment, software development will become "put an AI into a box, and tell it what to do". For example, to program a new Tetris game, you will simply take a generic AI and tell it "do Tetris"; and that's all. To create an entirely new game, you will provide a verbal explanations, a few example screenshots (generated by another AI), and tell the AI "do this".

There will be many "experts" online who will basically be an interface between you and an AI. You will call a human doctor or a financial advisor or a motivational coach, and the person will listen to you, while silently typing questions to an AI, and then will tell you the results, and charge you $100 per hour.

If many cheap appliances get an AI component, I expect most things to be commanded by voice. An AI with a camera will be a powerful combination, the appliance will be able to see what is there, and make some logical conclusion. Your gas stove will tell you "I think the food might be ready, would you like to check it?", your fridge will remind you that you are out of butter, and by the way the box of milk is already open for a week you might want to check it. Toys will talk to children, and accept verbal instructions.

...and this all will be used for surveillance, because of course an intelligent device can provide much better information about you. Not just what you do, but whether you smile or frown when you do it; whether you also use devices made by competition; whether it seems like you have extra money to buy an upgrade; what other devices are missing at your home.

...also, advertising everywhere. The toys will tell your kids to buy merchandise. The kitchen appliances will recommend you to buy food from specific distributors. The computer games will model their villains based on politicians who propose to regulate advertising and surveillance.

Online, it will be possible to create a cheap "Matrix" for people. Imagine being on a web forum where everyone else is a bot, and you are the only human there. But you will never know, because the bots will post interesting things, debate with you, flirt with you, invite you to read other websites... and occasionally do the thing they were made for, like try to make you buy some products or vote for a certain party. You will be a follower of a conspiracy theory that was tailor-made for you; thousands bots coming with random ideas, and whatever you upvote, there will be more of that, and whatever you write, will be incorporated in the theory, as long as it can be made compatible with the main goal. People online will never feel alone again. At the beginning, this will be an operation targeted at specific people (they may be met by actual humans who introduce them to a web forum where they receive a warm approval from bots, and then everything will be handled by the bots), but as it gets cheaper, they will try to catch everyone in their own separate "Matrix".

Even the real people you meet will often be working for similar companies -- their role will be to sometimes meet you offline, to provide evidence that they are actual humans, but as soon as you return to the screen, you will only interact with a bot that simulates them. So once in a year your private "Matrix" can have an offline meetup, where you meet dozen people from a specialized agency who will role-play members of your web forum. (Their job is to go to a different meetup every week with a different identity.)

Or to turn it around, people will use AIs to pretend to be them, and outsource the tasks such as calling your parents, or maintaining contacts with your former classmates and colleagues. Hey, it costs nothing to let an AI handle a personal relationship you don't really care about, and you never know when you can get some useful service from "an old friend". The future of seduction is to have a bot seduce someone and arrange a date, and you just go there to have sex with them. The future of prostitution is the same, only you also let the bot find out some pretext why the other person should send you money. The girlfriend you once met at a party, and since then you call each other every day and talk a lot about various topics and she is really smart and nice, but she is too busy with her work and she is living in another city, and how she has a financial problem, but if you send her some money she will be able to quit her current job and move closer to you... I am sorry, but she is not real. She is just a girl who goes to parties, collects phone numbers from guys and enters them in a computer; that's all, everything that happened afterwards was fake.

comment by Jonas Hallgren · 2024-06-27T08:13:20.798Z · LW(p) · GW(p)

Very intriguing, excited for the next post!

(We will watch your career with great interest.)

comment by Steven (steven-1) · 2024-07-02T03:11:45.583Z · LW(p) · GW(p)
  • Q. What is generalization really for? What does it offer you?


Based on the vibe of the post, it seems like you're trying to point at the concept of "being able to do many things". I guess generalization isn't 'for' anything, it's a concept. For an agent, generalization is a method of being able to achieve an outcome based on limited past experience without needing to waste resources figuring out strategies it could have made if only it could generalize better. I can't really tell based on what you said what I'm supposed to answer with "What does it offer you?". Like, generalization offers me the ability to recognize bad chess moves in new scenarios that I haven't seen, or it offers me the ability to take over the universe based on limited knowledge of physics. I don't know where you're trying to limit the word