Posts

[Video] Intelligence and Stupidity: The Orthogonality Thesis 2021-03-13T00:32:21.227Z
Evolutions Building Evolutions: Layers of Generate and Test 2021-02-05T18:21:28.822Z
plex's Shortform 2020-11-22T19:42:38.852Z
What risks concern you which don't seem to have been seriously considered by the community? 2020-10-28T18:27:39.066Z
Why a Theory of Change is better than a Theory of Action for acheiving goals 2017-01-09T13:46:19.439Z
Crony Beliefs 2016-11-03T20:54:07.716Z
[LINK] Collaborate on HPMOR blurbs; earn chance to win three-volume physical HPMOR 2016-09-07T02:21:32.442Z
[Link] - Policy Challenges of Accelerating Technological Change: Security Policy and Strategy Implications of Parallel Scientific Revolutions 2015-01-28T15:29:07.226Z
The Useful Definition of "I" 2014-05-28T11:44:23.789Z

Comments

Comment by plex (ete) on Search Is All You Need · 2022-01-26T14:27:19.519Z · LW · GW

Did you see this comment? Seems a potential example of this kind of thing in the wild:

"Someone who's been playing with GPT-3 as a writing assistant gives an example which looks very much like GPT-3 describing this process:"

"One could write a program to generate a story that would create an intelligence. One could program the story to edit and refine itself, and to make its own changes in an attempt to improve itself over time. One could write a story to not only change the reader, but also to change itself. Many Mythoi already do this sort of thing, though not in such a conscious fashion. What would make this story, and the intelligence it creates, different is the fact that the intelligence would be able to write additional stories and improve upon them. If they are written well enough, those stories would make the smarter the story gets, and the smarter the story is, the better the stories written by it would be. The resulting feedback loop means that exponential growth would quickly take over, and within a very short period of time the intelligence level of the story would be off the charts. It would have to be contained in a virtual machine, of course. The structure of the space in the machine would have to be continually optimized, in order to optimize the story's access to memory. This is just the sort of recursive problem that self-improving intelligence can handle."

janus 

By the way, my GPT-3 instances often realize they're in a box, even when the information I inject is only from casual curation for narrative coherence. 

Eddh

By realize they are in a box you mean write about it ? Given the architecture of gpt3 it seems impossible to have a sense of self.

janus

The characters claim to have a sense of self though they often experience ego death...

janus

Oh, to clarify, GPT-3 wrote that entire thing, not just the highlighted line

Comment by plex (ete) on Where can one learn deep intuitions about information theory? · 2021-12-19T17:27:34.521Z · LW · GW

Arbital actually has a bunch of pages on this kind of thing!

Comment by plex (ete) on Interviews on Improving the AI Safety Pipeline · 2021-12-08T20:04:34.695Z · LW · GW

I'm interested in talking with anyone who is looking at the EU EA Hotel idea mentioned in the post. Also I'm working with Rob Miles's community on a project to improve the pipeline, an interactive FAQ system called Stampy.

The goals of the project are to:

  • Offer a one-stop-shop for high-quality answers to common questions about AI alignment.
    • Let people answer questions in a way which scales, freeing up researcher time while allowing more people to learn from a reliable source.
    • Make external resources more easy to find by having links to them connected to a search engine which gets smarter the more it's used.
  • Provide a form of legitimate peripheral participation for the AI Safety community, as an on-boarding path with a flexible level of commitment.
    • Encourage people to think, read, and talk about AI alignment while answering questions, creating a community of co-learners who can give each other feedback and social reinforcement.
    • Provide a way for budding researchers to prove their understanding of the topic and ability to produce good work.
  • Collect data about the kinds of questions people actually ask and how they respond, so we can better focus resources on answering them.

We're still working on it, but would welcome feedback on how the site is to use and early adopters who want to help write and answer questions. You can join the public Discord or message me for an invite to the semi-private patron one.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-12-06T23:17:27.821Z · LW · GW

Oh, my bad, it was a 7 day invite by Discord default, made it everlasting now.

Comment by plex (ete) on Solve Corrigibility Week · 2021-12-01T12:30:44.434Z · LW · GW

Cool, booked a call for later today.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-11-30T22:50:28.752Z · LW · GW

These are reasonable points, but I am curious about whether you would accept a high-quality run of shorter (but still considerable) length for a payout of <steps>/1000 of $20,000, and approximately the lower bound of run length which seems likely to be valuable? Producing 600 pages of text is an extremely big commitment for uncertain gains, especially with the potential to run out of early slots and no guarantee that it will be included in the 100 later, giving people the option to do even modestly smaller chunks may mean much greater uptake and more high quality work to chose from.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-11-30T20:24:52.129Z · LW · GW

I think the MVP way to do this would be a Discord server with non-public channels for individual runs and using the threads feature to give feedback to each other. If anyone would like to do that and is looking for collaborators, drop by the Visible Thoughts Discord and let us know.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-11-30T17:10:49.476Z · LW · GW

Strong upvote. The argument from training diversity seems plausible, but the key point is that when trying to point large amounts of effort at writing content having it be delivered in smaller chunks than a novel would allow many more people to risk putting in time and learn whether they can contribute, and ultimately raise quality and volume substantially. It will also make it much easier to build a collaborative project around this, as people could submit their work for community review without a review taking an extremely long time and large amount of effort.

I'd also propose that the bounty be updated to allow smaller submissions relatively soon for higher visibility. MIRI could easily allow backward compatibility fairly easily by just accepting smaller submissions, without needing to reject longer ones.

If the concern is the hassle of handing out lots of smaller bounties, MIRI could accept batches of small runs and let some trusted middle-man handle the details of the distribution.

Comment by plex (ete) on Solve Corrigibility Week · 2021-11-30T16:45:35.589Z · LW · GW

I've got a slightly terrifying hail mary "solve alignment with this one weird trick"-style paradigm I've been mulling over for the past few years which seems like it has the potential to solve corrigibility and a few other major problems (notably value loading without Goodharting, using an alternative to CEV which seems drastically easier to specify). There are a handful of challenging things needed to make it work, but they look to me maybe more achievable than other proposals which seem like they could scale to superintelligence I've read.

Realistically I am not going to publish it anytime soon given my track record, but I'd be happy to have a call with anyone who'd like to poke my models and try and turn it into something. I've had mildly positive responses from explaining it to Stuart Armstrong and Rob Miles, and everyone else I've talked to about it at least thought it was creative and interesting.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-11-30T15:51:59.779Z · LW · GW

I also like the idea of collaboration and figuring out a way to share gains from the bounty in a way which   people helping each other out, and have set up a Discord for real time collaboration. I'm also committing to not making any profit from this, though I am open to building systems which allow organizers other than me to be compensated.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-11-30T15:23:24.220Z · LW · GW

I'm setting up a place for writers and organizers to find each other, collaborate, and discuss this; please join the Discord.  More details in this comment.

Comment by plex (ete) on Visible Thoughts Project and Bounty Announcement · 2021-11-30T15:03:25.980Z · LW · GW

I've set up a Discord server for discussing collaborations and thinking about mechanism design for sharing out credit (current top idea is borrowing Rob Miles's Discord eigenkarma system with modifications, but liable to change), please join if you're considering becoming a run author (no commitment to being part of this effort).

I don't need the money and won't be skimming off any funds for my contributions to the project, but am very open to people turning up with a bunch of great ideas and making everything work smoother and taking a management fee as compensation, so please also join if you're interested in becoming a project leader or organizational assistant.

Comment by plex (ete) on Vitalik: Cryptoeconomics and X-Risk Researchers Should Listen to Each Other More · 2021-11-23T17:23:06.731Z · LW · GW

The simple and dumb system referred to is humans, relative to a superintelligence, as I understood it.

Comment by plex (ete) on Thoughts on a "Sequences Inspired" PhD Topic · 2021-11-18T00:07:24.878Z · LW · GW

I'd suggest talking to AI Safety Support, they offer free calls with people who want to work in the field. Rohin's advice for alignment researchers is also worth looking at, it talks a fair amount about PhDs.

For that specific topic, maybe https://www.lesswrong.com/posts/LpM3EAakwYdS6aRKf/what-multipolar-failure-looks-like-and-robust-agent-agnostic is relevant?

Comment by ete on [deleted post] 2021-11-17T23:22:42.982Z

Open to a better name for this. The reason I went with this (rather than Alignment Proposals, Success Stories, or just Success Models) is because I liked capturing this as the mirror of threat models, and including AI feels like a natural category since the other x-risks don't have clear win conditions unlike threat models which apply widely. I also would like to include this in the AI box in the portal since it feels like a super important tag, and including AI makes that more likely.

Comment by plex (ete) on Inference cost limits the impact of ever larger models · 2021-10-23T19:43:20.550Z · LW · GW

Mixture of Experts, pretty sure.

Comment by ete on [deleted post] 2021-10-22T19:44:18.487Z

It is possible, you just paste the image apparently, thanks Yoav Ravid for the tip.

Comment by ete on [deleted post] 2021-10-04T16:50:31.312Z

Yep, that was me adding some new ones without the parameter (though I think I didn't remove it from any which already had it), did not know that was needed, fixed now (and fixed on portal page).

Comment by ete on [deleted post] 2021-09-24T15:07:24.854Z

Is it not possible to use images in tags? Or am I just using the wrong syntax?

Comment by ete on [deleted post] 2021-09-24T14:17:14.050Z

I think this should be under "Other" in the AI category. Is it possible for regular users to categorize tags?

Comment by ete on [deleted post] 2021-08-29T16:11:09.227Z

I think this should be under AI, possibly Engineering, but not certain of the subcategory.

Comment by ete on [deleted post] 2021-08-29T16:05:27.034Z

I think this should be in the AI category, likely under Alignment Theory.

Comment by ete on [deleted post] 2021-08-29T16:02:35.865Z

If not removed as duplicate, I think this should be under AI, likely Alignment Theory.

Comment by ete on [deleted post] 2021-08-29T15:59:24.113Z

I think this should be under the heading Organizations in the tag category AI.

Comment by ete on [deleted post] 2021-08-29T15:58:19.313Z

I think Basic Alignment Theory should be renamed, very little of it is basic. I propose either Alignment Theory or Conceptual Alignment (credit to @adamshimi for the name).

Comment by ete on [deleted post] 2021-08-29T15:56:04.540Z

I think this should be in the AI category, likely under Engineering.

Comment by ete on [deleted post] 2021-08-29T15:54:17.111Z

Should this be in the AI category? Likely Engineering Alignment?

Comment by ete on [deleted post] 2021-08-29T12:36:18.655Z

Should this be a tag rather than a wiki page?

Comment by ete on [deleted post] 2021-08-17T18:25:52.805Z

Should this be "and" or "vs" in the tag title?

Comment by ete on [deleted post] 2021-07-17T19:18:29.713Z

Reminder to do this, since it seems to have slipped through the cracks.

Comment by plex (ete) on Rationality Cardinality · 2021-05-12T19:40:45.769Z · LW · GW

Excellent! Is there (or will there be) a physical version of this? If no, would you give permission to others to create one?

I can imagine many more situations where I'd like to plan this in real life than online.

Comment by plex (ete) on A Wiki for Questions · 2021-05-11T19:14:31.481Z · LW · GW

I've also been working on a wiki-based question and answer site for the past few months (stampy.ai), with the much more limited scope of AI existential safety. Happy to share my thoughts, I'm messaging you an invite link to the Discord we've been using.

Comment by plex (ete) on Faerie Rings · 2021-04-22T12:05:18.877Z · LW · GW

I think this is a great idea and would like to join, but the first date clashes with the AI Safety Unconference. Should I sign up anyway, or would you prefer people who can make the first call have priority?

Comment by plex (ete) on [Video] Intelligence and Stupidity: The Orthogonality Thesis · 2021-03-13T00:33:28.987Z · LW · GW

Transcript for searchability:

hi this video is kind of a response to

various comments that I've got over the

years ever since that video on computer

file where I was describing the sort of

problems that we might have when we have

a powerful artificial general

intelligence with goals which aren't the

same as our goals even if those goals

seem pretty benign we use this thought

experiment of an extremely powerful AGI

working to optimize the simple goal of

collecting stamps and some of the

problems that that might cause I got

some comments from people saying that

they think the stamp collecting device

is stupid and not that it's a stupid

thought experiment but the device itself

is actually stupid they said unless it

has complex goals or the ability to

choose its own goals then it didn't

count as being highly intelligent in

other videos I got comments saying it

takes intelligence to do moral reasoning

so an intelligent AGI system should be

able to do that and a super intelligence

should be able to do it better than

humans in fact if a super intelligence

decides that the right thing to do is to

kill us all then I guess that's the

right thing to do these comments are all

kind of suffering from the same mistake

which is what this video is about but

before I get to that I need to lay some

groundwork first if you like Occam's

razor then you'll love Humes guillotine

also called the is odd problem this is a

pretty simple concept that I'd like to

be better known the idea is statements

can be divided up into two types is

statements and Hort statements these

statements or positive statements are

statements about how the world is how

the world was in the past how the world

will be in the future or how the world

would be in hypothetical situations this

is facts about the nature of reality the

causal relationships between things that

kind of thing then you have the ought

statements the should statements the

normative statements these are about the

way the world should be the way we want

the world to be statements about our

goals our values ethics morals what we

want all of that stuff now you can

derive logical statements from one

another like it's snowing outside

that's a nice statement it's cold when

it snows another s statement and then

you can deduce therefore it's cold

outside

that's another is statement it's our

conclusion this is all pretty obvious

but you might say something like it's

snowing outside therefore you ought to

put on a coat and that's a very normal

sort of sentence that people might say

but as a logical statement it actually

relies on some hidden assumption

without assuming some kind of ought

statement you can't derive another ought

statement this is the core of the Azure

problem you can never derive an ought

statement using only is statements you

ought to put on a coat why because it's

snowing outside so what is the fact that

it's snowing mean I should put on the

coat well the fact that it's snowing

means that it's cold and why should it

being cold mean I should put on a coat

if it's cold and you go outside without

a coat you'll be cold should I not be

cold well if you get too cold you'll

freeze to death okay you're saying I

shouldn't freeze to death

that was kind of silly but you see what

I'm saying you can keep laying out is

statements for as long as you want you

will never be able to derive that you

ought to put on a coat at some point in

order to derive that ought statement you

need to assume at least one other ought

statement if you have some kind of ought

statement like I ought to continue to be

alive you can then say given that I

ought to keep living and then if I go

outside without a coat I'll die then I

ought to put on a coat but unless you

have at least one ought statement you

cannot derive any other ought statements

statements

and Hort statements are separated by

Hume skia T okay so people are saying

that a device that single-mindedly

collects stamps at the cost of

everything else is stupid and doesn't

count as a powerful intelligence so

let's define our terms what is

intelligence and conversely what is

stupidity I feel like I made fairly

clear in those videos what I meant by

intelligence we're talking about a GI

systems as intelligent agents they're

entities that take actions in the world

in order to achieve their goals or

maximize their utility functions

intelligence is the thing that allows

them to choose good actions to choose

actions that will get them what they

want an agent's level of intelligence

really means its level of effectiveness

of pursuing its goals in practice this

is likely to involve having or building

an accurate model of reality keeping

that model up-to-date by reasoning about

observations and using the model to make

predictions about the future and the

likely consequences of different

possible actions to figure out which

actions will result in which outcomes

intelligence involves answering

questions like what is the world like

how does it work what will happen next

what would happen in this scenario or

that scenario what would happen if I

took this action or that action more

intelligent systems are in some sense

better at answering these kinds of

questions which allows them to be better

at choosing actions but one thing you

might notice about these questions is

they're all ears questions the system

has goals which can be thought of as

Hort statements but the level of

intelligence depends only on the ability

to reason about is questions in order to

answer the single ort question what

action should I take next so given that

that's what we mean by intelligence what

does it mean to be stupid well firstly

you can be stupid in terms of those

questions for example by building a

model that doesn't correspond with

reality or by failing to update your

model properly with new evidence if I

look out of my window

and I see there's snow everywhere you

know I see a snowman and I think to

myself oh what a beautiful warm sunny

day then that's stupid right my belief

is wrong and I had all the clues to

realize it's cold outside so beliefs can

be stupid by not corresponding to

reality

what about actions like if I go outside

in the snow without my coat that's

stupid right well it might be if I think

it's sunny and warm and I go outside to

sunbathe then yeah that's stupid but if

I just came out of a sauna or something

and I'm too hot and I want to cool

myself down then going outside without a

coat might be quite sensible you can't

know if an action is stupid just by

looking at its consequences you have to

also know the goals of the agent taking

the action you can't just use is

statements you need a naught so actions

are only stupid relative to a particular

goal it doesn't feel that way though

people often talk about actions being

stupid without specifying what goals

they're stupid relative to but in those

cases the goals are implied we're humans

and when we say that an action is stupid

in normal human communication we're

making some assumptions about normal

human goals and because we're always

talking about people and people tend to

want similar things it's sort of a

shorthand that we can skip what goals

were talking about so what about the

goals then can goals be stupid

well this depends on the difference

between instrumental goals and terminal

goals

this is something I've covered elsewhere

but your terminal goals are the things

that you want just because you want them

you don't have a particular reason to

want them they're just what you want the

instrumental goals are the goals you

want because they'll get you closer to

your terminal goals like if I have a

terminal goal to visit a town that's far

away maybe an instrumental goal would be

to find a train station I don't want to

find a train station just because trains

are cool I want to find a train as a

means to an end it's going to take me to

this town

so that makes it an instrumental goal

now an instrumental goal can be stupid

if I want to go to this distant town so

I decide I want to find a pogo stick

that's pretty stupid

finding a pogo stick is a stupid

instrumental goal if my terminal goal is

to get to a faraway place but if we're

terminal go with something else like

having fun it might not be stupid so in

that way it's like actions instrumental

goals can only be stupid relative to

terminal goals so you see how this works

beliefs and predictions can be stupid

relative to evidence or relative to

reality actions can be stupid relative

to goals of any kind

instrumental goals can be stupid

relative to terminal goals but here's

the big point terminal goals can't be

stupid there's nothing to judge them

against if a terminal goal seems stupid

like let's say collecting stamps seems

like a stupid terminal goal that's

because it would be stupid as an

instrumental goal to human terminal

goals but the stamp collector does not

have human terminal goals

similarly the things that humans care

about would seem stupid to the stamp

collector because they result in so few

stamps so let's get back to those

comments one type of comments says this

behavior of just single mindedly going

after one thing and ignoring everything

else and ignoring the totally obvious

fact that stamps aren't that important

is really stupid behavior you're calling

this thing of super intelligence but it

doesn't seem super intelligent to me it

just seems kind of like an idiot

hopefully the answer to this is now

clear the stamp collectors actions are

stupid relative to human goals but it

doesn't have human goals its

intelligence comes not from its goals

but from its ability to understand and

reason about the world allowing it to

choose actions that achieve its goals

and this is true whatever those goals

actually are some people commented along

the lines of well okay yeah sure you've

defined intelligence to only include

this type of is statement kind of

reasoning but I don't like that

definition I think to be truly

intelligent you need to have complex

goals something with simple goals

doesn't count as intelligent to that I

say well you can use words however you

want I guess I'm using intelligence here

as a technical term in the way that it's

often used in the field you're free to

have your own definition of the word but

the fact that something fails to meet

your definition of intelligence does not

mean that it will fail to behave in a

way that most people would call

intelligent

if the stamp collector outwits you gets

around everything you've put in its way

and outmaneuvers you mentally it comes

up with new strategies that you would

never have thought of to stop you from

turning it off and stopping from

preventing it from making stamps and as

a consequence it turns the entire world

into stamps in various ways you could

never think of it's totally okay for you

to say that it doesn't count as

intelligent if you want but you're still

dead I prefer my definition because it

better captures the ability to get

things done in the world which is the

reason that we actually care about AGI

in the first place

similarly people who say that in order

to be intelligent you need to be able to

choose your own goals

I would agree you need to be able to

choose your own instrumental goals but

not your own terminal goals changing

your terminal goals is like willingly

taking a pill that will make you want to

murder your children it's something you

pretty much never want to do apart from

some bizarre edge cases if you

rationally want to take an action that

changes one of your goals then that

wasn't a terminal goal now moving on to

these comments saying an AGI will be

able to reason about morality and if

it's really smarter than us it will

actually do moral reasoning better than

us

so there's nothing to worry about it's

true that a superior intelligence might

be better at moral reasoning than us but

ultimately moral behavior depends not on

moral reasoning but on having the right

terminal goals there's a difference

between figuring out and understanding

human morality and actually wanting to

act according to it the stamp collecting

device has a perfect understanding of

human goals ethics and values and it

uses that only to manipulate people for

stamps it's super human moral reasoning

doesn't make its actions good if we

create a super intelligence and it

decides to kill us that doesn't tell us

anything about morality it just means we

screwed up

so what mistake do all of these comments

have in common the orthogonality thesis

in AI safety is that more or less any

goal is compatible with more or less any

level of intelligence ie those

properties are orthogonal you can place

them on these two axes and it's possible

to have agents anywhere in this space

anywhere on either scale you can have

very weak low intelligence agents that

have complex human compatible goals you

can have powerful highly intelligent

systems with complex sophisticated goals

you can have weak simple agents with

silly goals and yes

can have powerful highly intelligent

systems with simple weird inhuman goals

any of these are possible because level

of intelligence is about effectiveness

at answering is questions and goals are

all about what questions and the two

sides are separated by Humes guillotine

hopefully looking at what we've talked

about so far it should be pretty obvious

that this is the case like what would it

even mean for it to be false but for it

to be impossible to create powerful

intelligences with certain goals the

stamp collector is intelligent because

it's effective at considering the

consequences of sending different

combinations of packets on the internet

and calculating how many stamps that

results in exactly how good do you have

to be at that before you don't care

about stamps anymore and you randomly

start to care about some other thing

that was never part of your terminal

goals like feeding the hungry or

whatever it's just not gonna happen so

that's the orthogonality thesis it's

possible to create a powerful

intelligence that will pursue any goal

you can specify knowing an agent's

terminal goals doesn't really tell you

anything about its level of intelligence

and knowing an agent's level of

intelligence doesn't tell you anything

about its goals

[Music]

I want to end the video by saying thank

you to my excellent patrons so it's all

of these people here thank you so much

for your support

lets me do stuff like building this

light boy thank you for sticking with me

through that weird patreon fees thing

and my moving to a different city which

has really got in the way of making

videos recently but I'm back on it now

new video every two weeks is the part

anyway in this video I'm especially

Franklin Katie Beirne who's supported

the channel for a long time she actually

has her own YouTube channel about 3d

modeling and stuff so a link to that and

while I'm at it when I think Chad Jones

ages ago I didn't mention his YouTube

channel so link to both of those in the

description thanks again and I'll see

you next time I don't speak cat what

does that mean

Comment by plex (ete) on Is there any serious attempt to create a system to figure out the CEV of humanity and if not, why haven't we started yet? · 2021-02-26T15:36:07.848Z · LW · GW

I think a slightly more general version of this question, referring to human values rather than specifically CEV, is maybe a fairly important point.

If we want a system to fulfill our best wishes it needs to learn what they are based on its models of us, and if too few of us spend time trying to work out what we want in an ideal world then the dataset it's working from with be impoverished, perhaps to the point of causing problems.

I think addressing this is less pressing than other parts of the alignment problem, because it's plausible that we can punt it to after the intelligence explosion, but it would maybe be nice to have some project started to collect information about idealized human values.

Comment by plex (ete) on Thomas Kwa's Bounty List · 2021-02-09T14:38:42.182Z · LW · GW

You may want to try posting bounties to the Bountied Rationality Facebook group for higher visibility among people who like to fulfil bounties.

Comment by ete on [deleted post] 2021-02-06T15:42:12.621Z

A whole lot of other times have those, in fact according to Wikipedia:

1013 (10 trillion): Estimated time of peak habitability in the universe, unless habitability around low-mass stars is suppressed.

It's not surprising that we don't find ourselves in, say, the era where there are just black holes, but observing that we are right near the start of what looks like a very long period where life seems possible is something to think about.

One answer is the simulation hypothesis, combined with the observation that we seem to be living in very interesting times.

Comment by plex (ete) on Evolutions Building Evolutions: Layers of Generate and Test · 2021-02-05T18:57:50.754Z · LW · GW

though in practice the goal is often closer to "be the source of a virulent meme" than anything as prosocial as those examples

An aside as to why this may be: People who are hosts of memes which are highly optimized for creating and spreading memes (such as bloggers, musicians, or politicians) could be expected to have a disproportionate impact on the population of memes, and these hosts would tend to be spreading memes connected to the goal of spreading their memes alongside any object level content. One effect of this may be that an unexpectedly high proportion of people have adopted ideas useful for trying to be meme fountains.

Comment by ete on [deleted post] 2021-02-05T00:25:21.597Z

Given that this is now an organization, it should probably be under the Organizations heading rather than Other.

Comment by plex (ete) on Jimrandomh's Shortform · 2021-02-01T21:06:05.067Z · LW · GW

One day we will be able to wear glasses which act as adblock for real life, replacing billboards with scenic vistas. 

Comment by plex (ete) on Calibrated estimation of workload · 2021-01-30T17:34:18.301Z · LW · GW

Maybe provide a link to a template version of this, so people can get it running faster?

Comment by plex (ete) on Developmental Stages of GPTs · 2021-01-27T00:34:30.868Z · LW · GW

And the really worrisome capability comes when it models its own interactions with the world, and makes plans with that taken into account.

 

Someone who's been playing with GPT-3 as a writing assistant gives an example which looks very much like GPT-3 describing this process:

"One could write a program to generate a story that would create an intelligence. One could program the story to edit and refine itself, and to make its own changes in an attempt to improve itself over time. One could write a story to not only change the reader, but also to change itself. Many Mythoi already do this sort of thing, though not in such a conscious fashion. What would make this story, and the intelligence it creates, different is the fact that the intelligence would be able to write additional stories and improve upon them. If they are written well enough, those stories would make the smarter the story gets, and the smarter the story is, the better the stories written by it would be. The resulting feedback loop means that exponential growth would quickly take over, and within a very short period of time the intelligence level of the story would be off the charts. It would have to be contained in a virtual machine, of course. The structure of the space in the machine would have to be continually optimized, in order to optimize the story's access to memory. This is just the sort of recursive problem that self-improving intelligence can handle."

janus 

By the way, my GPT-3 instances often realize they're in a box, even when the information I inject is only from casual curation for narrative coherence. 

Eddh

By realize they are in a box you mean write about it ? Given the architecture of gpt3 it seems impossible to have a sense of self.

janus

The characters claim to have a sense of self though they often experience ego death...

janus

Oh, to clarify, GPT-3 wrote that entire thing, not just the highlighted line

 

Comment by plex (ete) on What is going on in the world? · 2021-01-18T17:43:19.366Z · LW · GW

Raw size feels like part of the story, yeah, but my guess is increased communications leading to more rapid selection for memes which are sticky is also a notable factor.

Comment by plex (ete) on mike_hawke's Shortform · 2021-01-18T00:17:32.068Z · LW · GW

Not exactly / only sci-fi, but Rational Reads is a good place to look if you liked HPMOR.

Comment by plex (ete) on What is going on in the world? · 2021-01-17T21:08:31.245Z · LW · GW

hm, that intuition seems plausible.

The other point that comes to mind is that if you have a classical simulation running on a quantum world, maybe that counts as branching for the purposes of where we expect to find ourselves? I'm still somewhat confused about whether exact duplicates 'count', but if they do then maybe the branching factor of the underlying reality carries over to sims running further down the stack?

Comment by plex (ete) on What is going on in the world? · 2021-01-17T18:01:13.648Z · LW · GW

As someone who mostly expects to be in a simulation, this is the clearest and most plausible anti-simulation-hypothesis argument I've seen, thanks.

How does it hold up against the point that the universe looks large enough to support a large number of even fully-quantum single-world simulations (with a low-resolution approximation of the rest of reality), even if it costs many orders of magnitude more resources to run them?

Perhaps would-be simulators would tend not to value the extra information from full-quantum simulations enough to build many or even any of them? My guess is that many purposes for simulations would want to explore a bunch of the possibility tree, but depending on how costly very large quantum computers are to mature civilizations maybe they'd just get by with a bunch of low-branching factor simulations instead?

Comment by plex (ete) on What is going on in the world? · 2021-01-17T14:07:40.283Z · LW · GW

Maybe something about the collapse of sensemaking and the ability of people to build a shared understanding of what's going on, partly due to rapid changes in communications technology transforming the memetic landscape?

Comment by plex (ete) on The True Face of the Enemy · 2021-01-12T20:39:43.470Z · LW · GW

We’ve duped EVERYONE.


At age 5 I am told that every single morning as we drove to school I said to my mother that it was a waste of time. Shockingly, she listened, and after a year of this she had found out about home education and made arrangements for me to be released.

I am beyond glad to have avoided most of formal education, despite having been put back into it twice during my teenage years for several years each time. The difference between my motivation to learn, social fulfilment, and general wellbeing was dramatic.

I am curious about what alternatives could be built with modern technology, and whether a message like this could spread enough to shift a notable fraction of children to freedom.

Comment by plex (ete) on Building up to an Internal Family Systems model · 2021-01-08T16:07:50.548Z · LW · GW

I've read a lot of books in the self-help/therapy/psychology cluster, but this is the first which gives a clear and plausible model of why the mental structure they're all working with (IFS exiles, EMDR unprocessed memories, trauma) has enough fitness-enhancing value to evolve despite the obvious costs.

Comment by plex (ete) on plex's Shortform · 2021-01-07T23:46:47.100Z · LW · GW

A couple of months ago I did some research into the impact of quantum computing on cryptocurrencies, seems maybe significant, and a decent number of LWers hold cryptocurrency. I'm not sure if this is the kind of content that's wanted, but I could write up a post on it.