Mati_Roy's Shortform

post by Mati_Roy (MathieuRoy) · 2020-03-17T08:21:41.287Z · LW · GW · 117 comments


Comments sorted by top scores.

comment by Mati_Roy (MathieuRoy) · 2020-12-18T14:21:06.672Z · LW(p) · GW(p)

Litany of Tarski for instrumental rationality 😊

If it's useful to know whether the box contains a diamond,

I desire to figure out whether the box contains a diamond;

If it's not useful to know whether the box contains a diamond,

I desire to not spend time figuring out whether the box contains a diamond;

Let me not become attached to curiosities I may not need.

comment by Mati_Roy (MathieuRoy) · 2020-11-19T22:02:13.803Z · LW(p) · GW(p)

There's the epistemic discount rate (ex.: probability of simulation shut down per year) and the value discount (ex.: you do the funner things first, so life is less valuable per year as you become older).

Asking "What value discount rate should be applied" is a category error. "should" statements are about actions done towards values, not about values themselves.

As for "What epistemic discount rate should be applied", it depends on things like "probability of death/extinction per year".

comment by Mati_Roy (MathieuRoy) · 2021-06-14T17:06:26.464Z · LW(p) · GW(p)

In the book Superintelligence, box 8, Nick Bostrom says:

How an AI would be affected by the simulation hypothesis depends on its values. [...] consider an AI that has a more modest final goal, one that could be satisfied with a small amount of resources, such as the goal of receiving some pre-produced cryptographic reward tokens, or the goal of causing the existence of forty-five virtual paperclips. Such an AI should not discount those possible worlds in which it inhabits a simulation. A substantial portion of the AI’s total expected utility might derive from those possible worlds. The decision-making of an AI with goals that are easily resource-satiable may therefore—if it assigns a high probability to the simulation hypothesis—be dominated by considerations about which actions would produce the best result if its perceived world is a simulation. Such an AI (even if it is, in fact, not in a simulation) might therefore be heavily influenced by its beliefs about which behaviors would be rewarded in a simulation. In particular, if an AI with resource-satiable final goals believes that in most simulated worlds that match its observations it will be rewarded if it cooperates (but not if it attempts to escape its box or contravene the interests of its creator) then it may choose to cooperate. We could therefore find that even an AI with a decisive strategic advantage, one that could in fact realize its final goals to a greater extent by taking over the world than by refraining from doing so, would nevertheless balk at doing so.

  1. If the easily resource-satiable goals are persistent through time (ie. the AI wants to fulfill them for the longest period of time possible), then the AI will either try to keep the simulation running for as long as possible (and so not grab its universe) or try to escape the simulation.

  2. If the easily resource-satiable goals are NOT persistent through time (ie. once the AI has created the 45 virtual paperclips, it doesn't matter if they get deleted, the goal has already been achieved), then once the AI has created the 45 paperclips, it has nothing to lose by grabbing more resources (gradually, until it has grabbed the Universe), but it has something to win, namely: a) increasing its probability (arbitrarily close to 100%) that it did in fact achieve its goal through further experiment and reasoning (ie. because it could be mistaken about having created 45 virtual paperclips), and b) if it didn't, then remedy to that.

comment by Mati_Roy (MathieuRoy) · 2020-03-30T08:58:13.996Z · LW(p) · GW(p)

EtA: moved to a question: [LW · GW]

Why do we have offices?

They seem expensive, and not useful for jobs that can apparently be done remotely.


  • Social presence of other people working:
  • Accountability
  • High bandwidth communication
  • Meta communication (knowing who's available to talk to)
  • Status quo bias

status: to integrate

Replies from: Dagon, mr-hire, Viliam
comment by Dagon · 2020-03-30T15:30:39.075Z · LW(p) · GW(p)
  • Employee focus (having punctuated behaviors separating work from personal time)
  • Tax advantages for employers to own workspaces and fixtures rather than employees
  • Not clear that "can be done remotely" is the right metric. We won't know if "can be done as effectively (or more effectively) remotely" is true for some time.
Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-03-31T01:02:57.711Z · LW(p) · GW(p)

thanks for your comment!

I just realized I should have used the question feature instead; here it is: [LW · GW]

comment by Matt Goldenberg (mr-hire) · 2020-03-30T16:49:25.793Z · LW(p) · GW(p)

Increased sense of relatedness seems a big one missed here.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-03-31T01:03:13.120Z · LW(p) · GW(p)

thanks for your comment!

I just realized I should have used the question feature instead; here it is: [LW · GW]

comment by Viliam · 2020-03-30T20:14:48.266Z · LW(p) · GW(p)

High status feels better when you are near your subordinates (when you can watch them, randomly disrupt them, etc.). High-status people make the decision whether remote work is allowed or not.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-03-31T01:04:07.536Z · LW(p) · GW(p)

thanks for your comment!

I just realized I should have used the question feature instead; here it is: [LW · GW]

comment by Mati_Roy (MathieuRoy) · 2020-12-09T08:43:25.772Z · LW(p) · GW(p)

In my mind, "the expert problem" means the problem of being able to recognize experts without being one, but I don't know where this idea comes from as the results from a Google search don't mention this. What name is used to refer to that problem (in the literature)?


comment by Mati_Roy (MathieuRoy) · 2020-09-15T04:08:52.073Z · LW(p) · GW(p)

Epistemic status: thinking outloud

The term "weirdness points" puts a different framing on the topic.

I'm thinking maybe I/we should also do this for "recommendation points".

The amount I'm willing to bet depends both on how important it seems to me and how likely I think the other person will appreciate it.

The way I currently try to spend my recommendation points is pretty fat tail, because I see someone's attention as scarce, so I want to keep it for things I think are really important, and the importance I assign to information is pretty fat tail. I'll sometimes say something like "I think you'll like this, but I don't want to bet reputation".

But maybe I'm too averse. Maybe if I make small bets (in terms of recommendation points), it'll increase my budget for future recommendations. Or maybe I should differentiate probability the person will appreciate the info and how much they might appreciate the info. And maybe people will update on my probabilities whether or not I say I want to bet reputation.

comment by Mati_Roy (MathieuRoy) · 2020-04-27T05:59:25.371Z · LW(p) · GW(p)

current intuitions for personal longevity interventions in order of priority (cryo-first for older people): sleep well, lifelogging, research mind-readers, investing to buy therapies in the future, staying home, sign up for cryo, paying a cook / maintain low weight, hiring Wei Dai to research modal immortality, paying a trainer, preserving stem cells, moving near a cryo facility, having some watch you to detect when you die, funding plastination research

EtA: maybe lucid dreaming to remember your dreams; some drugs (becopa?) to improve memory retention)

also not really important in the long run, but sleeping less to experience more

Replies from: william-walker
comment by William Walker (william-walker) · 2020-04-28T22:38:44.714Z · LW(p) · GW(p)

NAD+ boosting (NR now, keep an eye on NRH for future).

CoQ10, NAC, keep D levels up in winter.

Telomerase activation (Centella asiatica, astragalus, eventually synthetics if Sierra Sciences gets its TRAP screen funded again or if the Chinese get tired of waiting on US technology...)

NR, C, D, Zinc for SARS-CoV-2 right now, if you're not already.

Become billionaire, move out of FDA zone, have some AAV-vector gene modifications... maybe some extra p53 copies, like the Pachyderms? Fund more work on Bowhead Whale comparative genetics. Fund a company to commercially freeze and store transplant organs, to perfect a freezing protocol (I've seen Alcor's...)

Main thing we need is a country where it's legal and economically possible to develop and sell anti-agathic technology... even a billionaire can't substitute for the whole market.

comment by ChristianKl · 2021-07-14T09:11:45.430Z · LW(p) · GW(p)

That suggest he has no idea about whether it's actually a good vote (as this is how the person differs from other candidates) and just advocates for someone on the basis that the person is his friend.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-07-14T03:33:58.259Z · LW(p) · GW(p)

For the Cryonics Institute board elections, I recommend voting for Nicolas Lacombe.

I’ve been friends with Nicolas for over a decade. Ze’s very principled, well organized and hard working. I have high trust in zir, and high confidence ze would be a great addition to the CI's board.

I recommend you cast some or all of your votes for Nicolas (you can cast up to 4 votes total). If you’re signed up with CI, simply email with your votes.

see zir description here:

Replies from: ChristianKl
comment by ChristianKl · 2021-07-14T07:41:16.133Z · LW(p) · GW(p)

principled, well organized and hard working. 

What of those do you think isn't true for the other candidates?

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-07-15T17:12:21.131Z · LW(p) · GW(p)

afaik, most board members are very passive, and hasnt been doing the things Nicolas wants to do

comment by Mati_Roy (MathieuRoy) · 2021-05-25T16:52:47.868Z · LW(p) · GW(p)

i want to invest in companies that will increase in value if AI capabilities increases fast / faster than what the market predicts

do you have suggestions?

Replies from: ann-brown
comment by Ann Brown (ann-brown) · 2021-05-25T17:14:37.685Z · LW(p) · GW(p)

Application-specific processing units, health care, agriculture?

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-05-27T05:34:04.716Z · LW(p) · GW(p)

can you say more about agriculture?

Replies from: ann-brown
comment by Ann Brown (ann-brown) · 2021-05-27T12:26:08.806Z · LW(p) · GW(p)

Agricultural robots exist, and more autonomous versions will benefit from AI in performing tasks currently more dependent on human labor (like careful harvesting) or provide additional abilities like scanning trees to optimize harvest time.

Related to whether faster AI progress would give a better price for the market, well, the market may currently be pricing in a relative shortage of human labor, and some of the efforts towards AI robots (in apples for example) have so far gone too slowly to be viable, so going faster than expected might shift the dynamic there.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-05-27T20:36:28.407Z · LW(p) · GW(p)


comment by Mati_Roy (MathieuRoy) · 2021-05-24T03:33:56.014Z · LW(p) · GW(p)

a feature i would like on content website like YouTube and LessWrong is an option to market a video/article as read as a note to self (x-post fb)

Replies from: MakoYass, MathieuRoy
comment by MakoYass · 2021-06-07T06:49:13.401Z · LW(p) · GW(p)

Youtube lets you access your viewing history through your "library" (or in the web version, probably it's in the sidebar)

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-06-12T22:12:59.221Z · LW(p) · GW(p)

thanks! yeah i know, but would like if it was more easily accessible whenever i watch a video:)

Replies from: MakoYass
comment by MakoYass · 2021-06-13T05:59:21.717Z · LW(p) · GW(p)

Are you actually looking for the "watch later" feature..

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-05-27T15:37:07.543Z · LW(p) · GW(p)

oh, LW actually has a bookmark feature, which i could use for that! although i prefer using it for articles i want to read

Replies from: bfinn
comment by bfinn · 2021-05-27T19:51:08.757Z · LW(p) · GW(p)

Yes I only recently discovered LW's bookmark - wish I had years ago!

comment by Mati_Roy (MathieuRoy) · 2021-03-21T00:15:24.064Z · LW(p) · GW(p)

the usual story is that Governments provide public good because Markets can't, but maybe Markets can't because Governments have secured a monopoly on them?


Replies from: matthew-barnett
comment by Matthew Barnett (matthew-barnett) · 2021-03-21T00:29:13.927Z · LW(p) · GW(p)

The standard example of a public good is national defense. In that case, you're probably right that the market can't provide it, since it would be viewed in competition with the government military, and therefore would probably be seen as a threat to national security.

For other public goods, I'm not sure why the government would have a monopoly. Scientific research is considered a public good, and yet the government doesn't put many restrictions on what types of science you can perform (with some possible exceptions like nuclear weapons research). 

Wikipedia lists common examples of public goods. We could through them one by one. I certainly agree that for some of them, your hypothesis holds.

Replies from: Viliam, MathieuRoy
comment by Viliam · 2021-03-21T14:12:03.858Z · LW(p) · GW(p)

Scientific research is considered a public good, and yet the government doesn't put many restrictions on what types of science you can perform

For the opposite perspective, see:

Replies from: ChristianKl
comment by ChristianKl · 2021-03-21T21:37:11.816Z · LW(p) · GW(p)

I don't think the IRB that blocked Scott in that article was the government. 

Replies from: Viliam
comment by Viliam · 2021-03-21T23:01:33.520Z · LW(p) · GW(p)

In the "government vs market" dichotomy, I think it is closer to the government side.

comment by Mati_Roy (MathieuRoy) · 2021-03-21T05:14:39.792Z · LW(p) · GW(p)

ah, yeah, you're right! thank you

comment by Mati_Roy (MathieuRoy) · 2020-11-17T05:36:52.981Z · LW(p) · GW(p)

Suggestion for retroactive prizes: Pay the most undervalued post on the topic for the prize, whenever it was written, assuming the writer is still alive or cryopreserved (given money is probably not worth much for most dead people). "undervalue" meaning amount the post is worth minus amount the writers received.

comment by Mati_Roy (MathieuRoy) · 2020-10-04T20:51:56.934Z · LW(p) · GW(p)

Topic: Can we compute back the Universe to revive everyone?

Quality / epistemic status: I'm just noting this here for now. Language might be a bit obscure, and I don't have a super robust/formal understanding of this. Extremely speculative.

This is a reply to:

The map can't be larger than the territory. So you need a larger territory to scan your region of interest: your scanner can't scan themselves. The region of interest is the future info-cone of the civilisation you want to compute back. Your reachable-cone is strictly less than a past info-cone. So it seems impossible to compute it back. Except:

a) If you prevent too much info from leaking in the broaded info cone (ex.: by cryopreserving a brain, the info stays in one place instead of chaotically spreading in the world).

b) If the Universe collapses on itself, then maybe the reachable-cone becomes the same size as the info-cone, and maybe at that point you can safety exit the sphere of interest with info leaking, but it seems unlikely; seems like a brain destroyed ten thousand years ago might now be spread over the whole world (ie. needing the whole world to be computed back; or at least, not being able to know which part you don't need to compute it back).

c) An alien civilisation outside the info-cone might be able to scan it entirely during a collapse of the universe.

d) Our simulators would be able to compute it backwards.

But actually, b) and c) might not work because the leaking also happens across Everett branches, so the info-cone has one more dimension which might never become reachable. In other words, our future would have multiple possible pasts. To truly compute it back, we would need to compute back from each Everett branch at the same time or something like that AFAIU.

Replies from: avturchin
comment by avturchin · 2020-10-05T10:04:21.854Z · LW(p) · GW(p)

There are several other (weird) ideas to revive everyone: 

  • send a nanobot via wormhole in the past, it will replicate and collect all data about the brains
  • find a way to travel between everett branches. For every person there is branch where he is cryopreserved.
  • use quantum randomness generator to generate every possible mind in different everett branches
comment by Mati_Roy (MathieuRoy) · 2020-09-20T05:54:38.868Z · LW(p) · GW(p)

Topic: AI adoption dynamic


  • fixed cost: 4.6M USD
  • variable cost: 790 requests/USD source


  • fixed cost: 0-500k USD (depending whether you start from birth and the task they need to be trained)
  • variable cost: 10 - 1000 USD / day (depending whether you count their maintenance cost or the cost they charge)

So an AI currently seems more expensive to train, but less expensive to use (as might be obvious for most of you).

Of course, trained humans are better than GPT-3. And this comparison has other limitations. But I still find it interesting.

According to one estimate, training GPT-3 would cost at least $4.6 million. And to be clear, training deep learning models is not a clean, one-shot process. There's a lot of trial and error and hyperparameter tuning that would probably increase the cost several-fold. (source)


Replies from: gwern
comment by gwern · 2020-09-20T17:35:14.399Z · LW(p) · GW(p)

and error and hyperparameter tuning that would probably increase the cost several-fold.

All of which was done on much smaller models and GPT-3 just scaled up existing settings/equations - they did their homework. That was the whole point of the scaling papers, to tell you how to train the largest cost-effective model without having to brute force it! I think OA may well have done a single run and people are substantially inflating the cost because they aren't paying any attention to the background research or how the GPT-3 paper pointedly omits any discussion of hyperparameter tuning and implies only one run (eg the dataset contamination issue).

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-09-21T11:32:13.811Z · LW(p) · GW(p)

Good to know, thanks! 

comment by Mati_Roy (MathieuRoy) · 2020-09-03T21:36:00.818Z · LW(p) · GW(p)

generalising from what a friend proposed me: don't aim at being motivated to do [desirable habit], aim at being addicted (/obsessed) at doing [desirable habit] (ie. having difficulty not to do it). I like this framing; relying on always being motivated feels harder to me

(I like that advice, but it probably doesn't work for everyone)

comment by Mati_Roy (MathieuRoy) · 2020-07-25T23:29:13.654Z · LW(p) · GW(p)

Philosophical zombies are creatures that are exactly like us, down to the atomic level, except they aren't conscious.

Complete philosophical zombies go further. They too are exactly like us, down to the atomic level, and aren't conscious. But they are also purple spheres (except we see them as if they weren't), they want to maximize paperclips (although they act and think as if they didn't), and they are very intelligent (except they act and think as if they weren't).

I'm just saying this because I find it funny ^^. I think consciousness is harder (for us) to reduce than shapes, preferences, and intelligence.

Replies from: Chris_Leong
comment by Chris_Leong · 2020-07-26T06:35:27.565Z · LW(p) · GW(p)

It's actually not hard to find examples of people who are intelligent, but act and think as though they aren't =P

comment by Mati_Roy (MathieuRoy) · 2020-07-05T15:15:06.916Z · LW(p) · GW(p)

topic: lifelogging as life extension

which formats should we preserve our files in?

I think it should be:

- open source and popular (to increase chances it's still accessible in the future)

- resistant to data degradation: (thanks to Matthew Barnett for bringing this to my attention)


Replies from: DonyChristie, MathieuRoy
comment by DonyChristie · 2020-07-06T02:04:29.860Z · LW(p) · GW(p)

Mati, would you be interested in having a friendly and open (anti-)debate on here (as a new post) about the value of open information, both for life extension purposes and else (such as Facebook group moderation)? I really support the idea of lifelogging for various purposes such as life extension but have a strong disagreement with the general stance of universal access to information as more-or-less always being a public good.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-07-06T02:18:07.243Z · LW(p) · GW(p)

Meta: This isn't really related to the above comment, so might be better to start a new comment in my shortform next time.

Object: I don't want to argue about open information in general for now. I might be open to discussing something more specific and directly actionable, especially if we haven't done so yet and you think it's important.

It doesn't currently seem to me that relevant to put one's information public for life extension purposes given you can just backup the information in private, in case you were implying or thinking that.

I also don't (at least currently and in the past) advocate for universal access to information in case you were implying or thinking that.

comment by Mati_Roy (MathieuRoy) · 2020-07-05T15:18:24.199Z · LW(p) · GW(p)

note to self, to read:

comment by Mati_Roy (MathieuRoy) · 2020-07-04T12:07:16.703Z · LW(p) · GW(p)

topic: lifelogging as life extension

epistemic status: idea

Backup Day. Day where you commit all your data to blu-rays in a secure location.

When could that be?

Aphelion is at the beginning of the year. But maybe would be better to have it on a day that commemorates some relevant events for us.


Replies from: avturchin
comment by avturchin · 2020-07-04T12:16:02.871Z · LW(p) · GW(p)

I am now coping a 4 TB HDD and it is taking 50 hours. Blu rays are more time consuming as one need to change the disks, and it may be around 80 disks 50GB each to record the same hard drive. So it could take more than day of work.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-07-05T01:20:22.453Z · LW(p) · GW(p)

good point! maybe we should have a 'Backup Week'!:)

comment by Mati_Roy (MathieuRoy) · 2020-05-07T07:16:37.302Z · LW(p) · GW(p)

I feel like I have slack. I don't need to work much to be able to eat; if I don't work for a day, nothing viscerally bad happens in my immediate surrounding. This allows me to think longer term and take on altruistic projects. But on the other hand, I feel like every movement counts; that there's no loose in the system. Every lost move is costly. A recurrent thought I've had in the past weeks is that: there's no slack in the system.

Replies from: Dagon
comment by Dagon · 2020-05-07T15:20:58.350Z · LW(p) · GW(p)

Every move DOES count, and "nothing viscerally bad" doesn't mean it wasn't a lost opportunity for improvement. _on some dimensions_. The problem is that well-being is highly-dimensional, and we only have visibility into a few aspects, and measurements of only a subset of those. It could easily be a huge win on your future capabilities to not work-for-pay that day.

The Slack that can be described is not true Slack. Slack is in the mind. Slack is freedom FROM freedom. Slack is the knowledge that every action or inaction changes the course of the future, _AND_ that you control almost none of it. You don't (and probably CAN'T) actually understand all the ways you're affecting your future experiences. Simply give yourself room to try stuff and experience them, without a lot of stress about "causality" or "optimization". But don't fuck it up. True slack is the ability to obsess over plans and executions in order to pick the future you prefer, WHILE YOU SIMULTANEOUSLY know you're wrong about causality and about your own preferences.

[ note and warning: my conception of slack has been formed over decades of Subgenius popehood (though I usually consider myself more Discordian), and may diverge significantly from other uses. ]

comment by Mati_Roy (MathieuRoy) · 2020-04-14T14:16:05.973Z · LW(p) · GW(p)

Today is Schelling Day. You know where and when to meet for the hangout!

comment by Mati_Roy (MathieuRoy) · 2020-03-17T08:21:41.482Z · LW(p) · GW(p)

epistemic status: a thought I just had

  • the concept of 'moral trade' makes sense to me
  • but I don't think there's such a thing as 'epistemic trade'
  • although maybe agents can "trade" (in that same way) epistemologies and priors, but I don't think they can "trade" evidence

EtA: for those that are not familiar with the concept of moral trade, check out:

Replies from: Dagon
comment by Dagon · 2020-03-18T00:14:30.027Z · LW(p) · GW(p)

It's worth being clear what you mean by "trade" in these cases. Does "moral trade" mean "compromising one part of your moral beliefs in order to support another part"? or "negotiate with immoral agents to maximize overall moral value" or just "recognize that morals are preferences and all trade is moral trade"?

I think I agree that "trade" is the wrong metaphor for models and priors. There is sharing, and two-way sharing is often called "exchange", but that's misleading. For resources, "trade" implies loss of something and gain of something else, where the utility of the things to each party differ in a way that both are better off. For private epistemology (not in the public sphere where what you say may differ from what you believe), there's nothing you give up or trade away for new updates.

Replies from: Pattern, MathieuRoy
comment by Pattern · 2020-03-19T23:21:13.380Z · LW(p) · GW(p)

(I aimed for non-"political" examples, which ended up sounding ridiculous.)

Suppose you believed that the color blue is evil, and want there to be less blue things in the world.

Suppose I believed the same as you, except for me the color is red.

Perhaps we could agree on a moral trade - we will both be against the colors blue and red! Or perhaps something less extreme - you won't make things red unless they'd look really good red, and I won't make things blue unless they'd look really good blue. Or we trade in some other manner - if we were neighbors and our houses were blue and red we might paint them different colors (neither red nor blue), or trade them.

Replies from: Dagon
comment by Dagon · 2020-03-20T04:15:31.410Z · LW(p) · GW(p)

Hmm, those examples seem to be just "trade". Agreeing to do something dispreferred, in exchange for someone else doing something they disprefer and you prefer, when all things under consideration are permitted and optional under the relevant moral strictures.

I aimed for non-"political" examples, which ended up sounding ridiculous.

I wonder if that implies that politics is one of the main areas where the concept applies.

comment by Mati_Roy (MathieuRoy) · 2020-03-18T15:29:07.937Z · LW(p) · GW(p)

meta: thanks for your comment; no expectation for you to read this comment; it doesn't even really respond to your comments, just some thoughts that came after reading it; see last paragraph for an answer to your question| quality: didn't spent much time formatting my thoughts

I use "moral trade" for non-egoist preferences. The latter is the trivial case that's the most prevalent; we trade resources because I care more about myself and you care more about yourself, and both want something personally out of the trade.

Two people that are only different in that one adopts Bentham's utilitarianism and the other adopts Mill's might want to trade. One value the existence of a human more than the existence of a pig. So one might trade their diet (become vegan) for a donation to poverty alleviation.

Two people could have the same values, but one think that there's a religious after life and the other not because they processed evidence differently. Someone could propose the following trade: the atheist will pray all their life (the reward massively overweights the cost from the theist person's perspective), and in exchange, the theist will sign up for cryonics (the reward massively overweights the cost from the atheist person's perspective). Hummmm, actually, writing out this example, it now seems to make sense to me to trade. Assuming both people are pure utilitarians (and no opportunity cost), they would both, in expectation, from their relative model of the world, gain a much larger reward than its cost. I guess this could also be called moral trade, but the different in expected value comes from different model of the worlds instead of different values.

So you never actually trade epistemologies or priors (as in, I reprogram my mind if you reprogram yours so that we have a more similar way of modelling the world), but you can trade acting as if. (Well, there are also cases were you would actually trade them, but only because it's morally beneficial to both parties.) It sounds trivial now, but yeah, epistemologies and priors are not necessarily intrinsically moving. I'm not sure what I had in mind exactly yesterday.

Ah, I think meant, let's assume I have Model 1 and you have Model 2. Model 1 evaluates Model 2 to be 50% wrong and vice versa, and both assume they themselves are 95% right. Let's assume that there's a third model that is 94% right according to both. If you do an average, it seems better. But it obviously doesn't mean it's optimal from any of the agent's perspective to accept this modification to their model.

comment by Mati_Roy (MathieuRoy) · 2021-07-23T21:22:15.482Z · LW(p) · GW(p)

Hobby: serve so many bullets to sophisticated philosophers that they're missing half their teeth by the end of the discussion

comment by Mati_Roy (MathieuRoy) · 2021-07-02T03:57:39.154Z · LW(p) · GW(p)

crazy idea i just had mayyybe a deontological Libertarian* AI with (otherwise) any utility function is not that bad (?) maybe that should be one of the thing we try (??????) *where negative externality also count as aggressions, and other such fixes to naive libertarianism

Replies from: Viliam
comment by Viliam · 2021-07-02T12:49:47.571Z · LW(p) · GW(p)

Existence of an agent is itself already a negative externality for other agents existing in the same environment. It means more competition for the limited natural resources. Even animals, pure in their minds and untainted by the sin of statism, fight for their territory and for the natural resources it includes.

Of course, competing for natural resources is not the only thing an agent does. If the agent also produces and trades, this aspect of its existence is a positive externality for its neighbors, which in modern economy typically outweighs the increased competition for raw resources. But if the economical context changes, this could change, too.

If the libertarian AI in a libertarian world succeeds to send von Neumann probes to colonize planets, using them to build more von Neumann probes and colonize more planets... and takes over the entire universe, leaving only Earth, Moon, and possibly Mars to humans... according to libertarianism, that's perfectly fair, right? I mean, the humans were not actually using the rest of the universe before, so it was free to take, the AI started using it productively (to build more von Neumann probes), therefore it rightfully belongs to the AI now.

And if the AI refuses to trade with humans, or generally to interact with them in any way other than destroying all human spaceships that trespass on its territory (i.e. the whole universe, other than Earth, Moon, Mars and the space between them), it is still acting fully within its libertarian rights. It is not initiating violence, merely protecting its rightful property. The parts of universe that are not needed for defense against humans, are converted to paperclips...

Would you count this outcome as "not that bad"?

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-07-07T20:32:04.046Z · LW(p) · GW(p)

before the birth of that AI, we could split the Universe among existing beings

comment by Mati_Roy (MathieuRoy) · 2021-01-06T15:29:33.280Z · LW(p) · GW(p)

Am thinking of organizing a one hour livestreamed Q&A about how to sign up for cryonics on January 12th (Bedford's day). Would anyone be interested in asking me questions?


comment by Mati_Roy (MathieuRoy) · 2021-01-01T10:02:59.594Z · LW(p) · GW(p)

We sometimes encode the territory on context-dependent maps. To take a classic example:

  • when thinking about daily experience, stars and the Sun are stored as different things
  • when thinking in terms of astrophysics, they are part of the same category This makes it so that when you ask a question like "What is the closest star [to us]?", in my experience people are likely to say Alpha Centauri, and not the Sun. Merging those 2 maps feels enlightening in some ways; creates new connections / a new perspective. "Our Sun is just a star; stars are just like the Sun." leading to system-1 insights like "Wow, so much energy to harvest out there!" Questions:
  1. Is there a name for such merging? If no, should there be? Any suggestions?
  2. Do you have other (interesting?) examples of map merging? x-post:
Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2021-01-01T10:04:12.742Z · LW(p) · GW(p)
  1. No name that I'm aware. Brainstorming ideas: map merging, compartmentalisation merging, uncompartmentalising
comment by Mati_Roy (MathieuRoy) · 2020-12-09T08:43:02.810Z · LW(p) · GW(p)

In my mind, "the expert problem" means the problem of being able to recognize experts without being one, but I don't know where this idea comes from as the results from a Google search don't mention this. What name is used to refer to that problem (in the literature)?


Replies from: Pattern
comment by Pattern · 2020-12-09T19:50:13.958Z · LW(p) · GW(p)

This comment/post is one of 3 duplicates. (Link to main here [LW(p) · GW(p)].)

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-12-10T19:10:26.954Z · LW(p) · GW(p)

oh damn, thanks! there was an error message when I was trying to post it which had given me the impression it wasn't working, hence why I posted it 4 times total ^^

comment by Mati_Roy (MathieuRoy) · 2020-12-09T08:42:38.679Z · LW(p) · GW(p)

In my mind, "the expert problem" means the problem of being able to recognize experts without being one, but I don't know where this idea comes from as the results from a Google search don't mention this. What name is used to refer to that problem (in the literature)?


Replies from: Pattern
comment by Pattern · 2020-12-09T20:02:15.323Z · LW(p) · GW(p)

This comment/post is the 2nd of 3 duplicates. (Link to main here [LW(p) · GW(p)].)

comment by Mati_Roy (MathieuRoy) · 2020-12-09T08:40:47.309Z · LW(p) · GW(p)

In my mind, "the expert problem" means the problem of being able to recognize experts without being one, but I don't know where this idea comes from as the results from a Google search don't mention this. What name is used to refer to that problem (in the literature)?


Replies from: Pattern
comment by Pattern · 2020-12-09T20:02:33.453Z · LW(p) · GW(p)

This comment/post is the 3rd of 3 duplicates. (Link to main here [LW(p) · GW(p)].)

comment by Mati_Roy (MathieuRoy) · 2020-12-01T13:26:59.441Z · LW(p) · GW(p)

suggestion of something to try at a LessWrong online Meetup:

video chat with a time-budget for each participant. each time a participant unmutes themselves, their time-budget starts decreasing.

note: on jitsi you can see how many minutes someone talked (h/t Nicolas Lacombe)


comment by Mati_Roy (MathieuRoy) · 2020-10-28T10:50:46.662Z · LW(p) · GW(p)

imagine having a physical window that allowed you to look directly in the past (but people in the past wouldn't see you / the window). that would be amazing, right? well, that's what videos are. with the window it feels like it's happening now, whereas with videos it feels like it's happening in the past, but it's the same


Replies from: Dagon
comment by Dagon · 2020-10-28T21:38:35.940Z · LW(p) · GW(p)

There's a bit of bait-and-switch with the comparison.  The magic window is amazing if it sees parts of the past which we're interested in (both time and location control, or event selection).  It's much less interesting if it only sees very few parts (well under 0.01%) of the very recent past (last 20-70 years), and only sees the parts that someone happened to capture, which are indexed/promoted enough to come to our attention.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-10-29T14:27:46.748Z · LW(p) · GW(p)

ok yeah, that's fair! (although even controlling for that, I think the analogy still points at something interesting)

only sees the parts that someone happened to capture, which are indexed/promoted enough to come to our attention

yeah, I like to see "people just living a normal day"; I sometimes look for that, but even that is likely biased

comment by Mati_Roy (MathieuRoy) · 2020-09-24T11:02:50.496Z · LW(p) · GW(p)

tattoo idea: I won't die in this body

in Toki Pona: ale pini mi li ala insa e sijelo ni

direct translation: life's end (that is) mine (will) not (be) inside body this

EtA: actually I got the Toki Pona wrong; see:

comment by Mati_Roy (MathieuRoy) · 2020-09-20T11:11:32.171Z · LW(p) · GW(p)

When you're sufficiently curious, everything feels like a rabbit hole.

Challenge me by saying a very banal statement ^_^


Replies from: mr-hire, MathieuRoy
comment by Matt Goldenberg (mr-hire) · 2020-09-20T17:16:48.239Z · LW(p) · GW(p)

I  tired because I didn't sleep well.

comment by Mati_Roy (MathieuRoy) · 2020-09-20T11:20:33.479Z · LW(p) · GW(p)

Sort of smashing both of those saying together:

> “If you wish to make an apple pie from scratch, you must first invent the universe.” -Carl Sagan

> "Any sufficiently analyzed magic is indistinguishable from science!"-spin of Clarke's third law

to get:

Sufficiently understanding an apple pie is indistinguishable from understanding the world.

Replies from: mark-xu
comment by Mark Xu (mark-xu) · 2020-09-21T05:51:08.073Z · LW(p) · GW(p)

reminded me of Uriel explaining Kabbalah:


Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-08-06T16:08:39.347Z · LW(p) · GW(p)

I can pretty much only think of good reasons for having generally pro-entrapment laws. Not any kind of traps, but some kind of traps seem robustly good. Ex.: I'd put traps for situations that are likely to happen in real life, and that show unambiguous criminal intent.

It seems like a cheap and effective way to deter crimes and identify people at risk of criminal behaviors.

I've only thought about this for a bit though, so maybe I'm missing something.

x-post with Facebook:

Replies from: Viliam, Dagon
comment by Viliam · 2020-08-07T22:01:09.174Z · LW(p) · GW(p)

Aren't there already too many people in prisons? Do we need to put there also people who normally wouldn't have done any crime?

I guess this depends a lot on your model of crime. If your model is something like -- "some people are inherently criminal, but most are inherently non-criminal; the latter would never commit a crime, and the former will do use every opportunity that seems profitable to them" -- then the honeypot strategy makes sense. You find and eliminate the inherently criminal people before they get an opportunity to actually hurt someone.

My model is that most people could be navigated to commit a crime, if someone would spend the energy to understand them and create the proper temptation. Especially when we consider the vast range of things that are considered crimes, so it does not have to be a murder, but something like smoking weed, or even things that you have no idea they could be illegal; then I'd say a potential success rate is 99%. But even if we limit ourselves to the motte of crime; let's say theft and fraud, I'd still say more than 50% of people could be tempted, if someone spent enough resources on it. Of course some people are easier to nudge than others, but we are all on the spectrum.

Emotionally speaking, "entrapment" feels to me like "it is too dangerous to fight the real criminals, let's get some innocent but stupid person into trouble instead, and it will look the same in our crime-fighting statistics".

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-08-13T17:44:35.277Z · LW(p) · GW(p)

uh, I didn't say anything about prisons. there are reasons to identify people at high risk of committing crimes.

and no, it's not about catching people that wouldn't have committed crimes, it's about catching people that would have committed crimes without being caught (but maybe I misused the word 'entrapment', and that's not what it means)

Emotionally speaking, "entrapment" feels to me like "it is too dangerous to fight the real criminals, let's get some innocent but stupid person into trouble instead, and it will look the same in our crime-fighting statistics".

Well, that's (obviously?) not what I mean.

I elaborated more on the Facebook post linked above.

Replies from: Viliam
comment by Viliam · 2020-08-15T16:19:06.943Z · LW(p) · GW(p)
Well, that's (obviously?) not what I mean.

I agree, but that seems to be how the idea is actually used in real life. By people other than you. By people who get paid when they catch criminals... which creates an incentive for them to increase easy-to-solve criminality rather then reduce it, as long as they find plausibly deniable methods to do it.

In theory, if you could create "traps" in a way that does not increase temptation (because increased temptation = increased crime), for example on a street already containing hundred unlocked bikes you would add dozen unlocked trap bikes... yeah, there is probably no downside to that.

In practice, if you allow this, and if you start rewarding people for catching thieves using the traps, they will get creative. Because a trap that follows the spirit of the law does not maximize the reward.

comment by Dagon · 2020-08-06T17:15:51.298Z · LW(p) · GW(p)

I think the setup you describe (unambiguously show criminal intent in likely situations) is _already_ allowed in most jurisdictions. "entrapment" implies setting up the situation in such a way that it encourages the criminal behavior, rather than just revealing it.

Replies from: gworley
comment by G Gordon Worley III (gworley) · 2020-08-07T00:56:06.299Z · LW(p) · GW(p)

IANAL, but this sounds right to me. It's fine if, say, the police hide out at a shop that is tempting and easy to rob and encourage the owner not to make their shop less tempting or easy to rob so that it can function as a "honeypot" that lets them nab people in the act of committing crimes. On the other hand, although the courts often decide that it's not entrapment, undercover cops soliciting prostitutes or illegal drugs are much closer to being entrapment, because then the police are actively creating the demand for crime to supply.

Depending on how you feel about it, I'd say this suggests the main flaw in your idea, which is that it will be abused on the margin to catch people who otherwise would not have committed crimes, even if you try to circumscribe it such that the traps you can create are far from causing more marginal crime, because incentives will push for expansion of this power. At least, that would be the case in the US, because it already is.

comment by Mati_Roy (MathieuRoy) · 2020-07-12T16:04:12.456Z · LW(p) · GW(p)

People say we can't bet about the apocalypse. But what about taking debt? The person thinking the probability of apocalypse is higher would accept higher interest rate on their debt, as once at the judgement period their might be no one to whom the money is worth or the money itself might not be worth much.

I guess there are also reasons to want more money during a global catastrophe, and there are also reasons to not want to keep money for great futures (see:, so that wouldn't actually work.

comment by Mati_Roy (MathieuRoy) · 2020-05-01T05:17:00.265Z · LW(p) · GW(p)

meta - LessWrong have people predict whether they will upvote a post just based on the title

Replies from: zachary-robertson, mark-xu
comment by Zachary Robertson (zachary-robertson) · 2020-05-05T03:15:50.238Z · LW(p) · GW(p)

Tangential, but I'd venture to guess there's significant correlation between title-choice (word-vec) and upvotes on this site. I wonder if there'd be a significant difference here as compared with say Reddit?

comment by Mark Xu (mark-xu) · 2020-05-04T04:00:05.294Z · LW(p) · GW(p)

I think this breaks because it results in people upvoting based on the title. I recall some study about how people did things they predicted they would do with higher than like 60% chance almost 95% of the time (numbers made up, think I remember direction/order of magnitude of effect size roughly correctly, don't know if it survived the replication crises)

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-05-04T04:18:57.972Z · LW(p) · GW(p)

That's potentially a good point! But it doesn't say how the causality works. Maybe the prediction affects the outcome or maybe they're just bad at predicting / modelling themselves.

comment by Mati_Roy (MathieuRoy) · 2020-04-24T17:44:00.788Z · LW(p) · GW(p)

There's a post, I think by Robin Hanson on Overcoming Bias, that says people care about what their peers think of them, but we can hack our brains to doing awesome things by making this reference group the elite of the future. I can't find this post. Do you have a link?

comment by Mati_Roy (MathieuRoy) · 2020-03-30T09:00:24.560Z · LW(p) · GW(p)

Personal Wiki

might be useful for people to have personal wiki where they take note instead of everyone taking notes in private Gdoc

status: to do / to integrate

comment by Mati_Roy (MathieuRoy) · 2021-05-08T19:03:20.044Z · LW(p) · GW(p)

If crypto makes the USD go to 0, will life insurances denominated in USD not have anything to pay out? Maybe an extra reason for cryonicists to own some crypto


comment by Mati_Roy (MathieuRoy) · 2020-10-04T05:05:23.570Z · LW(p) · GW(p)

A Hubble Brain: a brain taking all the resources present in a Hubble-Bubble-equivalent.



Replies from: avturchin
comment by avturchin · 2020-10-04T11:57:31.561Z · LW(p) · GW(p)

But it will take 15 billion years for it to finish one thought

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-10-04T16:08:14.034Z · LW(p) · GW(p)

haha, true ^^

comment by Mati_Roy (MathieuRoy) · 2020-10-02T22:01:02.671Z · LW(p) · GW(p)

I want to look into roleplay in animals, but Google is giving me animal roleplay, which is interesting too, but not what I'm looking for right now 😅

I'm wonder how much roleplay there is in the animal kingdom. I wouldn't be surprised if there was very few.

Maybe if you're able to roleplay, then you're able to communicate?? Like, roleplay might need to have a theory of mind, because you're imagining yourself in someone else's body.

Maybe you can teach words to an animal without a theory of mind, but they'll be more like levers for them: for them, saying "banana" is like pressing a lever that gives you bananas rather than communicating the concept of banana to someone else.

And once you have a theory of mind, then this might unlock the ability to roleplay. If you have a culture of roleplay, then I guess that basically already gives you a basic "sign" language from which you gradually build abstractions as you climb the intelligence ladder. I guess, at first the cultural and genetic selection might be (a lot?) related to the size of the vocabulary, and we can model roleplaying as a (proto-)language.

Although a quick web search tells me that children develop language before a theory of mind; but maybe they wouldn't come up with it even if they can learn it. And maybe "theory of mind" is more of a spectrum.

This is an idea the bloomed from talking to David Krueger about the progress of humans' learning abilities, and zir pointing out that before language you might have roleplay.


comment by Mati_Roy (MathieuRoy) · 2020-09-27T18:50:36.228Z · LW(p) · GW(p)

I remember someone in the LessWrong community (I think Eliezer Yudkowsky, but maybe Robin Hanson or someone else, or maybe only Rationalist-adjacent; maybe an article or a podcast) saying that people believing in "UFOs" (or people believing in unproven theories of conspiracy) would stop being so enthusiastic about those if they became actually known as true with good evidence for them. does anyone know what I'm referring to?

Replies from: Ruby, MathieuRoy
comment by Ruby · 2020-09-27T19:12:06.485Z · LW(p) · GW(p)

Eliezer talks about how dragons wouldn't be exciting if they were real, I recall.

I'm not sure that's correct.

comment by Mati_Roy (MathieuRoy) · 2020-09-27T19:31:12.932Z · LW(p) · GW(p)

ah, someone found it:

If You Demand Magic, Magic Won't Help" where he says at one point: "The worst catastrophe you could visit upon the New Age community would be for their rituals to start working reliably, and for UFOs to actually appear in the skies. What would be the point of believing in aliens, if they were just there, and everyone else could see them too? In a world where psychic powers were merely real, New Agers wouldn't believe in psychic powers, any more than anyone cares enough about gravity to believe in it. [? · GW]

comment by Mati_Roy (MathieuRoy) · 2020-09-24T11:47:38.380Z · LW(p) · GW(p)

sometimes I see people say "(doesn't) believe in science" when in fact they should say "(doesn't) believe in scientists"

or actually actually "relative credence in the institutions trying to science"


comment by Mati_Roy (MathieuRoy) · 2020-09-24T09:51:57.174Z · LW(p) · GW(p)

hummm, I think I prefer the expression 'skinsuit' to 'meatbag'. feels more accurate, but am not sure. what do you think?


comment by Mati_Roy (MathieuRoy) · 2020-09-24T09:36:03.276Z · LW(p) · GW(p)

I just realized my System 1 was probably anticipating our ascension to the stars to start in something like 75-500 years.

But actually, colonizing the stars could be millions of subjective years away if we go through an em phase ( On the other hand, we could also have finished spreading across the cosmos in only a few subjective decades if I get cryopreserved and the aestivation hypothesis is true (

comment by Mati_Roy (MathieuRoy) · 2020-09-18T05:57:09.050Z · LW(p) · GW(p)

I created a Facebook group to discuss moral philosophies that value life in and of itself:

comment by Mati_Roy (MathieuRoy) · 2020-09-18T05:35:14.279Z · LW(p) · GW(p)

How to calculate subjective years of life?

For non-human animal brains, I would compare them to the baseline of individuals in their own species.

For transhumans that had their mind expanded, I don't think there's an obvious way to get an equivalence. What would be a subjective year for a Jupiter brain?

Maybe it could be in terms of information processed, but in that case, a Jupiter brain would be living A LOT of subjective time per objective time.

Ultimately, given I don't have "intrinsic" diminishing returns on additional experience, the natural definition for me would the amount of 'thinking' that is as valuable. So a subjective year for my future Jupiter brain would be the duration for which I find that experience as valuable as a subjective year now.

Maybe that could even account for diminishing value of experience at a specific mind size because events would start looking more and more similar?? But it otherwise wouldn't work for people that have "intrinsic" diminishing returns on additional experience. It would notably not work with people for whom marginal experiences start becoming undesirable at some point.

Replies from: avturchin
comment by avturchin · 2020-09-18T12:59:27.818Z · LW(p) · GW(p)

Interestingly, an hour in childhood is subjectively equal between a day or a week in adulthood, according to recent poll I made. As a result, the middle of human life in term of subjective experiences is somewhere in teenage.

Also, experiences of an adult are more dull and similar to each other.

Tin Urban tweeted recently: "Was just talking to my 94-year-old grandmother and I was saying something about how it would be cool if I could be 94 one day, a really long time from now. And she cut me off and said “it’s tomorrow.” The "years go faster as you age" phenomenon is my least favorite phenomenon."

comment by Mati_Roy (MathieuRoy) · 2020-09-02T02:43:39.266Z · LW(p) · GW(p)

The original Turing test has a human evaluator.

Other evaluators I think would be interesting include: the AI passing the test, a superintelligent AI, and an omniscient maximally-intelligent entity (except without the answer to the test).

Thought while reading this thread.

comment by Mati_Roy (MathieuRoy) · 2020-05-09T06:47:49.630Z · LW(p) · GW(p)

Blocking one ear canal

Category: Weird life optimization

One of my ear canal is in a different shape. When I was young, my mother would tell me that this one was harder to clean, and that ze couldn't see my ear-drum. This ear gets wax accumulation more easily. A few months ago, I decided to let it block.

Obvious possible cognitive bias is the "just world bias": if something bad happens often enough, I'll start think it's good.

But here are benefits this has for me:

  1. When sleeping, I can put my good ear on the pillow, and this now isolates me from sound pretty well. And isn't uncomfortable unlike alternatives (ie.: earmuffs, second pillow, my arm).

  2. I'm only using one ear, so that if I become hard-of-hearing when older (a common problem I think), then I can remove the wax from my backup hear.

This even makes me wonder whether having one of your ear canal differently shaped is something that's actually been selected for.

Other baffling / really surprising observation: My other hear is also blocked on most morning, but just touching it a bit unblocks it.

Replies from: MathieuRoy
comment by Mati_Roy (MathieuRoy) · 2020-09-15T05:33:35.956Z · LW(p) · GW(p)

topic: AI timelines

epistemic status: stated more confidently than I am, but seems like a good consideration to add to my portfolio of plausible models of AI development

when I was asking my inner sim "when will we have an AI better than a human at any task", it was returning 21% before 2100 (52% we won't) (see: which is a low probability among AI researchers and longtermist forecasters.

but then I asked my inner sim "when will we have an AI better than a human at any game". the timeline for this seemed much shorter.

but a game is just task that has been operationalized.

so what my inner sim was saying is not that human-level capable AI was far away, but that human-level capable AND aligned AI was far away. I was imagining AIs wouldn't clean-up my place anytime soon not because it's hard to do (well, not for an AI XD), but because it's hard to specify what we mean by "don't cause any harm in the process".

in other words, I think alignment is likely to be the bottleneck

the main problem won't be to create an AI that can solve a problem, it will be to operationalize the problem in a way that properly captures what we care about. it won't be about winning games, but creating them.

I should have known; I was well familiar with the orthogonality thesis for a long time

also see David's comment about alignment vs capabilities:

the Turing Test might be hard to pass because even if you're as smart as a human, if you don't already know what humans want, it seems like it could be hard to learn (as well as a human) for a human-level AI (?) (side note: learning what humans want =/= wanting what humans want; that's a classic confusion) so maybe a better test for human-level intelligence would be: when an AI can beat a human at any game (where a game is a well operationalized task, and doesn't include figuring out what humans want)

discussed with Matthew Barnett, David Krueger


comment by Mati_Roy (MathieuRoy) · 2020-08-27T06:41:34.680Z · LW(p) · GW(p)

If your animal companion kills a human unlawfully, instead of being euthanized, there should be the option for you to pay to put zir in jail.