Posts

Formalising decision theory is hard 2019-08-23T03:27:24.757Z · score: 18 (19 votes)
Quantifying anthropic effects on the Fermi paradox 2019-02-15T10:51:04.298Z · score: 26 (12 votes)

Comments

Comment by lanrian on Implications of the Doomsday Argument for x-risk reduction · 2020-04-03T08:59:48.831Z · score: 4 (3 votes) · LW · GW

If ancestor simulations are one of the main uses of cosmic resources, we probably will go extinct soon (somewhat depending on how you define extinction), because we're probably in an ancestor simulation that will be turned off. If the simulators were to keep us alive for billions of years, it would be pretty unlikely that we didn't find ourselves living in those billions of years, by the same logic as the doomsday argument.

Comment by lanrian on April Coronavirus Open Thread · 2020-03-31T22:17:58.053Z · score: 3 (2 votes) · LW · GW

Link to paper, the relevant figure is on page 12.

Comment by lanrian on March Coronavirus Open Thread · 2020-03-26T17:14:51.785Z · score: 1 (1 votes) · LW · GW

That article is based on a twitter thread that is based on this article that is based on the parliamentary hearing that Wei Dai linked. The twitter thread distorted the article a lot, and seems to be mostly speculation.

Comment by lanrian on Authorities and Amateurs · 2020-03-25T20:23:21.921Z · score: 3 (2 votes) · LW · GW

The gaussian assumption makes no difference.

As I said, I do agree that the piece's qualitative conclusion was correct. However, the gaussian assumption does make a large quantitative difference. Comparing it to the extreme: If we always have the maximum number of people in ICUs, continuously, the time until herd-immunity would be 4.9 years, which is a factor 3 less than what the normal assumption gives you. Although it is still clearly too much, it's only one or two additional factors of 3 away from being reasonable. This extreme isn't even that unrealistic; something like it could plausibly be achieved if the government continuously alternated between more or less lock-down, keeping the average R close to 1.

To be clear, I think that it's good that the post was written, but I think it would have been substantially better if it had used a constant number of infected people. If you're going to use unrealistic models (which, in many cases, you should!) it's good practice to use the most conservative model possible, to ensure that your conclusion holds no matter what, and to let your reader see that it holds no matter what. In addition, it would have been simpler (you just have to multiply/divide 3 numbers), and it would have looked less like realistic-and-authoritative-math to the average reader, which would have better communicated its real epistemic status.

Comment by lanrian on Authorities and Amateurs · 2020-03-25T11:38:52.045Z · score: 6 (4 votes) · LW · GW

Notalgebraist posted a reasonable critique of Flattening The Curve Is a Deadly Delusion, explaining how it's incorrect to assume that flattenings won't reduce the total number of cases, and that it doesn't make sense to assume a normal distribution. I think that the the piece's main point was correct, though: that the curve would need to get crazy flat for hospitals to not be overloaded.

3 days later, the Imperial College study made the same point in a much better and more rigorous way (among lots of other good points), but it was less widely shared on social media.

I'm not sure what the conclusion here is. Non-experts will sometimes make false assumptions and get the details wrong, but they're still capable of making good points that only require you to multiply numbers together, and will do so a few days faster and in a way that's more memetically fit than papers from experts?

Comment by lanrian on Against Dog Ownership · 2020-03-23T14:19:54.850Z · score: 4 (3 votes) · LW · GW

This made me curious about where most people get their dogs from. Apparently, something like 34% are purchased from breeders and 23% are obtained from shelters, according to https://www.aspca.org/animal-homelessness/shelter-intake-and-surrender/pet-statistics (though their numbers don't add up to 100%, and I'm not sure why). Getting them from friends/relatives is also pretty common, at 20%.

Comment by lanrian on Are veterans more self-disciplined than non-veterans? · 2020-03-23T10:48:44.626Z · score: 2 (2 votes) · LW · GW

Is lack of discipline a big problem at an average workplace? I would expect most offices to provide sufficiently good social incentives that most people spend most of their time working, in which case any productivity-boon from increased discipline would be swamped by other (selection) effects from military training.

Increased discipline could translate to notably higher productivity for undergrads, PhD students, or people who work from home, though. My impression is that such people struggle much more with procrastination.

Relatedly, China seems to be doing their best to teach discipline in schools, so you could look at whether that seems to be working. This is obviously mixed in with lots of ongoing social incentives, though. Relevant ssc: https://slatestarcodex.com/2020/01/22/book-review-review-little-soldiers/

Comment by lanrian on What are good ways of convincing someone to rethink an impossible dream? · 2020-03-19T09:49:06.259Z · score: 1 (1 votes) · LW · GW

Why would you need to? Couldn't you just convince them that other people won't like their idea? In the second example above, there seems to be ample evidence that other people aren't interested.

Comment by lanrian on March Coronavirus Open Thread · 2020-03-16T21:12:42.331Z · score: 3 (3 votes) · LW · GW

Isn't this exactly what "flatten the curve" is about? Because a lot of people are talking about that as a solution, including some governments.

The main problem is that the curve needs to get really flat for hospitals to have time with everyone. Depending on how overwhelmed you want your hospitals to be, you could be in lock-down for several years. Some calculations in this article.

Comment by lanrian on A Significant Portion of COVID-19 Transmission Is Presymptomatic · 2020-03-14T10:38:17.124Z · score: 2 (2 votes) · LW · GW

Insofar as the virus mostly spreads through presymptomatic transmission in some countries, that's almost certainly because the people with symptoms are all isolated. Symptomatic people definitely spread the disease.

It'd be interesting if the R of these populations were <1 at the time the studies were done, though. If so, presymptomatic transmission might be insufficient to sustain exponential growth, as long as all symptomatic transmission is prevented.

Comment by lanrian on March Coronavirus Open Thread · 2020-03-11T21:09:36.931Z · score: 5 (3 votes) · LW · GW

Angela Merkel says that 60-70% of Germany is likely to be infected. That's useful if people believe that it won't infect that many. Example source, though you can google for others.

If they're willing to believe a redditor's summary, this one says that WHO says that 20% of infected people needed hospital treatment for weeks. (If they want primary sources, maybe you could find those claims in the links / somewhere else.)

Putting together 1 and 2 (and generalising from Germany to whatever country they're in), they ought to be convinced that it's pretty severe.

Comment by lanrian on Credibility of the CDC on SARS-CoV-2 · 2020-03-07T10:01:27.499Z · score: 10 (7 votes) · LW · GW

Our best guess is that the CDC is trying to conserve masks for health care professionals and others with the highest need, in the face of a looming mask shortage. That could easily be the optimum mask allocation. I can’t prove the lie wasn’t justified for the greater good. But it is another example of the CDC placing “getting the outcome it wants” over “telling people the literal truth.”

As far as I can tell, the CDC hasn't uttered a literal lie about this. In the link, they only say "CDC does not recommend that people who are well wear a facemask to protect themselves from respiratory diseases, including COVID-19", which is a recommendation, rather than a statement of efficacy. It could be motivated by a desire to stop mask-hoarding, as you say, or by the belief that typical usage of masks (including reuse, frequently readjusting the mask and thereby touching your face, etc) actually harms people more than it helps them.

(It's interesting that the link also says "The use of facemasks is also crucial for health workers and people who are taking care of someone in close settings (at home or in a health care facility)." This is (i) an admission that masks can protect you when you're close to someone sick, and (ii) does provide an incentive for people to hoard masks, if they think they're going to be taking care of someone.)

Regardless, it's fair to say that they're placing "getting the outcome it wants" at least over "telling people the full truth", and that this is a strike against the CDC's trustworthiness.

Edit: This SSC says that the CDC has been advising the public against using masks for a long time, so whatever they're saying, they're probably saying it for different reasons than to stop hoarding.

Comment by lanrian on What does Solomonoff induction say about brain duplication/consciousness? · 2020-03-06T17:12:55.013Z · score: 2 (2 votes) · LW · GW

This is highly related to UDASSA. In the linked post, especially Problem #2 (about splitting conscious computers) and bits of Problem #3 (e.g. "What happens if we apply UDASSA to a quantum universe? For one, the existence of an observer within the universe doesn't say anything about conscious experience. We need to specify an algorithm for extracting a description of that observer from a description of the universe"...)

Comment by lanrian on Is there a better way to define groups for COVID-19 impact? · 2020-03-06T11:32:06.216Z · score: 3 (2 votes) · LW · GW

Some data on infants: https://jamanetwork.com/journals/jama/fullarticle/2761659

Comment by lanrian on What will be the big-picture implications of the coronavirus, assuming it eventually infects >10% of the world? · 2020-03-04T23:30:35.381Z · score: 1 (1 votes) · LW · GW

My impression is that the WHO has been dividing up confirmed cases into mild/moderate (≈80%) and severe/critical (20%). The guesstimate model assumes that there are 80% "mild" cases, and 20% "confirmed" cases, which is inconsistent with WHO's terminology. If you got the 80%-number from WHO or some other source using similar terminology, I'd recommend changing it. If you got it from a source explicitly talking about asymptomatic cases or so-mild-that-you-don't-go-to-the-doctor, then it seems fine to keep it (but maybe change the name).

Edit: Wikipedia says that Diamond Princess had 392/705 asymptomatic cases by 26th February. Given that some of the patients might go on to develop symptoms later on, ≈55% might be an upper bound of asymptomatic cases?

Some relevant quotes from WHO-report (mostly to back up my claims about terminology; Howie questions the validity of the last sentences further down in this thread):

Most people infected with COVID-19 virus have mild disease and recover. Approximately 80% of laboratory confirmed patients have had mild to moderate disease, which includes non-pneumonia and pneumonia cases, 13.8% have severe disease (...) and 6.1% are critical (...). Asymptomatic infection has been reported, but the majority of the relatively rare cases who are asymptomatic on the date of identification/report went on to develop disease. The proportion of truly asymptomatic infections is unclear but appears to be relatively rare and does not appear to be a major driver of transmission.

Comment by lanrian on What will be the big-picture implications of the coronavirus, assuming it eventually infects >10% of the world? · 2020-03-04T18:45:51.270Z · score: 3 (2 votes) · LW · GW

This comment suggests maybe 85% fatality of confirmed cases if they don’t have a ventilator, and 75% without oxygen.

I don't understand how you get those kinds of numbers from the fb-comment, they're way too high. Maybe you mean fatality of severe or critical cases, or survival rates rather than fatality rates. Do you mind clarifying?

Flu is spread out over a few months, and it’s more transmissible than this, and not everyone gets it. (Maybe it’s because of immunity to flu from previous flus?)

Are you saying that the flu is more transmissible than corona? I think I've read that corona is spreading faster, but I don't have a good source, so I'd be curious if you do.

Comment by lanrian on Taking the Outgroup Seriously · 2020-02-16T18:10:46.560Z · score: 5 (3 votes) · LW · GW

I think that the position put forward here could usefully be applied to parts of the post itself.

In particular, I'd say that it's quite uncommon for people to claim that abortion opponents "oppose abortion as part of a conspiracy to curtail women's rights". There's no reason to posit a conspiracy, ie., a large number of people who have discussed and decided on this as a good method of suppressing women. I think a fair number of people claim that abortion opponents are motivated by religious purity norms, though, such that they don't mind inflicting suffering on women who have sex outside of marriages; or perhaps that they generally don't care enough about the welfare of women, because they're misogynist. Justifications for why abortion opponents want to prevent abortions in the first place range from thinking that they hate promiscious women so much that they want to punish them, to acknowledging that they may care about foetuses to some extent, to thinking that they mostly care about repeating religious shibboleths. Some of these seem silly to me, but not all of them.

There's even some evidence that you could cite for these claims, e.g. that abortion opponents are rarely strong supporters of birth control (which I'd guess is the best method of preventing abortions). And there's some arguments you could put forward against this, in turn, namely that people in general are bad at finding the best interventions for the things they care about. I haven't thought about this in depth, but I don't think the most sophisticated version of any of these sides is silly and obviously wrong.

On the margin, I think it'd be good if people moved towards the position that you're advocating in this post. But I don't think it's obvious that people generally "tend to explain their actual reasoning". I think there's often a mix-up between conversation norms and predictions about how people act in the real world, when talking about this kind of thing:

  • In a 1-on-1 conversation, it makes sense to take the other persons view seriously, because if you think they're arguing in bad faith, you should probably just stop talking with them.
  • In conversations with lots of listeners, people might be able to convince a lot of people by putting forth arguments other than those that convinced them, so I can't why we should predict that they always put forward their true reasoning. Whether they mostly do so or not seems like an empirical question with a non-obvious answer (and I would actually quite like to read an analysis of how common this is). However, we still want to strongly endorse and support the norm of responding to points under the assumption that they were made in good faith, because accusations will quickly destroy all value that the conversation might have generated. I think it's extremely important that we have such norms on lesswrong, for example (and I also believe that lesswrongers almost always do argue in good faith, partly because we generally don't discuss hot topics like, uhm, abortion... oops).
  • When thinking about other people's actions outside of conversations with them (whether in our own heads or in conversation with a third party), I think we'd be unnecessarily handicapped if we assumed that people always meant everything they said. If a politician makes a claim, I predict that a person who has "the politician made that claim just to gain votes" somewhere in their hypothesis-space is going to make better predictions than someone who steadfastely applies the conversation norms to their own thoughts.
Comment by lanrian on Some quick notes on hand hygiene · 2020-02-10T09:27:31.239Z · score: 1 (1 votes) · LW · GW

You mean this one? Yeah, that does suggest that there are increasing marginal returns to time spent per hand-washing session, at the typical level of effort.

Comment by lanrian on Some quick notes on hand hygiene · 2020-02-09T11:11:50.196Z · score: 3 (2 votes) · LW · GW

Cool to see that they're in the same ballpark.

“Handwashing can prevent 21% of respiratory sicknesses”—https://globalhandwashing.org/about-handwashing/why-handwashing/health/

Do they say which conditions are being compared? Is it no handwashing at all vs 30 seconds 15 times per day, or something else? (I would look myself, but I can't find the quote with cmd-f.)

I'd guess that washing your hands has some diminishing marginal returns, so if washing your hands for 30 seconds 15 times a day is approximately as good as not washing your hands at all, you can probably do better than both by being somewhere in the middle (e.g. washing your hands for 20 seconds at the 10 points during the day when they're most dirty).

Comment by lanrian on Moral public goods · 2020-02-02T19:24:05.384Z · score: 1 (1 votes) · LW · GW

According to the chart I linked earlier, the countries with highest ODA as %GNI are UAE, Norway, Luxembourg, and Sweden, all at around 0.9 %GNI.

Given random variation between countries, we shouldn't be surprised to find smaller countries on the top of such a list: (i) because there are more small countries than big countries, and (ii) because smaller countries are likely to be more internally homogenous, which means that e.g. the average inclination to give away money among the countries' population is likely to differ more from the global average.

I guessed that I'd find small countries at the bottom of the list, too. But then I actually looked, and found Thailand, Taiwan, Russia, and Romania on the bottom, two of which are big, and all of which are larger than UAE, Norway, Luxembourg, and Sweden. I don't know what's up with that, though part of the explanation might be that a bunch of poor, small countries are grouped as a single big "DAC-countries"-category. Edit: This last sentence is false, see Wei_Dai's comment ("DAC-countries" are apparently rich countries, rather than poor, and each of them are reported separately in the list). Seems like a lot of poor countries aren't included in the list at all.

Comment by lanrian on Moral public goods · 2020-01-26T11:27:32.019Z · score: 5 (3 votes) · LW · GW

I think there has been attempts to coordinate around foreign aid, see for example https://www.oecd.org/dac/stats/the07odagnitarget-ahistory.htm.

Also, several parts of the UN (e.g. WHO) does things that could be classidied as aid, and the UN is funded by member countries.

Comment by lanrian on Dominic Cummings: "we’re hiring data scientists, project managers, policy experts, assorted weirdos" · 2020-01-05T13:13:55.533Z · score: 10 (2 votes) · LW · GW

And yet here's a rationalist who upturned global politics singlehandedly, and credits LessWrong with his success.

Source? I've googled his name and LessWrong, but can't find him saying anything about it.

Comment by lanrian on Can fear of the dark bias us more generally? · 2019-12-22T23:20:18.756Z · score: 3 (3 votes) · LW · GW

This is just anecdotal, but me, a friend, and plausibly Randall Munroe are significantly more socially risk-taking at night than during other times of day. This might be directly connected to the time of day, or just be a consequence of sleep deprivation.

I also have irrational fears during the night, sometimes, but I would guess that this is largely due to being sleepy, stupid, and alone, which causes me to be more suggestible to stray thoughts in general. I wouldn't be surprised if darkness also contributes, though.

Comment by lanrian on [Personal Experiment] One Year without Junk Media · 2019-12-18T09:45:18.710Z · score: 2 (2 votes) · LW · GW

You can sculpt a service like this out of pretty much any service that keeps its old shows around. For example, you can use ublock origin to remove all side-bar video-suggestions from youtube (and also remove comments, if you want). Then you can just forbid yourself from going to the home page (or automatically block it with something like leechblock), and only ever access videos by doing search directly. If you have a way of adding any search engine to your browser (which I recommend getting; I think there are easy ways to do this in most browsers, though I use vimium), you can add youtube.com/results?search_query= or netflix.com/search?q= or whatever you want to it.

Comment by lanrian on How time tracking can help you prioritize · 2019-12-16T22:34:06.245Z · score: 4 (2 votes) · LW · GW

If you want automatic time tracking for a mac (with good support for manual assignment), and you're willing to pay for it, I tested several options two years back and decided that timing was best. I'm still happy with it. A nice thing with automatic time-tracking is that I have decent data for a year where I didn't use it actively at all.

That said, I don't want to ruin the call-to-action by forcing anyone to consider several options. I haven't tried toggle, but it seems like a good choice. Go forth and try time tracking!

Comment by lanrian on When would an agent do something different as a result of believing the many worlds theory? · 2019-12-16T20:01:57.051Z · score: 1 (1 votes) · LW · GW

Wait, what? If compatibilism doesn't suggest that I'm choosing between actions, what am I choosing between?

Comment by lanrian on When would an agent do something different as a result of believing the many worlds theory? · 2019-12-16T18:01:13.647Z · score: 1 (1 votes) · LW · GW

MWI is deterministic, so you can't alter the percentages by any kind of free will, despite what people keep asserting.

Neither most collapse-theories nor MWI allow for super-physical free will, so that doesn't seem relevant to this question. Since the question concerns what one should do, it seems reasonable to assume that some notion of choice is possible.

(FWIW, I'd guess compatibilism is the most popular take on free will on LW.)

Comment by lanrian on Toon Alfrink's sketchpad · 2019-12-12T21:01:46.552Z · score: 2 (2 votes) · LW · GW

We have absolutely no reason to believe that behavior correlates with consciousness.

The strong version of this can't be true. You claiming that you're conscious is part of your behaviour. Hopefully, it's approximately true that you would claim that you're conscious iff you believe that you're conscious. If behaviour doesn't at all correlate with consciousness, it follows that your belief in consciousness doesn't at all correlate with you being conscious. Which is a reductio, because the whole point with having beliefs is to correlate them with the truth.

Comment by lanrian on The Lesson To Unlearn · 2019-12-09T22:14:35.144Z · score: 4 (2 votes) · LW · GW

I'm not sure what you mean with "the spirit of passing tests is hacking them". Do you mean that the tests were intentionally designed to be hackable? Because it seems like Graham is very much not saying that:

Merely talking explicitly about this phenomenon is likely to make things better, because much of its power comes from the fact that we take it for granted. After you've noticed it, it seems the elephant in the room, but it's a pretty well camouflaged elephant. The phenomenon is so old, and so pervasive. And it's simply the result of neglect. No one meant things to be this way. This is just what happens when you combine learning with grades, competition, and the naive assumption of unhackability.

I'm less confident of what Hotel Concierge's point is (partly because it's a damn long essay with many distinct points), but at least the end of it I'd summarize as: "Success correlates with misery, because too much perfectionism contributes to both, and it's a problem when people are pushed and selected towards being too perfectionist". Some relevant passages:

I think the psychopathology term for TDTPT [this means The Desire To Pass Tests] is “perfectionism.” [...]
Perfectionism—like literally everything else—is part of a spectrum, good in moderation and dangerous in overdose. [...]
And so if you select for high TDTPT, if you take only the highest scores and most feverishly dedicated hoop-jumping applicants, then there is no way around it: you are selecting for a high fraction of unhappy people. [...]
I’ve used Scantron-centric examples because Scantrons are easy to quantify, but tests are everywhere, and I promise you that the same trait that made me check my answers ten times over is present in the girl that spends two and a half hours doing her makeup, pausing every five minutes to ask a roommate if she looks ugly. TDTPT is the source of anorexia, body dysmorphia, workaholism, anxiety (“I just can’t find anything to say that doesn’t sound stupid”), obsession, and a hundred million cases of anhedonia, fatigue, and inadequacy [...]

So they're saying that a little TDTPT is good for you, but many people have too much, and that's bad for their mental health. Looking at that last paragraph, the examples aren't particularly connected to the hackability of tests, as far as I can tell. That there are important tests which are testing arbitrary things certainly contribute to the problem, since being perfectionist about meaningless work is less productive and more likely to lead to burnout, but it's not essential to the point, and it's not something that Hotel Concierge emphasizes.

Comment by lanrian on The Lesson To Unlearn · 2019-12-09T18:36:33.968Z · score: 8 (5 votes) · LW · GW

I don't think the essays say the same thing. Paul Graham claims that school in particular encourages hacking tests, and that the spirit of hacking tests is bad for many real-world problems. Hotel Concierge equates the will to do well on tests with perfectionism, and thinks that this helps a lot with success in all parts of life, but causes misery for the perfectionists. The distinction between hackable tests and non-hackable tests is neither emphasized nor necessary for the latter, while it's central to the former.

Comment by lanrian on Karate Kid and Realistic Expectations for Disagreement Resolution · 2019-12-06T15:43:26.500Z · score: 7 (3 votes) · LW · GW
People have a really hard time with interventions often because they literally do not have a functioning causal model of the thing in question. People who apply deliberate practice to a working causal model often level up astonishingly quickly.

Could you give an example of a functioning causal model that quickly helps you learn a topic?

I'm not sure whether you're thinking about something more meta-level, "what can I practice that will cause me to get better", or something more object-level, "how does mechanics work", and I think an example would help clarify. If it's the latter, I'm curious about what the difference is between having a functioning causal model of the subject (the precondition for learning) and being good at the subject (the result of learning).

Comment by lanrian on Effect of Advertising · 2019-11-27T21:25:33.940Z · score: 2 (2 votes) · LW · GW
Maybe I am too negative about advertising, but it seems like its major strategy is to annoy me. Like that advertisement I won't mention that I have recently seen (the first five seconds of) perhaps several hundred times, because YouTube plays it at the beginning of almost every video I see.

FYI, adblockers (like ublock origin) work fine to prevent all of youtube's ads, including the video ones.

Comment by lanrian on Hazard's Shortform Feed · 2019-11-25T07:59:03.106Z · score: 1 (1 votes) · LW · GW

That does seem to change things... Although I'm confused about what simplicity is supposed to refer to, now.

In a pure bayesian version of this setup, I think you'd want some simplicity prior over the worlds, and then discard inconsistent worlds and renormalize every time you encounter new data. But you're not speaking about simplicity of worlds, you're speaking about simplicity of propositions, right?

Since a propositions is just a set of worlds, I guess you're speaking about the combined simplicity of all the worlds. And it makes sense that that would increase if the proposition is consistent with more worlds, since any of the worlds would indeed lead to the proposition being true.

So now I'm at "The simplicity of a proposition is proportional to the prior-weighted number of worlds that it's consistent with". That's starting to sound closer, but you seem to be saying that "The simplicity of a proposition is proportional to the number of other propositions that it's consistent with"? I don't understand that yet.

(Also, in my formulation we need some other kind of simplicity for the simplicity prior.)

Comment by lanrian on Hazard's Shortform Feed · 2019-11-24T19:39:37.532Z · score: 1 (1 votes) · LW · GW

Roughly, A is simpler than B if all data that is consistent with A is a subset of all data that is consistent with B.

Maybe the less rough version is better, but this seems like a really bad formulation. Consider (a) an exact enumeration of every event that ever happened, making no prediction of the future, vs (b) the true laws of physics and the true initial conditions, correctly predicting every event that ever happened and every event that will happen.

Intuitively, (b) is simpler to specify, and we definitely want to assign (b) a higher prior probability. But according to this formulation, (a) is simpler, since all future events are consistent with (a), while almost none are consistent with (b). Since both theories have equally much evidence, we'd be forced to assign higher probability to (a).

Comment by lanrian on RAISE post-mortem · 2019-11-24T19:28:51.358Z · score: 14 (7 votes) · LW · GW

Good post!

Maybe this is too nitpicky, but "the most impactful years of your life will be 100x more impactful than the average" is necessarily false, because your career is so short that those years will increase the average. For example, if you have a 50-year career and all of your impact happens during a period of two years, your average yearly impact is 2/50=1/25 times as high as your impact during those two years. However, "the most impactful years of your life will be 100x more impactful than the median" could be true.

Comment by lanrian on Building Intuitions On Non-Empirical Arguments In Science · 2019-11-07T20:33:31.702Z · score: 8 (4 votes) · LW · GW

I feel like this is mostly a question of what you mean with "atlantis".

  • If you want to calculate P(evidence | the_specific_atlantis_that_newagers_specified_after_hearing_the_evidence) * P(the_specific_atlantis_that_newagers_specified_after_hearing_the_evidence), then the first term is going to be pretty high, and the second term would be very low (because it specifies a lot of things about what the atleantans did).
  • But if you want to calculate P(evidence | the_type_of_atlantis_that_people_mostly_associate_to_before_thinking_about_the_sphinx) * P(the_type_of_atlantis_that_people_mostly_associate_to_before_thinking_about_the_sphinx), the first term would be very low, while the second term would be somewhat higher.

The difference between the two cases is whether you think about the new agers as holding exactly one hypothesis and lying about what it predicts (as it cannot assign high probability to all of the things, since you're correct that the different probabilities must sum to 1), or whether you think about the new agers as switching to a new hypothesis every time they discover a new fact about the sphinx / every time they're asked a new question.

In this particular article, Scott mostly wants to make a point about cases where theories have similar P(E|T) but differ in the prior probabilities, so he focused on the first case.

Comment by lanrian on Normative reductionism · 2019-11-07T19:54:33.906Z · score: 4 (3 votes) · LW · GW

It’s harmless (but silly (note: I LIKE silly, in part because it’s silly to do so)) to have such preferences, and usually harmless to act on them.

I don't really understand why preferences about things that you can't observe are more silly than other preferences, but that's ok. I mostly wanted to clear up the terminology, and note that it seems more like common usage of 'preference' and 'utility' to say "That's a silly preference to have, because X, Y, Z" and "I think we should only care about things that can affect us" instead of saying "Your satisfaction of that preference has nothing to do with their confidence, it’s all about whether you actually find out" and "Without some perceptible difference, your utility cannot be different".

Comment by lanrian on Normative reductionism · 2019-11-07T17:52:23.940Z · score: -2 (2 votes) · LW · GW

Does it help if you don't think about a 'preference' as something ontologically fundamental, but just as a convenient shorthand for something that an agent is optimising for? It's certainly possible for an agent to optimise for something even if they'll never receive any evidence of if they succeeded. gjm gives a few examples in the sibling-comment to mine.

Comment by lanrian on Normative reductionism · 2019-11-06T20:00:11.808Z · score: 1 (1 votes) · LW · GW

I've personally used total consequentialism for this in the past (when arguing that non-causal decision theories + total consequentialism implies that we should assume that alien civilisations are common) and would support it being standard terminology. Many people know what total utilitarianism is, and making the switch for consequentialism is quite intuitive.

Comment by lanrian on Normative reductionism · 2019-11-06T19:30:53.058Z · score: 1 (1 votes) · LW · GW

Without some perceptible difference, your utility cannot be different.

This definition of "utility" (and your definition of "preference") is different from the one that most LWers use, different from the one that economists use, and different from the one that (at least some) professional philosophers use.

Ecomomists use it to define any preference ordering over worlds, and don't require it to be defined only over your own experiences. Some ethical theories in philosophy (e.g. hedonistic utilitarianism) define it as a direct function of your experiences, but others, (e.g. preference utilitarianism) define it as something that can be affected by things you don't know about. As evidence for the latter, this SEP page states:

If a person desires or prefers to have true friends and true accomplishments and not to be deluded, then hooking this person up to the experience machine need not maximize desire satisfaction. Utilitarians who adopt this theory of value can then claim that an agent morally ought to do an act if and only if that act maximizes desire satisfaction or preference fulfillment (that is, the degree to which the act achieves whatever is desired or preferred). What maximizes desire satisfaction or preference fulfillment need not maximize sensations of pleasure when what is desired or preferred is not a sensation of pleasure. This position is usually described as preference utilitarianism.

If you're a hedonistic utilitarian, feel free to argue for hedonistic utilitarianism, but do that directly instead of making claims about what other people are or aren't allowed to have preferences about.

Comment by lanrian on Normative reductionism · 2019-11-06T00:10:02.166Z · score: 8 (4 votes) · LW · GW

This SEP page defines:

Aggregative Consequentialism = which consequences are best is some function of the values of parts of those consequences (as opposed to rankings of whole worlds or sets of consequences).

Total Consequentialism = moral rightness depends only on the total net good in the consequences (as opposed to the average net good per person).

as two related and potentially identical concepts. When defining normative reductionism, do you mean that the value of the world is equal to the sum of the value of its parts? If so, total consequentialism is probably the closest term (though it's a bit unfortunate that they only contrast it with average utilitarianism).

Comment by lanrian on jacobjacob's Shortform Feed · 2019-10-30T22:50:02.880Z · score: 13 (8 votes) · LW · GW

(Potential reason for confusion: "don't endorse it" in habryka's first comment could be interpreted as not endorsing "this comment", when habryka actually said he didn't endorse his emotional reaction to the comment.)

Comment by lanrian on Why are people so bad at dating? · 2019-10-28T18:51:06.860Z · score: 71 (30 votes) · LW · GW

This seems like a misapplication of the concept of efficiency. The reason that a $20 bill on the ground is surprising is that a single competent agent would be enough to remove it from the world. Similarly, the reason that the efficient market hypothesis is a good approximation isn't that everyone who invests in the stock market is rational; instead, it's that a few highly informed individuals working full time are doing a great job at using up inefficiencies, which causes them to go away.

For every example that you pick, it's certainly true that some people are taking advantage of it (some people are using PhotoFeeler, some people have read Mate, etc), but there's no reason why this would translate into the advantages going away, or would automatically lead to everyone in the dating scene doing it. (Indeed, if someone is highly successful at dating, they're more likely to disappear from the dating scene than to stay in it.) Thus, it's highly disanalogous to efficient markets.

My main point is that humans are frequently unstrategic and bad, absent a lot of time investment and/or selection effects, so there's no particular reason to expect them to be great at dating. It may be true that they're even worse at dating than we would expect, but to draw that conclusion, the relevant comparisons are other things that lay people do in their spare time (ryan_b mentions job search, which seems like a good comparison), while theories assuming perfect rationality are unlikely to be useful.

(Another reason that humans are sometimes good at things is when they were highly useful for reproduction in the ancestral environment. While finding a mate was certainly useful, all of the mentioned examples concern things that have only become relevant during the past few hundred years, so it's not surprising that we're not optimised to make use of them.)

Comment by lanrian on The Dualist Predict-O-Matic ($100 prize) · 2019-10-24T10:44:28.253Z · score: 1 (1 votes) · LW · GW

SGD is not going to play the future forward to see the new feedback mechanism you’ve described and incorporate it into the loss function which is being minimized

My 'new feedback mechanism' is part of the training procedure. It's not going to be good at that by 'playing the future forward', it's going to become good at that by being trained on it.

I suspect we're using SGD in different ways, because everything we've talked about seems like it could be implemented with SGD. Do you agree that letting the Predict-O-Matic predict the future and rewarding it for being right, RL-style, would lead to it finding fixed points? Because you can definitely use SGD to do RL (first google result).

Comment by lanrian on The Dualist Predict-O-Matic ($100 prize) · 2019-10-23T09:45:17.223Z · score: 1 (1 votes) · LW · GW

Assuming that people don't think about the fact that Predict-O-Matic's predictions can affect reality (which seems like it might have been true early on in the story, although it's admittedly unlikely to be true for too long in the real world), they might decide to train it by letting it make predictions about the future (defining and backpropagating the loss once the future comes about). They might think that this is just like training on predefined data, but now the Predict-O-Matic can change the data that it's evaluated against, so there might be any number of 'correct' answers (rather than exactly 1). Although it's a blurry line, I'd say this makes it's output more action-like and less prediction-like, so you could say that it makes the training process a bit more RL-like.

Comment by lanrian on Sets and Functions · 2019-10-22T19:15:59.560Z · score: 1 (1 votes) · LW · GW

As someone with half an undergrads worth of math background, I've found these posts useful to grasp the purpose and some of the basics of category theory. It might be true that there's exist some exposition out there which would work better, but I haven't found/read that one, and I'm happy that this one exists (among other things, it has the not-to-be-underestimated virtue of being uneffortful to read). Looking forward to the Yoneda and adjunction posts!

Comment by lanrian on The Dualist Predict-O-Matic ($100 prize) · 2019-10-22T09:26:22.640Z · score: 1 (1 votes) · LW · GW
Yes, that sounds more like reinforcement learning. It is not the design I'm trying to point at in this post.

Ok, cool, that explains it. I guess the main differences between RL and online supervised learning is whether the model takes actions that can affect their environment or only makes predictions of fixed data; so it seems plausible that someone training the Predict-O-Matic like that would think they're doing supervised learning, while they're actually closer to RL.

That description sounds a lot like SGD. I think you'll need to be crisper for me to see what you're getting at.

No need, since we already found the point of disagreement. (But if you're curious, the difference is that sgd makes a change in the direction of the gradient, and this one wouldn't.)

Comment by lanrian on The Dualist Predict-O-Matic ($100 prize) · 2019-10-18T10:58:38.810Z · score: 1 (1 votes) · LW · GW

I think our disagreement comes from you imagining offline learning, while I'm imagining online learning. If we have a predefined set of (situation, outcome) pairs, then the Predict-O-Matic's predictions obviously can't affect the data that it's evaluated against (the outcome), so I agree that it'll end up pretty dualistic. But if we put a Predict-O-Matic in the real world, let it generate predictions, and then define the loss according to what happens afterwards, a non-dualistic Predict-O-Matic will be selected for over dualistic variants.

If you still disagree with that, what do you think would happen (in the limit of infinite training time) with an algorithm that just made a random change proportional to how wrong it was, at every training step? Thinking about SGD is a bit complicated, since it calculates the gradient while assuming that the data stays constant, but if we use online training on an algorithm that just tries things until something works, I'm pretty confident that it'd end up looking for fixed points.

Comment by lanrian on The Dualist Predict-O-Matic ($100 prize) · 2019-10-17T09:24:34.155Z · score: 5 (4 votes) · LW · GW
If dualism holds for Abram’s prediction AI, the “Predict-O-Matic”, its world model may happen to include this thing called the Predict-O-Matic which seems to make accurate predictions—but it’s not special in any way and isn’t being modeled any differently than anything else in the world. Again, I think this is a pretty reasonable guess for the Predict-O-Matic’s default behavior. I suspect other behavior would require special code which attempts to pinpoint the Predict-O-Matic in its own world model and give it special treatment (an “ego”).

I don't see why we should expect this. We're told that the Predict-O-Matic is being trained with something like sgd, and sgd doesn't really care about whether the model it's implementing is dualist or non-dualist; it just tries to find a model that generates a lot of reward. In particular, this seems wrong to me:

The Predict-O-Matic doesn't care about looking bad, and there's nothing contradictory about it predicting that it won't make the very prediction it makes, or something like that.

If the Predict-O-Matic has a model that makes bad prediction (i.e. looks bad), that model will be selected against. And if it accidentally stumbled upon a model that could correctly think about it's own behaviour in a non-dualist fashion, and find fixed points, that model would be selected for (since its predictions come true). So at least in the limit of search and exploration, we should expect sgd to end up with a model that finds fixed points, if we train it in a situation where its predictions affect the future.

If we only train it on data where it can't affect the data that it's evaluated against, and then freeze the model, I agree that it probably won't exhibit this kind of behaviour; is that the scenario that you're thinking about?

Comment by lanrian on Misconceptions about continuous takeoff · 2019-10-09T18:09:36.060Z · score: 9 (3 votes) · LW · GW
Possibly you'd want to rule out (c) with your stipulation that the tests are "robust"? But I'm not sure you can get tests that robust.

That sounds right. I was thinking about an infinitely robust misalignment-oracle to clarify my thinking, but I agree that we'll need to be very careful with any real-world-tests.

If I imagine writing code and using the misalignment-oracle on it, I think I mostly agree with Nate's point. If we have the code and compute to train a superhuman version of GPT-2, and the oracle tells us that any agent coming out from that training process is likely to be misaligned, we haven't learned much new, and it's not clear how to design a safe agent from there.

I imagine a misalignment-oracle to be more useful if we use it during the training process, though. Concretely, it seems like a misalignment-oracle would be extremely useful to achieve inner alignment in IDA: as soon as the AI becomes misaligned, we can either rewind the training process and figure out what we did wrong, or directly use the oracle as a training signal that severely punish any step that makes the agent misaligned. Coupled with the ability to iterate on designs, since we won't accidentally blow up the world on the way, I'd guess that something like this is more likely to work than . This idea is extremely sensitive to (c), though.