Posts

What are we predicting for Neuralink event? 2019-07-12T19:33:57.759Z · score: 33 (12 votes)
LW Dev question: FB-style tagging? 2019-06-20T19:19:45.807Z · score: 11 (3 votes)
Using rationality to debug Machine Learning 2018-04-10T20:03:44.357Z · score: 55 (12 votes)
Bayesian statistics as epistemic advantage 2017-07-25T17:07:49.660Z · score: 0 (0 votes)
MILA gets a grant for AI safety research 2017-07-21T15:34:55.493Z · score: 9 (9 votes)
Learning from Human Preferences - from OpenAI (including Christiano, Amodei & Legg) 2017-06-13T15:52:00.294Z · score: 9 (9 votes)
NIPS 2015 2015-12-07T20:31:11.779Z · score: 7 (8 votes)
Velocity of behavioral evolution 2014-12-19T17:34:36.217Z · score: 3 (4 votes)
What Peter Thiel thinks about AI risk 2014-12-11T21:22:27.167Z · score: 12 (13 votes)
Cognitive distortions of founders 2014-12-11T03:19:58.802Z · score: 3 (6 votes)
FAI PR tracking well [link] 2014-08-15T21:23:48.622Z · score: 7 (8 votes)
Hawking/Russell/Tegmark/Wilczek on dangers of Superintelligent Machines [link] 2014-04-21T16:55:56.240Z · score: 18 (21 votes)
Gunshot victims to be suspended between life and death [link] 2014-03-27T16:33:49.413Z · score: 24 (27 votes)
Huffington Post article on DeepMind-requested AI ethics board, links back to LW [link] 2014-01-30T01:20:10.579Z · score: 13 (26 votes)
H+ review of James Miller's Singularity Rising [link] 2014-01-17T02:16:08.126Z · score: -4 (9 votes)
PSA for LW futurists/academics 2013-10-31T15:54:39.861Z · score: 7 (12 votes)
Nudging around the world - [link] 2013-09-05T15:52:05.786Z · score: 8 (10 votes)
Course - Saving Millions of lives at a time [link] 2013-08-12T16:30:29.192Z · score: 4 (7 votes)
Proposal: periodic repost of the Best Learning resources 2013-08-10T16:21:19.138Z · score: 9 (10 votes)
RIP Doug Engelbart 2013-07-03T19:19:59.616Z · score: 11 (16 votes)
NES-game playing AI [video link and AI-boxing-related comment] 2013-04-12T13:11:36.365Z · score: 30 (35 votes)
Eliezer's YU lecture on FAI and MOR [link] 2013-03-07T16:09:54.710Z · score: 2 (9 votes)
Interesting discussion of concentration and productivity [link] 2013-02-06T13:58:35.082Z · score: 8 (13 votes)
CFAR’s Inaugural Fundraising Drive 2012-12-18T01:19:00.272Z · score: 10 (13 votes)
Popular media coverage of Singularity Summit -the Verge [link] 2012-10-23T03:19:03.676Z · score: 2 (7 votes)
Judea Pearl's Turing Award Lecture video now online 2012-09-10T15:18:43.251Z · score: 6 (9 votes)
Luke' AMA gets a plug @ Wired [link] 2012-08-26T04:15:49.745Z · score: 6 (13 votes)
Opinions on Boltzmann brain arguments constraining modern multiverse theories 2012-08-10T18:05:52.417Z · score: 1 (6 votes)
Russian plan for immortality [link] 2012-08-01T20:49:41.319Z · score: 5 (10 votes)
SIAI May report 2012-06-16T14:01:37.488Z · score: 5 (10 votes)
Arrison, Vassar and de Grey speaking at Peter Thiel's startup class [link] 2012-06-16T13:58:35.578Z · score: 10 (13 votes)
Sebastian Thrun AMA on reddit [link] 2012-06-14T02:53:15.258Z · score: 4 (7 votes)
MIT is working on industrial robots that (attempt to) learn what humans want from them [link] 2012-06-13T15:42:58.015Z · score: 3 (8 votes)
Peter Thiel's AGI discussion in his startups class @ Stanford [link] 2012-06-07T12:27:48.075Z · score: 13 (16 votes)
Audio interview with Judea Pearl [link] 2012-05-10T12:47:07.615Z · score: 7 (8 votes)
"Big Surprise" - the famous atheists are actually Bayesians [link] 2012-04-08T16:11:30.166Z · score: -9 (20 votes)
Singularity-oriented Sci-Fi collection to be published [link] 2012-02-29T16:39:40.714Z · score: 3 (6 votes)
Model Thinking class [link] 2012-02-20T18:27:51.909Z · score: 6 (7 votes)
Top 5 regrets of the dying [link] 2012-02-04T17:56:22.156Z · score: 3 (10 votes)
Bill Gates asks HS students "What are most important choices the world faces"? 2012-01-11T20:34:11.272Z · score: 0 (11 votes)
Nick Bostrom TED talk on world's biggest problems 2012-01-06T18:52:09.991Z · score: 17 (20 votes)
Roger Williams (Author of Metamorphosis of Prime Intellect) on Singularity 2012-01-06T05:09:36.511Z · score: 5 (14 votes)
Future of Moral Machines - New York Times [link] 2011-12-26T14:44:01.763Z · score: 0 (5 votes)
SingInst bloomberg coverage [link] 2011-12-19T19:31:41.651Z · score: 5 (8 votes)
For those in the Stanford AI-class 2011-12-10T22:54:38.294Z · score: 8 (11 votes)
Good interview with Kahneman [link] 2011-12-02T15:04:24.480Z · score: 2 (3 votes)
FAI-relevant XKCD 2011-11-22T13:28:44.070Z · score: -4 (13 votes)
Three more classes coming from Stanford of interest here 2011-11-17T17:57:42.468Z · score: 15 (16 votes)
Human augmented with CBT algorithms games Jeopardy [link] 2011-11-17T03:22:25.901Z · score: 10 (11 votes)
Biointelligence Explosion 2011-11-07T14:05:08.985Z · score: 4 (16 votes)

Comments

Comment by dr_manhattan on Covid 8/6: The Case of the Missing Data · 2020-08-06T21:48:57.695Z · score: 4 (2 votes) · LW · GW

thought this was interesting

https://twitter.com/HealthAllegheny/status/1291466180386062340?s=20

Comment by dr_manhattan on Be impatient · 2020-07-28T19:18:23.490Z · score: 8 (3 votes) · LW · GW

Sounds like vaguely-good conclusions from my pattern-matching experience but very poorly argued, with much overloading of "impatience" and many cherry picked examples. Surprisingly bad quality from you. Also, "patience" is a great virtue, context matters a lot.

Comment by dr_manhattan on Credibility of the CDC on SARS-CoV-2 · 2020-03-09T14:59:54.730Z · score: 0 (2 votes) · LW · GW

Ok. To clarify, one of them is to blame. Maybe it's not the CDC. History will tell.

Comment by dr_manhattan on Credibility of the CDC on SARS-CoV-2 · 2020-03-09T14:09:19.561Z · score: 2 (1 votes) · LW · GW
Most obviously, blaming the CDC for the FDA and HHS not allowing 3rd party detection kits is somewhere between false and misleading.

Please support this claim. It seems obvious that they shat the bed (don't know which agency, let god sort them out for now, history and FOIA requests will sort them out in the future). It seems obvious from reading the news that many many local and commercial labs would have been ready with capacity a lot sooner than they are if FDA/CDC/HHS conglomerate got out of the way sooner.

It's quite plausible that this is due to Trump pressure, history will sort this out, but my estimation of guilt will likely just move from "weasel" to "weak for not resisting", and the facts remain the same

Comment by dr_manhattan on What are the long-term outcomes of a catastrophic pandemic? · 2020-03-03T12:37:19.890Z · score: 2 (1 votes) · LW · GW
  • Significant political regime disruption in some places, Iran specifically (probably for the better there)
  • Depending on how much Chinese government fudges with the current numbers they will appear to a) have bungled at first b) actually efficiently handled the problem at an enormous scale and US will appear to do things in the opposite order. Which leads me to...
  • Trump is weakened and Bloomberg's position is going to look very strong: some actual history of crisis management, strong-man people feel comfortable with in times of uncertainty, plus he "owns" one of coutry's top med schools
  • Massive disruption in service sector as people cut down on non-necessities...
Comment by dr_manhattan on Coronavirus: Justified Practical Advice Thread · 2020-03-01T22:03:05.603Z · score: 10 (7 votes) · LW · GW

This is useful in case you have facing a choice of riding it out at home and going to a hospital with high probability of getting infected if you're not already. E.g. if you have fever chances are still high you're just experiencing regular flu, and should not go to the hospital, but if your oxigen starts dropping into the danger zone you need to go.

Comment by dr_manhattan on Draft: Models of Risks of Delivery Under Coronavirus · 2020-02-29T21:10:56.109Z · score: 4 (5 votes) · LW · GW

Potential method of coping: disinfecting room. Unpack the stuff in protective gear, then after unpacked blast it with UV light?

Comment by dr_manhattan on Coronavirus: Justified Practical Advice Thread · 2020-02-29T15:24:33.879Z · score: 4 (2 votes) · LW · GW
But it's possible we could even isolate there in the in-law unit.

I thought this was the pretty clear cut answer before you wrote it. Totally endorse. Wear masks on the flight if possible. Ask your parents to stock up or start sending prep packages there (Amazon, Costco delivers)

Comment by dr_manhattan on Quarantine Preparations · 2020-02-26T12:22:44.070Z · score: 6 (3 votes) · LW · GW
The first question for me is are people starving in Wuhan due to the outbreak?

Answer is no, as of now, though food situation is uncomfortable. (my wife has relatives there she's in contact with). Trucks come to apartment complexes and people pick up.

I'm not sure the analogy translates well to US though. For better or worse us people are less organized. Also large % population live in suburbs where such deliveries are not feasible.

OTOH we have an excellent general delivery system in Amazon, UPS etc.

I'm slightly worried.

Comment by dr_manhattan on Making Sense of Coronavirus Stats · 2020-02-23T18:08:55.222Z · score: 2 (1 votes) · LW · GW

There's some but not a lot of interest in this topic on LW; I have a mailing list with primarily rationalist types on the topic; PM me email address to be added

Comment by dr_manhattan on Making Sense of Coronavirus Stats · 2020-02-21T13:40:17.167Z · score: 3 (2 votes) · LW · GW

Data always says something unless it's randomly generated. At the very least Chinese data provides lower bounds on some things. You can get somewhat better estimates if you model their incentives (though the lying will greatly increase the uncertainty and complexity of any model)

Comment by dr_manhattan on Rationalist prepper thread · 2020-01-30T13:42:52.951Z · score: 2 (1 votes) · LW · GW

There's the opposite effect: the early diagnosed cases tend to concentrate on the more obviously serious ones (more likely to die).

https://twitter.com/robertwiblin/status/1222577306515976192

Comment by dr_manhattan on Rationalist prepper thread · 2020-01-28T18:14:46.858Z · score: 5 (3 votes) · LW · GW
It is also good to invest in improving ones’ immune system by health food, vitamins, light therapy, as it is our best protection of the virus. Evacuation into a cold county house would weaken the immune system.

How much can one "improve" one's immune system by these methods in a short time? Is there any data to back this up?

In general agree with the rest. In worst-case scenario ability to self-isolate for a while ("bug in" in prepper lingo) seems worthwhile.

Comment by dr_manhattan on CFAR Participant Handbook now available to all · 2020-01-03T17:13:40.146Z · score: 3 (2 votes) · LW · GW

What do you use to listen to pdfs?

Comment by dr_manhattan on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2019-12-27T13:29:47.640Z · score: 0 (2 votes) · LW · GW

On the one hand this post does a great job of connecting to previous work, leaving breadcrumbs and shortening the inferential distance. On the other hand what is this at the end?

But one thing I'm pretty sure won't help much is clever logic puzzles about implausibly sophisticated Nazis.

I have no idea what this is talking about.

Comment by dr_manhattan on T-Shaped Organizations · 2019-12-18T13:39:23.196Z · score: 4 (2 votes) · LW · GW
[edit] It also seems like this is the sort of thing that marketing pretty strongly encourages misrepresentation of. "All children are above-average," in that the restaurant wants to present itself as serving healthy, cheap, tasty food, while also paying its employees well and having good returns for its investors. But several of those variables are in direct tension with each other, and there's not great language for speaking publicly about the tradeoffs you're making. 

Couple of reasons spring to mind:

  • Marketing leverages the Halo Effect
  • Not emphasizing being above average on a particular dimension is a chink in the armor against competitors (who can establish beach head by claiming to be superior in that dimension)

Comment by dr_manhattan on Approval Extraction Advertised as Production · 2019-12-16T20:09:49.582Z · score: 5 (3 votes) · LW · GW

The way I understand the objection it that YC promotes "building great products", which attracts (a lot of) certain kinds of founders, but in fact YC is optimizing for something else (primarily described in Black Swan Farming, confirmed by other sources). I believe they are quite value-additive to the companies they accept, but attract more founders than if they were "honest about their optimization function", where some founders could have been better off engaging with other VCs on possibly better terms.

Comment by dr_manhattan on Rule Thinkers In, Not Out · 2019-12-13T17:11:04.824Z · score: 7 (3 votes) · LW · GW
>> Fiddly puttering with something that fascinates you is the source of most genuine productivity. (Anything from hardware tinkering, to messing about with cost spreadsheets until you find an efficiency, to writing poetry until it “comes out right”.) Sometimes the best work of this kind doesn’t look grandiose or prestigious at the time you’re doing it.
Hmm, I use to spend quite a bit of time fiddling with assembly language implementations of encryption code to try to squeeze out a few more percent of speed. Pretty sure that is not as productive as more "grandiose" or "prestigious" activities like thinking about philosophy or AI safety, at least for me... I think overall I'm more afraid that someone who could be doing productive "grandiose" work chooses not to in favor of "fiddly puttering", than the reverse.

I suspect this might be a subtler point?

http://paulgraham.com/genius.html

suggests really valuable contributions are more bottlenecked on obsession rather than being good at directing attention in a "valuable" direction

For example, for the very ambitious, the bus ticket theory suggests that the way to do great work is to relax a little. Instead of gritting your teeth and diligently pursuing what all your peers agree is the most promising line of research, maybe you should try doing something just for fun. And if you're stuck, that may be the vector along which to break out.
Comment by dr_manhattan on Rule Thinkers In, Not Out · 2019-12-13T17:03:53.876Z · score: 8 (4 votes) · LW · GW

I think in part these could be "lessons relevant to Sarah", a sort of a philosophical therapy that can't be completely taken out of context. Which is why some of these might seem of low relevance or obvious.

Comment by dr_manhattan on Rule Thinkers In, Not Out · 2019-12-13T16:55:59.912Z · score: 2 (1 votes) · LW · GW
Fiddly puttering with something that fascinates you is the source of most genuine productivity. (Anything from hardware tinkering, to messing about with cost spreadsheets until you find an efficiency, to writing poetry until it "comes out right".) Sometimes the best work of this kind doesn't look grandiose or prestigious at the time you're doing it.

http://paulgraham.com/genius.html seems to be promoting a similar idea

Comment by dr_manhattan on Bayesian examination · 2019-12-11T13:25:47.813Z · score: 2 (1 votes) · LW · GW

I'll claim LW priority for pointing to the idea (but not to elaborating it in a post) https://www.lesswrong.com/posts/isSMDR8rMr5pTzJK5/example-of-poor-decision-making-under-pressure-from-game?commentId=YzvDcA357NboxD2fE :)

Comment by dr_manhattan on Bayesian examination · 2019-12-09T22:24:52.982Z · score: 9 (4 votes) · LW · GW

Ross https://web.stanford.edu/~shachter/ uses something like this to score answers to (appropriately) Decision Analysis homework questions. (Don't remember the exact rule, but the intent was the same)

Comment by dr_manhattan on Ways that China is surpassing the US · 2019-11-04T13:28:27.435Z · score: 1 (4 votes) · LW · GW
What does it imply for things like AI governance and global coordination on x-risks?

I've read the article a while ago, and vaguely concluded there should be some implications here (but largely uncertain about the direction or magnitude, being a non-expert). Interested to hear what people think (esp. people who concentrate on policy)

Comment by dr_manhattan on Unstriving · 2019-08-22T12:14:31.776Z · score: 2 (1 votes) · LW · GW
How about we let go of success, but keep doing challenging stuff anyway, just for the fun of it?

This sort of feels like Feynman's attitude, despite him being extremely successful.

Comment by dr_manhattan on GPT-2: 6-Month Follow-Up · 2019-08-21T12:41:46.516Z · score: 6 (3 votes) · LW · GW

Also notable: NVIDIA trained a half-order-of-magnitude larger model https://nv-adlr.github.io/MegatronLM?utm_campaign=NLP%20News&utm_medium=email&utm_source=Revue%20newsletter

Comment by dr_manhattan on What are we predicting for Neuralink event? · 2019-07-17T19:27:57.727Z · score: 7 (3 votes) · LW · GW

Seems way off from the actual release; any post-mortem?

Comment by dr_manhattan on Integrity and accountability are core parts of rationality · 2019-07-16T14:55:07.325Z · score: 4 (2 votes) · LW · GW

A decent solution to the "who should you be accountable to", from the wisdom of the ancients (shows thought on many of the considerations mentioned)

When in doubt, remember Warren Buffett’s rule of thumb: “… I want employees to ask themselves whether they are willing to have any contemplated act appear the next day on the front page of their local paper—to be read by their spouses, children and friends—with the reporting done by an informed and critical reporter.”
Comment by dr_manhattan on What are we predicting for Neuralink event? · 2019-07-16T14:17:51.917Z · score: 2 (1 votes) · LW · GW

(might contain spoilers/private info) https://www.technologyreview.com/s/613961/elon-musks-brain-interface-company-is-promising-big-news-heres-what-it-could-be/

Comment by dr_manhattan on Open Thread July 2019 · 2019-07-10T15:59:46.855Z · score: 2 (1 votes) · LW · GW

I think the Hypothesis is not about Open Threads specifically

Comment by dr_manhattan on What would be the signs of AI manhattan projects starting? Should a website be made watching for these signs? · 2019-07-03T20:55:52.786Z · score: 4 (2 votes) · LW · GW

Tracking employment/location and publishing/conference attendance records of researchers will probably be good source data for this.

Comment by dr_manhattan on What was the official story for many top physicists congregating in Los Alamos during the Manhattan Project? · 2019-07-03T20:53:47.200Z · score: 9 (5 votes) · LW · GW

I think it was easier in that era; AFAIK they used conventional secrecy methods (project names, locations, misdirection, need to know, obfuscation) to pull it off. Feynman's "Surely you're joking" and Rhodes "making the atomic bomb" are good sources for some examples (and otherwise recommended)

Comment by dr_manhattan on Contest: $1,000 for good questions to ask to an Oracle AI · 2019-07-01T21:27:06.741Z · score: 4 (2 votes) · LW · GW

Small typo:

Hence it has no motivation to manipulate[d] humans through its answer.
Comment by dr_manhattan on What is the evidence for productivity benefits of weightlifting? · 2019-06-27T16:34:19.929Z · score: 1 (3 votes) · LW · GW

IN MICE

Comment by dr_manhattan on Jordan Peterson on AI-FOOM · 2019-06-27T12:34:12.255Z · score: 6 (3 votes) · LW · GW

I somewhat overlooked this line and yes, it's a nod in the right direction

Comment by dr_manhattan on Jordan Peterson on AI-FOOM · 2019-06-26T19:19:06.026Z · score: 23 (11 votes) · LW · GW

Based on the transcript this does not sound like a FOOM discussion (as in rapid self-improvement) other than mentioning "group learning" by autonomous cars, which is maybe somewhat related. Also the pregnancy ad story is much more about pattern recognition with lots of data than any serious AI.

Basically JP is, in this area, a complete layman (unlike Gates, Musk, or, from the other side, Pinker) whose opinion counts for little and not talking about FOOM anyway.

Comment by dr_manhattan on Being the (Pareto) Best in the World · 2019-06-25T12:32:24.217Z · score: 18 (8 votes) · LW · GW

Is this much different from Scott Adams' advice https://dilbertblog.typepad.com/the_dilbert_blog/2007/07/career-advice.html

of


if you want something extraordinary, you have two paths:
1. Become the best at one specific thing.
2. Become very good (top 25%) at two or more things.

?


Comment by dr_manhattan on Reason isn't magic · 2019-06-20T19:10:45.161Z · score: 2 (1 votes) · LW · GW

Do you mean Zvi's "Change is bad"?

Comment by dr_manhattan on Reason isn't magic · 2019-06-19T14:34:39.633Z · score: 3 (2 votes) · LW · GW
Second, we're not actually comparing reason to tradition - we're comparing changing things to not changing things. Change, as we know, is bad.

Request for clarification: isn't "reasonable solution" always a "change" when compared to preexisting tradition?

Comment by dr_manhattan on Counterfactuals about Social Media · 2019-04-23T15:50:52.829Z · score: 3 (3 votes) · LW · GW

I get that my usage hurts indirectly, my question was specifically if everyone used FB occasionally and for similar purposes as I do, would FB still be detrimental? Harming other people because they have unhealthy patterns of usage is still a concern but a lesser one to me

Comment by dr_manhattan on Counterfactuals about Social Media · 2019-04-22T18:15:02.660Z · score: 10 (7 votes) · LW · GW

I use FB occasionally to stay in touch with family/friends.

I subscribe to interesting people on Twitter and find it a great source of intellectual information.

I know these are harmful to some people, and I've occasionally noticed addictive behavior in myself, but overall seems like a good trade. If someone wants to explain/convince me that this is highly dangerous to me in a non-obvious way or that **my kind** of usage is endangering the commons I'm open to hear it.

Comment by dr_manhattan on Crypto quant trading: Intro · 2019-04-21T11:56:25.406Z · score: 2 (1 votes) · LW · GW

Are you working with Satvik? I know he was into this stuff.

Comment by dr_manhattan on Crypto quant trading: Intro · 2019-04-18T17:31:36.799Z · score: 5 (3 votes) · LW · GW

I have 1 foot in finance, and still learning a lot. As it appears you self-taught this stuff, what were good resources for you?

Comment by dr_manhattan on How do people become ambitious? · 2019-04-09T13:14:12.202Z · score: 5 (3 votes) · LW · GW

What is good literature on learned helplessness?

Comment by dr_manhattan on What I've Learned From My Parents' Arranged Marriage · 2019-03-29T17:34:30.952Z · score: 9 (3 votes) · LW · GW
The One is a myth.

Yes, 100%. But really good chemistry comes in grades, in this case I feel really really good chemistry. There are probably partners "above the acceptable threshold" nearby, and if I date date long enough I might feel the same about someone else, but in this case LDR is a temporary inconvenience, I just want to make it work in LDR format until I feel good/convinced enough to turn it into N(ear)DR, which is well in my power to do.

Comment by dr_manhattan on What I've Learned From My Parents' Arranged Marriage · 2019-03-29T17:31:56.155Z · score: 3 (2 votes) · LW · GW

Not retracted (I guess no more delete feature) just expanding longer comment elsewhere

Comment by dr_manhattan on What I've Learned From My Parents' Arranged Marriage · 2019-03-29T17:21:23.628Z · score: 2 (1 votes) · LW · GW

It's an LDR that I intend to turn into NDR. LDR temporarily. Hope that helps.

Comment by dr_manhattan on What would you need to be motivated to answer "hard" LW questions? · 2019-03-29T17:19:27.596Z · score: 6 (3 votes) · LW · GW

Sure. I know something about general CS stuff, ML, applied Bayesian stats and finance. Generally I would not be answering questions for a bounty (I'm well compensated to do this at work and I don't want *more work*) but would spend some time if I think it helped people of contributed to an important body of knowledge. For me it comes from a different "time budget". I realize many people would feel differently but there's probably a class of people like me.

Comment by dr_manhattan on What would you need to be motivated to answer "hard" LW questions? · 2019-03-28T20:20:24.628Z · score: 2 (3 votes) · LW · GW

I think the answer will be highly dependent on the question. The opportunity cost of someone answering a q-n on yoga (being a yoga expert) is very different from someone answering an investment question.

Comment by dr_manhattan on What I've Learned From My Parents' Arranged Marriage · 2019-03-26T12:50:59.904Z · score: 8 (5 votes) · LW · GW

Thanks for sharing. Good story and happens to give me encouragement as I seems to be falling into a (for now) LDR relationship with somebody in another country. It feels more "doable" based on this.

Comment by dr_manhattan on [Question] Tracking accuracy of personal forecasts · 2019-03-25T12:59:07.576Z · score: 3 (2 votes) · LW · GW

Agree! Tricky territory. I think it's fair to take an outside view as a first cut (e.g. how many people survived Everest), then very carefully evaluate if the reference class is relevant. Yudkowsky writes about this quite a bit in places cannot recall which particular place.