I may have just had a dangerous thought.

post by Fivehundred · 2014-09-22T20:04:17.171Z · LW · GW · Legacy · 42 comments

I'm interested in discussing this with someone, non-publicly. It's safe to know about personally, but it's not something I'd like people in general to know.

I'm really not sure if there is a protocol for this sort of thing.

42 comments

Comments sorted by top scores.

comment by shminux · 2014-09-22T20:50:19.621Z · LW(p) · GW(p)

10:1 it is not as dangerous as you think and 1:1 it's common enough to be on Wikipedia.

Replies from: Scott Garrabrant, Fivehundred
comment by Scott Garrabrant · 2014-09-22T21:49:32.953Z · LW(p) · GW(p)

After hearing the idea, I believe that it is not at all dangerous. However, I think the general strategy of being more cautious than you think you have to be whenever you think you have a dangerous idea is a good one. If shminux's comment made you feel any negative emotions associated with being too cautious, I would like to cancel those out by applauding your choice to err on the side of caution.

Replies from: shminux
comment by shminux · 2014-09-22T21:55:59.590Z · LW(p) · GW(p)

After hearing the idea, I believe that it is not at all dangerous.

Same here. However I advised Eitan to not make a post about it, because the potential, however small, for negative publicity ("look what these crazy LWers think up") clearly outweighs the potential benefit (vanishingly tiny). I suppose a question in the open thread would be OK. The thought is neither very original nor particularly interesting.

applauding your choice to err on the side of caution.

i absolutely agree here.

Replies from: Scott Garrabrant
comment by Scott Garrabrant · 2014-09-22T22:00:27.573Z · LW(p) · GW(p)

Out of curiosity, did you make this recommendation before or after reading jimrandomh's comment? I ask because I think that his comment would have caused me to have an "appear to be even more cautious" bias.

comment by Fivehundred · 2014-09-22T20:58:27.467Z · LW(p) · GW(p)

So should I or should I not message you?

Replies from: jimrandomh, shminux
comment by jimrandomh · 2014-09-22T21:24:44.270Z · LW(p) · GW(p)

While he might be right about it being less dangerous than you think, he did not have sufficient information to take 10:1 odds, and commenting as he did signals low caution. Unless there is some point specifically in his favor, you probably should not message him.

Replies from: shminux, Fivehundred
comment by shminux · 2014-09-22T21:32:48.747Z · LW(p) · GW(p)

A few notes:

  • I estimated the odds, I did not say I would take a bet. There is a large difference.
  • risky behavior does not need to (anti)correlate with the ability to keep someone else's secrets.

That said, I tend to agree that there are plenty of other people on this forum better qualified to evaluate if an idea is dangerous. However, the odds (here it is, again) of Eitan's idea needing high qualifications to evaluate are tiny.

Replies from: ShardPhoenix
comment by ShardPhoenix · 2014-09-22T23:52:58.378Z · LW(p) · GW(p)

I estimated the odds, I did not say I would take a bet. There is a large difference.

I don't think there should be a difference. In practice the friction of actually making the bet is generally too high to bother, but in principle if you're giving odds you should be willing to make a small bet at (slightly better than) those odds.

comment by Fivehundred · 2014-09-22T21:29:49.758Z · LW(p) · GW(p)

Already did, so fingers crossed. :)

comment by shminux · 2014-09-22T21:27:17.063Z · LW(p) · GW(p)

Feel free.

comment by Gunnar_Zarncke · 2014-09-23T04:35:15.568Z · LW(p) · GW(p)

I like the idea of asking for advice on possible dangerous ideas.

But I'd like to improve on the means: Just inviting arbitrary private communication isn't exactly save. Now OK, LW isn't really arbitrary, but telling 7 people replying on an open post... If the idea really were dangerous... Framing the idea in such a way at least causes curiousity and I had to restrain myself a bit to not immediately jump to the "Send message" button.

A solution could involve trusted rationality counselors. A proxy for this on LW could be contributors with high karma and and high positive rating.

ADDED: I think a wiki page "Rationality counseling" might be a good idea.

comment by Scott Garrabrant · 2014-09-22T21:18:43.105Z · LW(p) · GW(p)

*volunteers

Also, I am in general for moving as much stuff from open thread to discussion as possible, but this is one of the few cases that I think open thread would have been more appropriate. Not a big deal though.

Replies from: Fivehundred
comment by Fivehundred · 2014-09-22T21:25:59.918Z · LW(p) · GW(p)

I... don't know what an open thread is.

Replies from: MaximumLiberty, None, Scott Garrabrant
comment by MaximumLiberty · 2014-09-22T22:36:16.537Z · LW(p) · GW(p)

Upvoted for recognizing ignorance.

comment by [deleted] · 2014-09-22T21:37:00.325Z · LW(p) · GW(p)

This is the latest open thread.

comment by Scott Garrabrant · 2014-09-22T21:30:19.233Z · LW(p) · GW(p)

Theres a weekly thread on less wrong discussion for people to just post stuff that is short enough, off topic enough, or low quality enough to not deserve a top level post. In this case there is no information content in your post, at the ideal case would be that a very small minority of Less Wrong sees it, so it would be perfect for open thread. (It is not worth doing anything about it now, and minimal/no harm was done.)

comment by MugaSofer · 2014-09-26T15:13:55.650Z · LW(p) · GW(p)

Hey, I've listened to a lot of ideas labelled "dangerous", some of which were labeled "extremely dangerous". Haven't gone crazy yet.

I'd definitely like to discuss it with you privately, if only to compare your idea to what I already know.

comment by private_messaging · 2014-09-25T18:08:58.589Z · LW(p) · GW(p)

It seems to me that the only real dangerous 'idea' here is this persistent promotion of the notion of dangerous ideas. Either way, you can message me if you want, I don't believe in most of the common transhumanist nonsense here, nor do I think that some homebrew decision theories based on the notion that you can alter the way mathematics work, including mathematics that been already computed by the world and made known to the agent, are of such cosmic relevance.

comment by Voltairina · 2014-09-23T05:31:05.871Z · LW(p) · GW(p)

Thank you for letting us know. Don't tell me your idea:).

comment by Username · 2014-09-23T19:24:15.339Z · LW(p) · GW(p)

I'm not aware of a protocol for this sort of thing either.

Replies from: Fivehundred
comment by Fivehundred · 2014-09-23T20:22:54.617Z · LW(p) · GW(p)

Sometimes I wonder how many things RationalWiki has missed out on.

comment by [deleted] · 2014-09-23T02:03:25.470Z · LW(p) · GW(p)

I volunteer to give you honest feedback.

Generally err on the side of caution. I don't know if we have aprocedure for this though...

Replies from: Fivehundred
comment by Fivehundred · 2014-09-23T02:06:04.707Z · LW(p) · GW(p)

I don't think I'm going to tell any more people about this, aside from Eliezer. I haven't gotten any answers.

comment by Dallas · 2014-09-23T01:57:36.335Z · LW(p) · GW(p)

Can you be slightly more specific on the context? Like, at least the vague fields of study it might apply to? This would allow us to make an informed decision.

Replies from: Fivehundred
comment by Fivehundred · 2014-09-23T02:03:51.303Z · LW(p) · GW(p)

Let's see. What other ideas in Lesswrong have been considered dangerous?

comment by knb · 2014-09-23T06:11:32.039Z · LW(p) · GW(p)

Tell me your idea.

comment by Fivehundred · 2014-09-22T22:08:22.239Z · LW(p) · GW(p)

Should I be limiting this to a certain amount of people? I think like seven already know.

Replies from: jimrandomh, gjm
comment by jimrandomh · 2014-09-22T22:42:30.343Z · LW(p) · GW(p)

Two hours ago, you said it was "not something I'd like people in general to know". Since then, you told SEVEN people!? Stop it! You fail at caution! Aaaaargh!

Replies from: Fivehundred
comment by Fivehundred · 2014-09-22T22:44:36.684Z · LW(p) · GW(p)

Well, I meant that I don't want the information to be accessible for just anyone who stumbles across it. It's probably safe for most Lesswrongers.

Replies from: L29Ah
comment by L29Ah · 2014-09-23T08:42:39.605Z · LW(p) · GW(p)

PM me; I'm curious of what kind of information all this could be about.

comment by gjm · 2014-09-22T22:30:55.864Z · LW(p) · GW(p)

Surely that depends on what kind of danger you see and who the people are. Without knowing that, my generic suggestion is: wait until you have feedback from the ones you've talked to already, and then proceed as seems appropriate.

comment by polymathwannabe · 2014-09-22T21:00:23.798Z · LW(p) · GW(p)

You can PM me with the details.

comment by ChristianKl · 2014-09-22T20:51:07.100Z · LW(p) · GW(p)

Feel free to send me a personal message.

comment by Elo · 2014-09-28T06:42:29.701Z · LW(p) · GW(p)

message it to me.

comment by AABoyles · 2014-09-24T18:29:40.754Z · LW(p) · GW(p)

[content deleted]

Replies from: None
comment by [deleted] · 2014-09-25T20:15:12.737Z · LW(p) · GW(p)

There could be a FAQ in the wiki or somewhere.

Q. I've had a dangerous idea.
A. No you haven't.

Replies from: AABoyles
comment by AABoyles · 2014-09-25T21:18:08.941Z · LW(p) · GW(p)

Look at the other comments on this post. Some of us don't take the threat seriously, but some do. The idea which prompted this thread has been 1) deemed non-dangerous and 2) the OP has been advised (and resolved) not to share it anyway. Why such precaution if no one ever has dangerous ideas?

I agree that the probability is low, but the potential damage is sufficiently high that the expected damage is not inconsiderable. More immediately, it may needlessly cause anguish for people who consider it, well beyond the level of frustration imparted by, say, ordinary trolling.

comment by Algernoq · 2014-09-24T02:31:50.293Z · LW(p) · GW(p)

How can I distinguish dangerous thoughts from magical thinking? In my experience, my thoughts alone have no effect whatsoever on the physical world. In other words, actions have consequences but thoughts not acted upon are undetectable by others.

(Exception: brain scanning, where other people are observing what's happening inside my brain, and even then it's a really coarse scan.)

(Non-exception: observing quantum-random events like Geiger counter beeps. My thoughts have no discernible effect on these.)

If your theory is right, that (entity/force AB) is able to affect my physical world here, then I would expect other opportunities/consequences. For example, there would be ways to persuade (entity/force AB) to provide beneficial, immediately observable consequences. Any ideas for positive, non-dangerous ways to test your theory?

If I misread the tone of your question, and you're writing about something that's dangerous to you personally for conventional reasons, then best of luck with your exciting life-style.

Replies from: Fivehundred
comment by Fivehundred · 2014-09-24T02:49:05.136Z · LW(p) · GW(p)

It's not an entity I'm worried about. Your concerns aren't applicable.

comment by buybuydandavis · 2014-09-23T10:16:54.394Z · LW(p) · GW(p)

Have you googled to see if your idea is original?

There is very little new under the sun.

But I understand your reticence. I'm often struck by how unserious terrorists are. Ooooh, go booom! Really? That's the best they can come up with? I think death and destruction is pretty easy. I'm curious if I'm wrong (and just curious in general), but I don't want to be giving anyone else any ideas, even if the ideas are already out there.

I'd be interested in hearing your dangerous idea.

Replies from: ChristianKl
comment by ChristianKl · 2014-09-23T11:04:35.163Z · LW(p) · GW(p)

Without going into the details, as far as I understand it's supposed to be a clever way of pascal's mugging people. I don't consider it dangerous but I can see how Eitan Zohar does. I think there's no need to have a discussion about this issue in public.

I personally don't consider myself deeply knowledgeable about pascal's mugging related issues. In case anybody does and wants to debate the issue, that might be a reason to contact Eitan Zohar over it.

comment by blacktrance · 2014-09-22T22:04:17.483Z · LW(p) · GW(p)

Feel free to message me.