Posts

$100 for the best article on efficient charty - the winner is ... 2010-12-12T15:02:06.007Z · score: 19 (20 votes)
$100 for the best article on efficient charity: the finalists 2010-12-07T21:15:31.102Z · score: 5 (6 votes)
$100 for the best article on efficient charity -- Submit your articles 2010-12-02T20:57:31.410Z · score: 5 (6 votes)
Superintelligent AI mentioned as a possible risk by Bill Gates 2010-11-28T11:51:50.475Z · score: 7 (8 votes)
$100 for the best article on efficient charity -- deadline Wednesday 1st December 2010-11-24T22:31:57.215Z · score: 16 (15 votes)
Competition to write the best stand-alone article on efficient charity 2010-11-21T16:57:35.003Z · score: 15 (22 votes)

Comments

Comment by formallyknownasroko on Should LW have a public censorship policy? · 2010-12-12T20:44:26.040Z · score: 6 (22 votes) · LW · GW

Brennan is a fucking retard. No, you don't want to know. You want to signal affiliation with desirable groups, to send hard-to-fake signals of desirable presonality traits such as loyalty, intelligence, power and the presence of informed allies. You want to say everything bad you possibly can about the outgroup and everything good about the ingroup. You want to preech altruism and then make a plausible but unlikely reasoning error which conveniently stops you from having to give away anything costly.

All the other humans do all of these things. This is the true way of our kind. You will be punished if you deviate from the way, or even if you try to overtly mention that this is the way.

Comment by formallyknownasroko on $100 for the best article on efficient charty - the winner is ... · 2010-12-12T19:10:11.175Z · score: 1 (3 votes) · LW · GW

Yes but in reality the amounts concerned are good value for what they get.

Comment by formallyknownasroko on $100 for the best article on efficient charty - the winner is ... · 2010-12-12T17:59:33.872Z · score: 0 (0 votes) · LW · GW

Yes, I will message you with details

Comment by formallyknownasroko on Unpacking the Concept of "Blackmail" · 2010-12-12T17:51:58.286Z · score: 1 (1 votes) · LW · GW

The problem with "lose-lose" is that it relies upon there being a "defualt outcome given no interaction". Vladimir is trying to taboo this concept, at least in general. So I am going to focus on a relevant special case, namely specific interactions available in the ancestral environment.

Comment by formallyknownasroko on How To Lose 100 Karma In 6 Hours -- What Just Happened · 2010-12-12T12:51:34.092Z · score: 0 (4 votes) · LW · GW

the only information that I know and wish I did not know.

I don't think it's quite that extreme. For example, I wish I wasn't as intelligent as I am, wish I was more normal mentally and had more innate ability at socializing and less at math, wish I didn't suffer from smart sincere syndrome. I think these are all in roughly the same league as the banned material.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-12T12:19:33.887Z · score: -1 (3 votes) · LW · GW

This all reminds me of the dirac delta function. Its width is infinitesimal but its area is 1. Sure, it's worth trying in the "Dirac Delta Function" sense.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-12T12:06:59.485Z · score: -1 (1 votes) · LW · GW

Ok, I see. Well, that's just a big factual disagreement then.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-12T11:56:10.598Z · score: 0 (0 votes) · LW · GW

What risks there are, we can collectively do things about.

Not necessarily. The risk might be virtually unstoppable, like a huge oil tanker compared to the force of a single person swimming in the water trying to slow it down.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-12T11:51:25.792Z · score: 0 (0 votes) · LW · GW

Agreed that there are vested interests potentially biasing reasoning.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-12T00:58:20.545Z · score: 2 (2 votes) · LW · GW

I meant if you divide the number of galaxies by the number of seconds to an event 100 years from now. Yes, not all reachable. Probably need to discount by an order of magnitude for reachability at lightspeed.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-12T00:16:29.694Z · score: 4 (6 votes) · LW · GW

Agreed that x-risk orgs are a biased source of info on P(risk) due to self-selection bias. Of course you have to look at other sources of info, you have to take the outside view on these questions, etc.

Personally I think that we are so ignorant and irrational as a species (humanity) and as a culture that there's simply no way to get a good, stable probability estimate for big important questions like this, much less to act rationally on the info.

But I think your pooh-pooh'ing such infantile and amateurish efforts as there is silly when the reasoning is entirely bogus.

Why don't you refocus your criticism on the more legitimate weakness of existential risks: that is highly likely to be irrelevant (either futile or unnecessary), since by its own prediction, the relevant risks are highly complex and hard to mitigate against, and people in general are highly unlikely to either understand the issues or cooperate on them.

The most likely route to survival would seem to be that the entire model of the future propounded here is wrong. But in that case we move into the domain of irrelevance.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T20:55:24.567Z · score: 4 (4 votes) · LW · GW

The sort of thing that could give you nightmares is more like the stuff that is banned. This is different than the mere "existential risk" message.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T16:47:21.028Z · score: 4 (4 votes) · LW · GW

A moment's googling finds this:

http://www.buglife.org.uk/Resources/Buglife/Buglife%20Annual%20Report%20-%20web.pdf

"Total Income £546,415"

($863 444)

I leave it to readers to judge whether Tim is flogging a dead horse here.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T16:41:30.139Z · score: 8 (12 votes) · LW · GW

To me, $200,000 for a charity seems to be pretty much the smallest possible amount of money. Can you find any charitable causes that recieve less than this?

Basically, you are saying that SIAI DOOM fearmongering is a trick to make money. But really, it fails to satisfy several important criteria:

  • it is shit at actually making money. I bet you that there are "save the earthworm" charities that make more money.

  • it is not actually frightening. I am not frightened; quick painless death in 50 years? boo-hoo. Whatever.

  • it is not optimized for believability. In fact it is almost optimized for anti-believability, "rapture of the nerds", much public ridicule, etc.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T14:26:52.042Z · score: 0 (0 votes) · LW · GW

Many plausible ways to S^+ involve something odd or unexpected happening. WBE might make computational political structures, i.e. political structures based inside a computer full of WBEs. This might change the way humans cooperate.

Suffices to say that FAI doesn't have to come via the expected route of someone inventing AGI and then waiting until they invent "friendliness theory" for it.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T14:12:16.396Z · score: 0 (0 votes) · LW · GW

(t(positive singularity) | positive singularity)

I'm going to say 75 years for that. But really, this is becoming very much total guesswork.

I do know that AGI -ve singularity won't happen in the next 2 decades and I think one can bet that it won't happen after that for another few decades either.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T12:55:06.811Z · score: 1 (3 votes) · LW · GW

I think you're trying to fit the facts to the hypothesis. Negatve singularity in my opinion is at least 50 years away. Many people I know will already be dead by then, including me if I die at the same point in life as the average of my family.

And as a matter of fact it is failing to actually get much in the way of donations, compared to donations to the church which is using hell as a superstimulus, or even compared to campaigns to help puppies (about $10bn in total as far as I can see).

It is also not well-optimized to be believable.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-11T12:37:57.899Z · score: 5 (5 votes) · LW · GW

Point of fact: the negative singularity isn't a superstimulus for evolved fear circuits: current best-guess would be that it would be a quick painless death in the distant future (30 years+ by most estimates, my guess 50 years+ if ever). It doesn't at all look like how I would design a superstimulus for fear.

Comment by formallyknownasroko on Unpacking the Concept of "Blackmail" · 2010-12-10T23:55:46.145Z · score: 3 (3 votes) · LW · GW

Suppose that Blackmail is

merely an affective category, a class of situations activating a certain psychological adaptation

-- then we should ask what features of the ancestral environment caused us to evolve it. We might understand it better in that case.

I suspect that the ancestral environment came with a very strong notion of a default outcome for a given human, in the absence of there being any particular negotiation, and also came with a clear notion of negative interaction (stabbing, hitting, kicking) versus positive interaction (giving fish, teaching how to hunt better, etc).

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T23:35:25.723Z · score: -3 (5 votes) · LW · GW

We're all stuck on the train anyway, so saving it is worth a shot.

I disagree with this argument. Pretty strongly. No selfish incentive to speak of.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T22:34:52.623Z · score: 0 (2 votes) · LW · GW

No, dude, you're wrong

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T22:28:14.115Z · score: 7 (11 votes) · LW · GW

No, this is the "collective-action-problem" - where the end of the world arrives - despite a select band of decidedly amateurish messiahs arriving and failing to accomplish anything significant.

You are looking at those amateurs now.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T22:25:41.024Z · score: 4 (4 votes) · LW · GW

I mean I guess I shouldn't complain that you don't find this bothers you, because you are, in fact, helping me by doing what you do and being very good at it, but that doesn't stop it being demotivating for me! I'll see what I can do regarding quant jobs.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T20:18:52.055Z · score: 1 (3 votes) · LW · GW

Maybe it will all be OK. Maybe the trains fly past each other on separate tracks. We don't know. There sure as hell isn't a driver though. All the inside-view evidence points to bad things,with the exception that Big Worlds could turn out nicely. Or horribly.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T20:15:33.267Z · score: 4 (6 votes) · LW · GW

The compelling argument for me is that knowing about bad things is useful to the extent that you can do something about them, and it turns out that people who don't know anything (call them "non-cogniscenti") will probably free-ride their way to any benefits of action on the collective-action-problem that is the at issue here, whilst avoiding drawing any particular attention to themselves ==> avoiding the risks.

Vladimir Nesov doubts this prima facie, i.e. he asks "how do you know that the strategy of being a completely inert player is best?".

-- to which I answer, "if you want to be the first monkey shot into space, then good luck" ;D

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T20:05:01.206Z · score: 2 (2 votes) · LW · GW

I had a private email conversation with Eliezer that did involve a process of logical discourse, and another with Carl.

Also, when I posted the material, I hadn't thought it through. One I had thought it through, I realized that I had accidentally said more than I should have done.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T19:32:49.854Z · score: 4 (4 votes) · LW · GW

Specifics. Details. The lesson of science is that details can sometimes change the overall conclusion. Also some amount of nerdyness meaning that the statements about human nature weren't obvious to me.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T19:23:36.007Z · score: 10 (16 votes) · LW · GW

Well I guess this is our true point of disagreement. I went to the effort of finding out a lot, went to SIAI and Oxford to learn even more, and in the end I am left seriously disappointed by all this knowedge. In the end it all boils down to:

"most people are irrational, hypocritical and selfish, if you try and tell them they shoot the messenger, and if you try and do anything you bear all the costs, internalize only tiny fractions of the value created if you succeed, and you almost certainly fail to have an effect anyway. And by the way the future is an impending train wreck"

I feel quite strongly that this knowledge is not a worthy thing to have sunk 5 years of my life into getting. I don't know, XiXiDu, you might prize such knowledge, including all the specifics of how that works out exactly.

If you really strongly value the specifics of this, then yes you probably would on net benefit from the censored knowledge, the knowledge that was never censored because I never posted it, and the knowledge that I never posted because I was never trusted with it anyway. But you still probably won't get it, because those who hold it correctly infer that the expected value of releasing it is strongly negative from an altruist's perspective.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T19:00:19.170Z · score: 1 (1 votes) · LW · GW

Well look, one has to do cost/benefit calculations, not just blindly surge forward in some kind of post-enlightenment fervor. To me, it seems like there is only one positive term in the equation:: the altrustic value of giving money to some existential risk charity.

All the other terms are negative, at least for me. And unless I actually overcome excuses, akrasia, etc to donate a lot, I think it'll all have been a mutually detrimental waste of time.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T18:32:22.697Z · score: 0 (2 votes) · LW · GW

Probably less than 1% of cave-men died by actively seeking out the sabre-toothed tiger to see if it was friendly. But I digress.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T18:29:00.224Z · score: 0 (2 votes) · LW · GW

Yes, we are, in fact, safer than in the EEA, in contemporary USA.

But still, there are some real places where danger is real, like the Bronx or scientology or organized crime or a walking across a freeway. So, don't go rubbishing the heuristic of being frightened of potentially real danger.

I think it would only be legitimate to criticize fear itself on "outside view" grounds if we lived in a world with very little actual danger, which is not at all the case.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T18:17:43.223Z · score: 2 (2 votes) · LW · GW

What's wrong with using "this bias paid off in the evolutionary environment!" as an argument? I think people who paid more attention to this might make fewer mistakes, especially in domains where there isn't a systematic, exploitable difference between EEA and now.

The evolutionary environment contained enetities capable of dishing out severe punishments, unertainty, etc.

If anything, I think that the heuristic that an idea "obviously" can't be dangerous is the problem, not the heuristic that one should take care around possibilities of strong penalites.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T18:14:51.677Z · score: 1 (3 votes) · LW · GW

Oh, I see. Well, I guess it depends upon how much I eventually donate and how much of an incremental difference that makes.

It would certainly be better to just donate, AND to also not know anything about anything dangerous. I'm not even sure that's possible, though. For all we know, just knowing about any of this is enough to land you in a lot of trouble either in the causal future or elsewhere.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T18:04:07.523Z · score: 5 (5 votes) · LW · GW

For me truth is the most cherished good.

More so than not opening yourself up to a small risk of severe consequences? E.g. if you found a diary that clearly belonged to some organized crime boss, would you open it up and read it? I see this situation as analogous.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T18:01:07.754Z · score: -1 (5 votes) · LW · GW

Sure I want to donate. But if you express it as a hypothetical choice between being a person who didn't know about any of this and had no way of finding out, versus what I have now, I choose the former. Though since that is not an available choice, it is a somewhat academic question.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T17:58:02.010Z · score: 0 (2 votes) · LW · GW

being terrified of very unlikely terrible events is a known human failure mode

one wonders how something like that might have evolved, doesn't one? What happened to all the humans who came with the mutation that made them want to find out whether the sabre-toothed tiger was friendly?

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T17:34:09.246Z · score: 0 (6 votes) · LW · GW

Whatever man, go ahead and make your excuses, you have been warned.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T17:06:28.736Z · score: 5 (19 votes) · LW · GW

Look, you have three people all of whom think it is a bad idea to spread this. All are smart. Two initially thought it was OK to spread it.

Furthermore, I would add that I wish I had never learned about any of these ideas. In fact, I wish I had never come across the initial link on the internet that caused me to think about transhumanism and thereby about the singularity; I wish very strongly that my mind had never come across the tools to inflict such large amounts of potential self-harm with such small durations of inattention, uncautiousness and/or stupidity, even if it is all premultiplied by a small probability. (not a very small one, mind you. More like 1/500 type numbers here)

If this is not enough warning to make you stop wanting to know more, then you deserve what you get.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-10T16:58:32.359Z · score: 0 (0 votes) · LW · GW

Upvoted, agree strongly.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T22:47:42.385Z · score: 2 (2 votes) · LW · GW

But you think that it is not a good thing for this to propagate more?

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T22:37:59.997Z · score: 5 (15 votes) · LW · GW

There is still a moral sense in which if, after careful thought, I decided that that material should not have been posted, then any posts which resulted solely from my post are in a sense a violation of my desire to not have posted it. Especially if said posts operate under the illusion that my original post was censored rather than retracted.

But in reality such ideas tend to propagate like the imp of the perverse: a gnawing desire to know what the "censored" material is, even if everyone who knows what it is has subsequently decided that they wished they didn't! E.g both me and Nesov have been persuaded (once fully filled in) that this is really nasty stuff and shouldn't be let out. (correct me if I am wrong).

This "imp of the perverse" property is actually part of the reason why the original post is harmful. In a sense, this is an idea-virus which makes people who don't yet have it want to have it, but as soon as they have been exposed to it, they (belatedly) realize they really didn't want to know about it or spread it.

Sigh.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T22:26:55.953Z · score: 4 (4 votes) · LW · GW

Right. Bottle. Genie.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T22:23:37.473Z · score: 0 (6 votes) · LW · GW

Yes. THIS IS NOT CENSORSHIP. Just in case anyone missed it.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T22:18:01.267Z · score: 2 (8 votes) · LW · GW

And I would have taken it down. My bad for not asking first most importantly.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T22:17:05.090Z · score: 11 (15 votes) · LW · GW

May I at this point point out that I agree that the post in question should not appear in public. Therefore, it is a question of the author's right to retract material, not of censorship.

Comment by formallyknownasroko on Open Thread: July 2010, Part 2 · 2010-12-09T21:41:29.273Z · score: 15 (15 votes) · LW · GW

Correct. I was not asked to leave.

Comment by formallyknownasroko on Open Thread: July 2010, Part 2 · 2010-12-09T21:40:52.899Z · score: 25 (25 votes) · LW · GW

Can I just state categorically that this "ways to increase existential risks" thing is stupid and completely over the top.

We should be able to discuss things, sometimes even continuing discussion in private. We should not stoop to playing silly games of brinksmanship.

Geez, if we can't eve avoid existential brinksmanship on the goddamn LW forum when the technology is hypothetical and the stakes are low, what hope in hell do we have when real politicians get wind of real progress in AGI?

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T21:32:38.847Z · score: 18 (20 votes) · LW · GW

Dude, don't be an idiot. Really.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T20:44:18.452Z · score: 0 (2 votes) · LW · GW

The Andromeda Galaxy is 2.5 million light-years away. The universe is about 1250 million years old. Therefore that's not far enough away to protect us from colonizing aliens travelling at 0.5c or above.

Comment by formallyknownasroko on Best career models for doing research? · 2010-12-09T20:18:16.549Z · score: 1 (1 votes) · LW · GW

Basically Rare Earth seems to me to be the only tenable solution to Fermi's paradox.