Posts
Comments
I found it really helpful to have a list of places where Eliezer and Paul agree. It's interesting to see that there is a lot of similarity on big picture stuff like AI being extremely dangerous.
A number of people seem to have departed OpenAI at around the same time as you. Is there a particular reason for that which you can share? Do you still think that people interested in alignment research should apply to work at OpenAI?
In the case of reducing mutational load to near zero, you might be doing targeted changes to huge numbers of genes. There is presumably some point at which it's easier to create a genome from scratch.
I agree it's an open question though!
An alternative to editing many genes individually is to synthesise the whole genome from scratch, which is plausibly cheaper and more accurate.
I would find this more useful if you spelled out a bit more about your scoring method. You say:
They must be loyal, intelligent, and hardworking, they must have a sense of dignity, they must like humans, and above all they must be healthy.
Which of these do you think are the most important? Why do these traits matter? (for example, hardworking dogs are not really necessary in the modern world)
And why these traits and not others? (for example: size, cleanliness, appearance, getting along with other animals)
a dog which is as close to being a wolf as one can get without sacrificing any of those essential characteristics which define a dog as such
Why do you think a dog that is close to a wolf is objectively better than dogs which are further away?
Vanguard has a UK website, I use them and it works well.
Monevator also has a good guide to investment firms in the UK, along with a bunch of UK specific advice.
OpenPhil gave Carl Shulman $5m to re-grant
I didn't realise this was happening. Is there somewhere we can read about grants from this fund when/if they occur?
Would this approach have any advantages vs brain uploading? I would assume brain uploading to be much easier than running a realistic evolution simulation, and we would have to worry less about alignment.
I filled in the survey! Like many people I didn't have a ruler to use for the digit ratio question.
Also, I'm torn between how to interpret Snape's last question - my first thought was that he was verifying the truth of a story he had been told("Your master tortured her, now join the light side already!" being the most likely), but upon rereading, I wonder if he was worried that she had been used as Horcrux fuel.
Or verifying a deal he made with Voldemort, though that might not make as much sense with Snape's character.
Slightly off topic, but I'm very interested in the "policy impact" that FHI has had - I had heard nothing about it before and assumed that it wasn't having very much. Do you have more information on that? If it were significant, it would increase the odds that giving to FHI was a great option.
Possible consideration: meta-charities like GWWC and 80k cause donations to causes that one might not think are particularly important. E.g. I think x-risk research is the highest value intervention, but most of the money moved by GWWC and 80k goes to global poverty or animal welfare interventions. So if the proportion of money moved to causes I cared about was small enough, or the meta-charity didn't multiply my money much anyway, then I should give directly (or start a new meta-charity in the area I care about).
A bigger possible problem would be if I took considerations like the poor meat eater problem to be true. In that case, donating to e.g. 80k would cause a lot of harm even though it would move a lot of money to animal welfare charities, because it causes so much to go to poverty relief, which I could think was a bad thing. It seems like there are probably a few other situations like this around.
Do you have figures on what the return to donation (or volunteer time) is for 80,000 hours? i.e. is it similar to GWWC's $138 of donations per $1 of time invested? It would be helpful to know so I could calculate how much I would expect to go to the various causes.
Something on singletons: desirability, plausibility, paths to various kinds (strongly relates to stable attractors)
"Hell Futures - When is it better to be extinct?" (not entirely serious)
Recommendations are up!
Maybe some kinds of ems could tell us how likely Oracle/AI-in-a-box scenarios were to be successful? We could see if ems of very intelligent people run at very high speeds could convince a dedicated gatekeeper to let them out of the box. It would at least give us some mild evidence for or against AIs-in-boxes being feasible.
And maybe we could use certain ems as gatekeepers - the AI wouldn't have a speed advantage anymore, and we could try to make alterations to the em to make it less likely to let the AI out.
Minor bad incidents involving ems might make people more cautious about full-blown AGI (unlikely, but I might as well mention it).
I was the one who asked that question!
I was slightly disappointed by his answer - surely there can only be one optimal charity to give to? The only donation strategy he recommended was giving to whichever one was about to go under.
I guess what I'm really thinking is that it's pretty unlikely that the two charities are equally optimal.
Point taken. This post seems unlikely to reach those people. Is it possible to communicate the importance of x-risks in such a short space to SL0's - maybe without mentioning exotic technologies? And would they change their charitable behavior?
I suspect the first answer is yes and the second is no (not without lots of other bits of explanation).
I thought this article was for SL0 people - that would give it the widest audience possible, which I thought was the point?
If it's aimed at the SL0's, then we'd be wanting to go for an SL1 image.
Whilst I really, really like the last picture - it seems a little odd to include it in the article.
Isn't this meant to seem like a hard-nosed introduction to non-transhumanist/sci-fi people? And doesn't the picture sort of act against that - by being slightly sci-fi and weird?
Due to the absence of any signs of intelligence out there, especially paper-clippers burning the cosmic commons, we might conclude that unfriendly AI could not be the most dangerous existential risk that we should worry about.
I view this as one of the single best arguments against risks from paperclippers. I'm a little concerned that it hasn't been dealt with properly by SIAI folks - aside from a few comments by Carl Shulman on Katja's blog.
I suspect the answer may be something to do with anthropics - but I'm not really certain of exactly what it is.
What initiatives is the Singularity Institute taking or planning to take to increase it's funding to whatever the optimal level of funding is?
I'm guessing they mean a university affiliated person doing a formal philosophy degree of some kind.
I didn't think of that - given that a huge chuck here have probably taken such tests, if Yvain allowed such an estimation, it would be very helpful.
excluded-middle bias
Yes! That's what I was thinking of :)
This is great! I hope there's a big response.
It seems likely you're going to get skewed answers for the IQ question. Mostly it's the really intelligent and the below average who get (professional) IQ tests - average people seem less likely to get them.
I predict high average IQ, but low response rate on the IQ question, which will give bad results. Can you tell us how many people respond to that question this time? (no. of responses isn't registered on the previous survey)
The obvious solution is to stop eating all those kinds of animal/animal products. That would satisfy CO2 concerns and killing concerns.
Of course, it might not satisfy things like fun of eating meat, ease of eating meat, health etc.
Here maybe?
I take it that you partially changed "my mistakes" to include nicotine. I enjoyed your article on it - but how are you using it?
Are you rotating with other stimulants on a regular basis, using when you like, using to promote habit formation etc.
Would anyone care to comment on the recent Mt Gox hack n' crash?
Personally, I'm thinking that this very bad. The currency won't look as good the mainstream, and I'm anticipating panic sells as soon as the exchanges get up and running again. I'm agnostic as to whether Bitcoin will die or not though...
The obvious extra question is:
"If you think it's so great, how come you're not using it?" Unless the sales girl's enjoyable life includes selling the machine she's in to disinterested customers.
And if you do assume "fiat money is doomed, doomed!" then why wouldn't something like bitcoin become the world's reserve currency?
Okay, I'm willing to grant that if the dollar/fiat money in general is doomed then something along the lines of bitcoin would probably take over. But I don't assume this. I guess it is rational to put lots of money into bitcoin if you do take this premise though.
I agree that the dollar becoming effectively worthless would be pretty bad to put it mildly!
Weirdly, though I think that bitcoins will succeed (and accordingly have some) I don't think Calacanis' article is well-founded. To focus just on the points I feel I can judge with some merit:
Bitcoin is unstoppable without end-user prosecution.
I don't think this is true. Shutting down of all legitimate currency exchanges would tend to increase the barrier to investment by legitimate investors and be likely to decrease interest in Bitcoins. Anecdote: I would get less interested in bitcoins if this happened. Also, a focused government campaign against it might succeed in branding it as evil in public eyes, again reducing interest. I do recognize that these factors wouldn't destroy bitcoin, but they would reduce the chances of it "changing the world".
Bitcoins will change the world unless governments ban them with harsh penalties.
Again, I'm not so sure of this. Can anyone give me any legitimate reasons to anticipate this? So far I've just people on the bitcoin forums saying that the dollar/euro will collapse and bitcoin will attain worldwide dominance.
Basically, I predict they'll find a decent niche, won't replace any currency system in any major country and won't increase in value to insane levels.
Hi Less Wrong!
Decided to register after seeing this comment and wanting to post give a free $10 to a cause I value highly.
I got pulled into less wrong by being interested in transhumanist stuff for a few years, finally decided to read here after realizing that this was the best place to discuss this sort of stuff and actually end up being right as opposed to just making wild predictions with absolutely no merit. I'm an 18 year old male living in the UK. I don't have a background in maths or computer sci as a lot of people here do (though I'm thinking of learning them). I'm just finishing up at school and then going on to do a philosophy degree (hopefully - though I'm scared of it making me believe crap things)
I've found the most useful LW stuff to be along the lines of instrumental rationality (the more recent stuff). Lukeprog's sequence on winning at life is great! My favorite LW-related posts have been:
The Cynic's Conundrum: Because I used to think idealistically about my own thought processes and cynically about other people's. In essence I fell into comfortable cynicism.
Tsuyoku Naritai! (I Want To Become Stronger): Because this was just really galvanizing and made me want to do better, much more than any self-help stuff ever did!
A Suite of Pragmatic Considerations in Favor of Niceness: Fantastic as I tended (and still tend to) be mean for no real reason and this post put a lot of motivation towards stopping. I've actually started to have niceness as a terminal value now, which is a tad odd.
So anyway, I'm happy to have registered and I hope to get stronger and have fun here!
Your right action is most excellent!