Posts
Comments
Clearly. But whatever they are, they should still be represented, and there may be similarly wild things in the outlying areas of their worldview - and if I can help them reach it I will. In my experience it's the ideas in the hinterlands that are the way out of the problems of our time, regardless what our time is - merely seeing that our worldview has outliers, and the overton window is limiting us in one area is usually enough to see that we're limited elsewhere, too.
Pretty sure this is the question underlying https://www.overcomingbias.com/2007/01/disagree_with_s.html
Similarly, an even more defensible position might be Buddhist one, or that happiness is transitory and mostly a construction of the mind, and virtually always attached to suffering, but suffering is real and worth minimizing.
This post is generalizable, even if you don't think that it's wrong to kill people as a general rule there's probably some other moral act #G_30429 that you probably don't think that it would be appropriate and the point still holds: Rowhammering the bit that says "Don't do #G_30429" is probably not as impossible as it seems in the long run.
(Meta: when thinking about this I found it difficult to recall all of the arguments I've learned in moral philosophy over the past 16 years of trying that would have been applicable. I knew where you were going, roughly, but it was like traveling through a city I haven't been to in years in terms of whether or not I recognized the territory. This gave me an extra impression of 'this bit could be easily flipped')
This assumes that the people around you generally do the right thing. If you operate under the alternative assumption (which is much more reasonable) you would likely still be alive.
Modernized version as of 2017, of the first part of this post : http://82.221.128.217/trolley-lw.png
More serious reply: depending when you encountered me, I'd be more boring in some ways, since a lot of what I spend my time doing is towards a moral end. All the things I've learned in life I learned from trying to live in a moral universe. I would never have gotten a degree, I did that virtually entirely for what I perceived to be reasons of altruism. Since I'm assuming here that everyone else will continue to live under the illusion that they are in such a universe, and that only I leave it...even it were merely 2008 when I encountered this revelation, I would have not donated so much to charity, I would have not gone into teaching children science...my whole of my thereafter short life would have been hedonism, torture, probably serial rape/murder and hard drugs. I wouldn't have lived with decent, hardworking people -- I'd probably have been kidnapped by gangsters or something and OD'd on heroin by now. I sure wouldn't care about the state of my country, my family, or mathematics or anything like that.
This brings up the Sapir Worf hypothesis, or the newspeak for it, "Linguistic Relativity". After all, memes must be expressible, musn't they? If they are then if it were true, then the memes that you have bound the memes that you can espouse -- linguistic relativity in a nutshell.
Many memes these days come in picture form, but for that you need a medium capable of showing pictures, and the culture that places value in making such media universally available. Without that culture, and without the apparatus to share picture-memes those memes would quickly die out, though some abstract notion of some of them, perhaps carried along the linguistic pathway in the same way that even though we don't use floppy disks for anything everyone uses them to 'save'. So in a sense not only is it the media and its memes that has to be prior to memes expressed via it, but also the language memes have to be there in order for them to be used. Language might as well just be thought as the structure and set of of meme universals.
Looks like there's been activity on Wikipedia since I've dug up this issue last suggesting that at least since the 1980's there's been recent research on how language, and memes influence thought/future use of memes/language. Reddit in particular has some really good data on this that they last I heard were not sharing with the world.
The big question is if memes are different, which evidence suggests, why is this so?
Here's what you actually wanted to link to for "looking back"
(edit: search 'looking back', i used to have it indexed with hyperlinks, but I lost that copy)My concern isn't with the interview per se(everything I would add would best be put in another thread). It's with the reaction here in the comments here.
That 90% wasn't a waste anymore than overcomingbias as a blog is a waste. Horgan is hardly alone in remembering the Fifth Generation Project and it was worth it to get Yudkowsky to hammer out, once more, to a new audience why what happened in the 80's was not representative of what is to come in the 10ky timeframe. Those of you who are hard on Horgan he is not one of you. You cannot hold him to LW standards. Yudkowsky has spent a lot of time and effort trying to get other people to not make mistakes, for example mislabeling broad singulitarian thought on him as if he's kurzweil, vinge, the entirety of MIRI and whatnot personified and so it's understandable why he might be annoyed, but at the same time...the average person is not going to bother with the finer details. He probably put in about as much or more journalistic work as the average topic requires. This just goes to really drive home how different intelligence is from other fields, how hard science journalism in a world with AI research can be.
It's frustrating because it's hard. It's hard for many reasons, but one reason is because the layman's priors are very wrong. This it shares in common(for good reason) with economics and psychology more generally that people who are not in the field bring to the table a lot of preconceptions that have to be dismantled. Dismantling them all is a lot of work for a 1 hour podcast. Like those who answer Yahoo Answers! questions, Horgan is a critical point needed to convince on his own terms between Yudkowsky & a substantial chunk of a billion+ people who lived in the 80's who are not following where Science is being taken here.
The parent made 3 claims(the 3rd one was snuck into the conclusion). I only addressed 2 and 3. 1 is a credible point that stands on its own merit. Without points 2 and 3 however with 1 it's no longer a sound argument.
edit PaleMoon lost original reply. I will try to recreate it :(
Not saying you're incorrect in criticizing the above(the two claims do seem incompatible), but isn't it the case that algorithms are just structures and that only they take time only to run? What I mean is that within the block-universe view there would be structures that we would be in ignorance of their nature and in order for us to learn about them we might have to count them (and since we are living in a timeline with computers that operate per cycle our accounting of them would take some time to complete), but that it's only ignorance to an observer like us that necessitates this? That if you knew some property of some local region of the block-universe you could use it to estimate some other property via the algorithm that represents their (mutual) structure, but that the algorithm describing their structure merely is. There's plenty of times when choosing algorithms to describe mathematical objects that we choose algorithms that fall along a space-time tradeoff, so it stands to reason that there should be a 'all-space' choice that only encodes the answer we seek in structure alone.
Our children will look back at the fact that we were STILL ARGUING about this in the early 21st-century, and correctly deduce that we were nuts.
We're still arguing whether or not the world is flat, whether the zodiac should be used to predict near-term fate and whether we should be building stockpiles of nuclear weapons. There's billions left to connect to the internet, and most extant human languages to this day have no written form. Basic literacy and mathematics is still something much of the world struggles with. This is going to go on for awhile: the future will not be surprised that the finer details of after the 20th decimal point were being debated when we can't even agree on whether intelligent design is the best approach to cell biology or not.
(this is the second copy of this comment, the first was regrettably lost in a browser crash. Use systems that back up your comments automatically)
This advice seems to fly in the face of Richard Hamming's advice to keep an open door. However perhaps the difference is subtle: Hamming suggested to have an open door but not necessarily to share your secrets, so perhaps there is room for a big science mystery cult to retain its own mysteries at every level of initiation. Perhaps there is a middle ground[1] to be found between this and current 'open science' wherein secrets and ritual are more emphasized, but where the public has the ability to always query deep into the bureaucracy of the science temple/university.
More likely, however the best approach is all of the above, some kinds of thinking are enhanced by a certain size of a team, and there may be some problems that require an open-science sized 'ingroup', and some problems that are more tractable with an ingroup the size of a mystery cult.
- 1 (keep in mind the middle ground is not necessarily the best path -- it's just a path)
The question may have once been which poet gets quoted when rainbows are brought up. If Keats isn't adding to the discussion in a meaningful way anymore since his metaphors will play second fiddle to the ones that of Newton, which were wonderful and exciting enough that Newton was driven to poking himself in the eye with a needle over them. I don't know if Keats even in his heyday could have claimed that. It may have been that his views on rainbows were propagated in some ingroup, until someone from that ingroup quoted them to someone in an ingroup with exposure to Newton's ideas on the same. They would have looked bad when that happened, but they would likely bring up the same thing to a person who might quote Keats to them, and so on until Keats himself was bested at his own game.
The problem isn't that Science is taking away from Rainbows, the problem is that Science is taking the power of controlling perception and justifying belief (mostly in other people) from Keats. No kidding he's going to be unhappy about it.
Science changes the poetry dynamic Keats' is used to because suddenly there's competition for what gets associated with what idea in such a way that poets don't necessarily get first dibs in the minds of people that they care about. Similar to how Galileo got in trouble for changing the scope of mathematicians from strictly below philosophers, this may be another instance of Newton changing how we view things by raising the social position of those who participate in science to where it is acceptable to challenge the status of a poet. Poets were important enough in Keats' day that the heads of governments had their own poet on staff.
Keats just could not keep up with what was actually still wonderful to the people he would have seduced with his ideas: Darwin came later, and found wonder still left:
"There is grandeur in this view of life, with its several powers, having been originally breathed into a few forms or into one; and that, whilst this planet has gone cycling on according to the fixed law of gravity, from so simple a beginning endless forms most beautiful and most wonderful have been, and are being, evolved. " - Charles Darwin
Of course this dynamic may be changing yet. This framing of the problem leaves open the possibility that our personal ability to perceive wonder can get very broken when our computer systems produce the models for us, as described by radiolab (tl; dr when you have computer systems that can derive laws describing phenomena better than we can understand the reason behind those laws, but which nevertheless describe those systems that generate the phenomena, we may be at something of a loss when it comes to our 'right' to perceive wonder). Being unable to physically train your brain to assign wonder to wonderful thing seems to be a different problem than this one, more of a disability rather than anything.
This seems to me more evidence that intelligence is in part a social/familial thing: that like human beings that have to be embedded in a society in order to develop a certain level of intelligence, a certain level of an intuition for "don't do this it will kill you" informed by the nuance that is only possible with a wide array of individual failures informing group success or otherwise: it might be a prerequisite for higher level reasoning beyond a certain level (and might constrain the ultimate levels upon which intelligence can rest).
I've seen more than enough children try to do things that would be similar enough to dropping an anvil on their head to consider this 'no worse than human' (in fact our hackerspace even has an anvil, and one kid has ha ha only serious even suggested dropping said anvil on his own head). If AIXI/AIXItl can reach this level, at the very least it should be capable of oh-so-human level reasoning(up to and including the kinds of risky behaviour that we all probably would like to pretend we never engaged in), and could possibly transcend it in the same way that humans do: by trial and error, by limiting potential damage to individuals, or groups, and fighting the neverending battle against ecological harms on its own terms on the time schedule of 'let it go until it is necessary to address the possible existential threat'.
Of course it may be that the human way of avoiding species self-destruction is fatally flawed, including but not limited to creating something like AIXI/AIXItl. But it seems to me that is a limiting, rather than a fatal flaw. And it may yet be that the way out of our own fatal flaws, and the way out of AIXI/AIXItl's fatal flaws are only possible by some kind of mutual dependence, like the mutual dependence of two sides of a bridge. I don't know.
Either way, the question is guaranteed to have an answer. You even have a nice, concrete place to begin tracing—your belief, sitting there solidly in your mind.
In retrospect this seems like an obvious implication of belief in belief. I would have probably never figured it out on my own, but now that I've seen both, I can't unsee the connection.
No one has complicated thoughts about several dreams from totally different genres while experiencing that one is unable to move a muscle without being awake.
...I've had some pretty complicated dreams, where I've woken up from a dream(!), gone to work, made coffee, had discussions about the previous dream, had thoughts about the morality or immorality of the dream, then sometime later come to a conclusion that something was out of place(I'm not wearing pants?!) then woken up to realize that I was dreaming. I've had nested dreams a good couple of layers deep with this sort of thing going on.
That said I think you have something there, though. Sometimes I wake up (Dream or otherwise) and I remember my dream really vividly, especially when I awake suddenly, due to an alarm clock or something
But I've never had a dream that I struggled to remember what was in my dream inside of my dream. At the least, such an activity should really raise my priors that I'm toplevel.
Looks like somewhere along the transition to lesswrong, the trackback to this related OB post appears to have been lost. It's worth digging a step deeper for the context, here.
Including: "twitter", "altruism", "trust", "start" and "curiosity" apparently?
Obviously the above is copypasta from Wikipedia at no doubt the time of the parent's posting.
In case it's edited/the edit history is wiped in the future:
[2] Susan Carey (2009). The Origin of Concepts. Oxford University Press. ISBN 978-0-19-536763-8.
[3] Gregory Murphy (2002). The Big Book of Concepts. Massachusetts Institute of Technology. ISBN 0-262-13409-8.
[4] Stephen Lawrence; Eric Margolis (1999). Concepts and Cognitive Science. in Concepts: Core Readings: Massachusetts Institute of Technology. pp. 3–83. ISBN 978-0-262-13353-1
[5] Roger Brown (1978). A New Paradigm of Reference. Academic Press Inc. pp. 159–166. ISBN 0-12-497750-2.
This link is 404ing. Anyone have a copy of this?
Couldn't you argue this the opposite way? That life is such misery, that extra torture isn't really adding to it.
The world with the torture gives 3^^^3+1 suffering souls a life of misery, suffering and torture.
The world with the specs gives 3^^^3+1 suffering souls a life of misery, suffering and torture, only basically everyone gets extra specks of dust in their eye.
In which case, the first is better?
This strikes me as an unfortunately place and time-sensitive OvercomingBias/LessWrong post. As the moral character and fashions change with the change of generations, it's going to lose its edge. While the reader is going to vaguely understand the general idea...they may not really 'get' why or that cryonics was that far outside the overton window to begin with. It might warrant relooking at or retelling this particular set of stories in a more recent context later on. I wonder if the retelling of the Sequences later on end up doing just this.
There seems to be one class of a political topic that seems to lead inevitably to this (and to which I'm guilty of). Any political stance that affects the means of creative production(for example, copyright/free culture). If you get to the point where you cannot stand the people who are making art for political reasons, you are forced to create your own. The result is going to be usually awful. Warning people against creating awful art in that case goes too far -- awful art probably needs to be created in order for masterpieces to emerge from in relation to. But a reminder that it is awful and that it's likely unsubjected to the scrutiny of a billion eyes for the period of time more mainstream art is(thus weeding out most crap in many iterative processes of crap removal), and can lead to affective death spirals, is a Good Thing.
According to http://www.technologyreview.com/view/513781/moores-law-and-the-origin-of-life/?utm_content=bufferc6744&utm_source=buffer&utm_medium=twitter&utm_campaign=Buffer http://arxiv.org/abs/1304.3381
This rate is increasing with time, or the earth is younger than life.
Your second link is broken. In addition to the Internet archive I have posted a blog post inspired by some of my experiences with a cult, containing the article in its entirety for posterity.
A sufficiently powerful genie might make safe genies by definition more unsafe. Then your wish could be granted.
edit (2015) caution: I think this particular comment is harmless in retrospect... but I wouldn't give it much weight
Is not your second link dealt with by http://lesswrong.com/lw/iv/the_futility_of_emergence/ or am I misreading one of the two? It seems to leave the main causal mechanism abstract enough to prove anything.
It appears I'm less rational than I thought. I suppose another way to rephrase that would be that to draw the outline of VNM-rational decisions only up to preferences that are meaningfully resolvable(and TORTURE vs SPECK does not appear to be to me at least) with a heuristic of how to resolve them clearer given intereaction with unresolveable areas. I would still be making a choice, albeit one with the goal of expanding rational decisionmaking to the utmost possible(it would be rational to be as rational as permissable). That seems pretty cheap though, reeking of 'explaining everything'. Worse, one interpretation of this dilemma would be that you have to resolve your preferences and that 'middle' is excluded, in which case it is a hard problem to which case I can likely offer no further suggestion.
I suppose you could view the utility as a meaninful object in this frame and abstract away the dust, too, but in the end the dust-utility system is going to encompaps both anyway so solving the problem on either level is going to solve it on both.
I chose RANDOM* and feel that this
Satisfies the suggestion of making sure that you choose/'state a preference' (the result of RANDOM is acceptable to me and I would be willing to work past it and not dwelling on it).
Satisfies the suggestion of making sure you state assumptions to the extent you're able to resolve them (RANDOM implies a structure upon which RANDOM acts and I was already thinking about implications of either choice, though perhaps I could have thought more clearly about the consequences of RANDOM specifically)
does not compromise me as a (wannabe) rational person (ie I use the situation to update previous beliefs)
Not to allow the alternatives to distract afterwards (as once the choice RANDOM is made, it cannot be unmade -- future choices can be made RANDOM, TORTURE, SPECKS or otherwise)
Does not compromise future escape routes (RANDOM, SPECK, RANDOM, TORTURE is just as an acceptable sequence of choices to me as SPECK, TORTURE, SPECK, TORTURE -- it just depends what evidence and to what extent evidence has been entangled)
but has the additional benefit of
not biasing me towards my choice very much. If SPECKS or TORTURE is chosen, it is tempting to 'join team SPECKS'. I suppose I'll be tempted to join team RANDOM, but since RANDOM is a team that COOPERATEs with teams SPECKS and TORTURE something GOOD will come of that anyway.
Reserving my agency, and the perception of my agency for other decisions(though they may perhaps be less important(3^^^3 dust specks is a potentially VERY IMPORTANT!!!!!!!!!!!!!!!!!!!! decision), they will be mine), such as meta-decisions on future cases involving and not involving RANDOM)
in fact let's see if I can rephrase this post
META-TORTURE and META-SPECKS stances exist that disposition us away from TORTURE and SPECKS that are harder to express when making a decision or discussing decisions with people and that to avoid holding these stances that cannot be held to rational scrutiny by ourselves and others so well that we should avoid making them. That it is possible to get into a situation where we fail to resolve a Third Alternative where we must choose and that making the correct choice, as an altruist/rationalist/etc is important even in these cases. SPECKS or TORTURE seem to be the only choices, pick one.
I maintain however that RANDOM or DEFAULT will always be by the nature of what a choice is, always, logically, available.
*actually I chose DEFAULT/RANDOM but the more I think about it the more I think RANDOM is justified
The probability I'm the only person person selected out of 3^^^3 for such a decision p(i) is less than any reasonable estimate of how many people could be selected, imho. Let's say well below 700dB against. The chances are much greater that some probability fo those about to be dust specked or tortured also gets this choice (p(k)). p(k)*3^^^3 > p(i) => 3^^^3 > p(i)/p(k) => true for any reasonable p(i)/p(k)
So this means that the effective number of dust particles given to each of us is going to be roughly (1-p(i))p(k)3^^^3.
I'm going to assume any amount of dust larger in mass than a few orders of magnitude above the Chandrasekhar limit (1e33 kg) is going to result in a black hole. I can even assume a significant error margin in my understanding of how black holes work, and the reuslts do not change.
The smallest dust particle is probably a single hydrogen atom(really everything resoles to hydrogen at small enough quantities, right?). 1 mol of hydrogen weighs about 1 gram. So (1-p(i))(p(k)3^^^3 (1 gram/mol)(6e-23 'specks'/mol) (1e-3 kg/g) (1e-33 kg/black hole) = roughly ( 3^^^3 ) (~1e-730) = roughly 3^^^3 black holes.
ie 3^(3_1^3_2^3_3^...^3_7e13 -730) = roughly 3^(3_1^3_2^3_3^...^3_7e13)
ie 3_1^3_2^3_3^...^3_7e13 - 730 = roughly 3_1^3_2^3_3^...^3_7e13.
In conclusion, I think at this level, I would choose 'cancel' / 'default' / 'roll a dice and determine the choice randomly/not choose' BUT would woefully update my concept of the sizee of the universe to contain enough mass to even support a reasonably infentessimal probability of some proportion of 3^^^3 specks of dust, and 3^^^3 people or at least some reasonable proportion thereof.
The question I have now is how is our model of the universe to update given this moral dillema? What is the new radius of the universe given this situation? It can't be big enough for 3^^^3 dust specks piled on the edge of our universe outside of our light cone somewhere. Either way I think the new radius ought to be termed the "Yudkowsky Radius".
Your link is 404ing. Is http://spot.colorado.edu/~norcross/Comparingharms.pdf the same one?
Likewise, if this was iterated 3^^^3+1 times(ie 3^^^3 plus the reader),it could easily be 50*3^^^3 (ie > 3^^^3+1) people tortured. The odds are if it's possible for you to make this choice, unless you have reason to believe otherwise they may too, making this an implicit prisoner's dilemma of sorts. On the other side, 3^^^3 specks could possibly crush you, and/or your local cluster of galaxies into a black hole, so there's that to consider if you consider the life within meaningful distance of of every one of those 3^^^3 people valuable.
link for the lazy
Sadly, your link is broken. Do you have a copy of this one?
edit : nevermind internet archive comes through.
A few thoughts:
Whether this is a good idea or not might very well be something you have to try for yourself to know; but on the flip side we are fairly different people, 20 years after any particular decision. While some decisions may be final, especially in an age where we can display attributes of ourself very publicly it might make more sense to have a publicly accessible Crocker's Flag somewhere that might be unset perhaps 20 years down the line so that you don't damn your future self to a life of shameful feelings beyond necessary.
Secondly, 'none of your business' is neither radically honest, and with the possible exception of the person hiding from the secret police, I've long maintained that since MAD/nuclear weapons became a possibility that there is no such thing as 'none of your business'; we all have a vested interest in the emotional situation and financial incentives of those in the global village. Should a veil of secrecy exist, it may very well cover that which will undo us all.
edit 2013-me did not understand the full consequences of global surveillance. While it's true that what's covered by a veil of secrecy would doubtless cover the seeds of our destruction, we are all hiding from the secret police, post 2013. Proceed with caution
This link is broken.
http://www.youtube.com/watch?v=mthDxnFXs9k looks like it might be the same video though.
Or alternatively you could go more recent & use the Baltimore dialect and use 'Yo' as a gender-neutral pronoun.
(ref: Stotko, E. and Troyer, M. "A new gender-neutral pronoun in Baltimore, Maryland: A preliminary study." American Speech, Vol. 82. No. 3, Fall 2007, p. 262.)
Earlier on in internet history there was a movement to make 'tse' a gender-neutral pronoun. It didn't take, but I still use it.
"No results found for \" expert moral system \"."
-google.
Remember: it doesn't have to be perfect, just better than us.
edit google was misinformed - this has been discussed. Nevertheless the point stands -- unless there's a particular reason why we think that we would perform better than an expert system in this topic I am skeptical that acting except insofar as to create one is anything but short-term context dependent moral.
I would explain the concepts of the craziest, most non-obvious ideally moral ideas I've ever had [such as the idea that Nick Bostrom's Interstellar Opportunity Cost paper completely changes the nature of the pro-life debate, such that it no longer is sensible to freeze all fetuses instead of aborting them, instead, if we're serious about being pro-life we should crop humanity down to only what is needed to spread human life to other stars, and that there are economic considerations to freedom but that they are subtle and complex]. Something that is so ...off the wall might not go through directly, but it might come through as something equally out there [that the greeks should dedicate all their energies and efforts to seafaring and trade]
And in fact the further in the future I would be aiming to talk about the better. What matters about 2,000 paltry years when we're talking about post-singularity or near-singularity times? The differences between us will be minor in comparison, and not be 'smudged' by the chronophone.
Wouldn't he have just discarded it as he was trained with other notation?
It would be like someone in the modern english world trying to learn chinese math notation. We could probably understand the concepts so could in principle do it, but it would seem relatively unweildly, even if it turns out that it's a much more elegant way of doing math. We'd never know.
He might very well learn it and then go "that's cute" and then ignore it.
All the studies say that your odds are just not good enough to be worth it.
...and even if you are, people who are able to re-arrange the odds to their favour may end up crowding out the honest ones ;)
Closely related: escallation of commitment While it's possible to not escalate commitment when you're in a losing situation, it is often our default tendency.