Posts
Comments
nothing has 100% certainty, nothing can have a 0% uncertainty
That's my understanding as well. I was trying to say that, if you were to formalize all this mathematically, and took the limit as number of Bayesian updates n went to infinity, uncertainty would go to zero.
Since we don't have infinite time to do an infinite number of updates, in practice there is always some level of uncertainty > 0%.
Comment 1: If anyone wants to comment or reply here, but can’t afford the karma hit, feel free to PM me and I’ll comment for you without listing your name. I have 169 karma to burn (97% positive!), from comments going back to Feb 2015. However, I’ve wanted to update to a different username, so I don’t mind destroying this one.
Comment 2: It might be wise not to discuss tactics where eugine can read it. (Also, causing lots of discussion might be his goal, but so far we haven’t talked about it much and it’s just been a background annoyance.)
Is there interest in a skype call or some other private forum to discuss possible solutions?
[pollid:1160]
I believe CronoDAS is referring to Algernon's Law. Gwern describes the issues pretty well here, including several classes on "loopholes" we might employ to escape the general rule.
The classifications of different types of loopholes is still pretty high level, and I'd love to see some more concrete and actionable proposals. So, don't take this as saying "this is old hat", but only as a jumping off point for further discussion.
This may not be a generalized solution, but it looks like you have rigorously defined a class of extremely common problems. I suspect deriving a solution from game theory would be the formalized version of John Stuart Mill trying to derive various principles of Liberty from Utilitarianism.
Meta: 4.5 hours to write, 30mins to take feedback and edit.
I always find this sort of info interesting. Same for epistemic status. It's nice to know whether someone is spit-balling a weird idea they aren't at all sure of, versus trying to defend a rigorous thesis. That context is often missing in online discussions, and I'd love for it to become the norm here. I suppose knowing how much time you spent writing something only gives me a lower bound on how much thought has gone into the idea total, and some ideas really can be fleshed out completely in a couple hours while others may take generations.
I was surprised to see mention of MIRI and Existential Risk. That means that they did a little research. Without that, I'd be >99% sure it was a scam.
I wonder if this hints at their methodology. Assuming it is a scam, I'd guess they find small but successful charities, then find small tight-knit communities organized around them and target those communities. Broad, catch-all nets may catch a few gullible people, but if enough people have caught on then perhaps a more targeted approach is actually more lucrative?
Really, it's a shame to see this happen even if no one here fell for it, because now we're all a little less likely to be receptive to weird requests/offers. I suspect it's useful for EAs to be able to make random requests of specific people. For example, I can imagine needing a couple hours or days of consulting work from a domain expert. In that situation, I'd be tempted to PM someone knowledgeable in that area, and offer to pay them for some consulting work on the side.
I can actually think of 2 instances where this community has done things like this out in the open (not PM), so it wouldn't surprise me if there are occasional private transactions. (I'd link to examples, but I'd rather not help a potential scammer improve on their methods.) Perhaps a solution would be to route anything that looks suspicious through Bitcoin, so that the transaction can't be cancelled? I wouldn't want to add trivial inconveniences to non-suspicious things, though.
Although compressing a complex concept down to a short term obviously isn't lossless compression, I hadn't considered how confusing the illusion of transparency might be. I would have strongly preferred that "Thinking Fast and Slow" continue to use the words "fast" and "slow". As such, these were quite novel points:
they don't immediately and easily seem like you already understand them if you haven't been exposed to that particular source
they don't overshadow people who do know them into assuming that the names contain the most important features
The notion of using various examples to "triangulate" a precise meaning was also a new concept to me too. It calls to mind the image of a Venn diagram with 3 circles, each representing an example. I don't think I have mental models for several aspects of learning. Gwern's write up on spaced repetition gave me an understanding about how memorization works, but it hadn't occurred to me that I had a similar gap in my model (or lack thereof) for how understanding works.
(I'm not sure the triangulation metaphor lends much additional predictive power. However, an explicit model is a step up from a vague notion that it's useful to have more examples with more variety.)
I've always hated jargon, and this piece did a good job of convincing me of its necessity. I plan to add a lot of jargon to an Anki deck, to avoid hand-waving at big concepts quite so much.
However, there are still some pretty big drawbacks in certain circumstances. A recent Slate Star Codex comment expressed it better than I ever have:
One cautionary note about “Use strong concept handles”: This leans very close to coining new terms, and that can cause problems.
Dr. K. Eric Drexler coined quite a few of them while arguing for the feasibility of atomically precise fabrication (aka nanotechnology): “exoergic”, “eutactic”, “machine phase”, and I think that contributed to his difficulties.
If a newly coined term spreads widely, great! Yes it will an aid to clarity of discussion. If it spreads throughout one group, but not widely, then it becomes an in-group marker. To the extent that it marks group boundaries, it then becomes yet another bone of contention. If it is only noticed and used within a very small group, then it becomes something like project-specific jargon – cryptic to anyone outside a very narrow group (even to the equivalent of adjacent departments), and can wind up impeding communications.
Meta note before actual content: I've been noticing of late how many comments on LW, including my own, are nitpicks or small criticisms. Contrarianism is probably the root of why our kind can't cooperate, and maybe even the reason so many people lurk and don't post. So, let me preface this by thanking you for the post, and saying that I'm sharing this just as an FYI and not as a critique. This certainly isn't a knock-down argument against anything you've said. Just something I thought was interesting, and might be helpful to keep in mind. :)
Clearly it was a moral error to assume that blacks had less moral weight than whites. The animal rights movement is basically just trying to make sure we don't repeat this mistake with non-human animals. (Hence the use of terms like "speciesism".) You use a couple reductio ad absurdum arguments with bacteria and video game characters, but it’s not entirely clear that we aren’t just socially biased there too. If the absurd turns out to be true, then the reductio ad absurdum fails. These arguments are valid ways of concluding "if A than B", but keep in mind that A isn't 100% certain.
There are actually some surprisingly intelligent arguments that insects, bacteria, some types of video game characters, and even fundamental particles might have non-zero moral weight. The question is what probability one gives to those propositions turning out to be true. IF one has reviewed the relevant arguments, and assigns them infinitesimally small credence, THEN one can safely apply the reductio ad absurdum. IF certain simple algorithms have no moral weight and the algorithms behind human brains have high moral weight, THEN algorithms almost as simple are unlikely to have whatever property gives humans value, while complex algorithms (like those running in dolphin brains) might still have intrinsic value.
As I understand it, Eliezer Yudkowski doesn't do much coding, but mostly purely theoretical stuff. I think most of Superintelligence could have been written on a typewriter based on printed research. I also suspect that there are plenty of academic papers which could be written by hand.
However, as you point out, there are also clearly some cases where it would take much, much longer to do the same work by hand. I'd disagree that it would take infinite time, and that it can't be done by hand, but that's just me being pedantic and doesn't get to the substance of the thing.
The questions that would be interesting to answer would be how much work falls into the first category and how much falls into the second. We might think of this as a continuum, ranging from 0 productivity gain from computers, to trillions of times more efficient. What sub-fields would and wouldn't be possible without today's computers? What types of AI research is enabled simply by faster computers, and which types are enabled by using existing AI?
Maybe I can type at 50 words a minute, but I sure as hell can't code at 50 WPM. Including debugging time, I can write a line of code every couple minutes, if I'm lucky. Looking back on the past 2 things I wrote, one was ~50 lines of code and took me at least an hour or two if I recall, and the other was ~200 lines and took probably a day or two of solid work. I'm just starting to learn a new language, so I'm slower than in more familiar languages, but the point stands. This hints that, for me at least, the computer isn't the limiting factor. It might take a little longer if I was using punch cards, and at worst maybe twice as long if I was drafting everything by hand, but the computer isn't a huge productivity booster.
Maybe there's an AI researcher out there who spends most of his or her day trying different machine learning algorithms to try and improve them. Even if not, It'd still take forever to crunch that type of algorithm by hand. It'd be a safe bet that anyone who spends a lot of time waiting for code to compile, or who rents time on a supercomputer, is doing work where the computer is the limiting factor. It seems valuable to know which areas might grow exponentially alongside Moore's law, and which might grow based on AI improvements, as OP pointed out.
I like this idea. I'd guess that a real economist would phrase this problem as trying to measure productivity. This isn't particularly useful though. Productivity is output (AI research) value over input (time), so this begs the question of how to measure the output half. (I mention it mainly just in case it's a useful search term.)
I'm no economist, but I do have an idea for measuring the output. It's very much a hacky KISS approach, but might suffice. I'd try and poll various researchers, and ask them to estimate how much longer it would take them to do their work by slide-rule. You could ask older generations of researchers the same thing about past work. You could also ask how much faster their work would have been if they could have done it on modern computers.
It would also probably be useful to know what fraction of researcher's time is spent using a computer. Ideally you would know how much time was spent running AI-specific programs, versus things like typing notes/reports into Microsoft Word. (which could clearly be done on a typewriter or by hand.) Programs like RescueTime could monitor this going forward, but you'd have to rely on anecdotal data to get a historical trend line. However, anecdote is probably good enough to get an order-of-magnitude estimate.
You'd definitely want a control, though. People's memories can blur together, especially over decades. Maybe find a related field for whom data actually does exist? (From renting time on old computers? There must be at least some records.) If there are old computer logs specific to AI researchers, it would be fantastic to be able to correlate something like citations/research paper or number of papers per researcher per year with computer purchases. (Did such-and-such universitys new punch-card machine actually increase productivity?) Publication rates in general are skyrocketing, and academic trends shift, so I suspect that publications is a hopelessly confounded metric on a timescale of decades, but might be able to show changes from one year to the next.
Another reason for good control group, if I remember correctly, is that productivity of industry as a whole didn't actually improve much by computers; people just think it was. It might also be worth digging around in the Industrial-Organizational Psychology literature to see if you can find studies involving productivity that are specific to AI research, or even something more generic like Computer Science. (I did a quick search on Google Scholar, and determined that all my search terms were far too common to narrow things down the the oddly-specific target.)
I could get behind most of the ideas discussed here, but I'm wary of the entire "Standards of Discourse and Policy on Mindkillers" section. It's refreshing to have a section of the internet not concerned with politics. Besides, I don't think the world is even Pareto optimized, so I don't think political discussions are even useful, since acquiring better political views incur opportunity costs. Why fight the other side to gain an inch of ground when we could do something less controversial but highly efficient at improving things? I'm all for discussing weird, counterintuitive, and neglected topics, but politics is only interesting for the same reason soap operas and dramas are interesting. The most viral arguments aren't necessarily the most worthwhile.
As for mandatory Crocker's rules, the wiki article has this to say:
Crocker emphasized, repeatedly, in Wikipedia discourse and elsewhere, that one could only adopt Crocker's rules to apply to oneself, and could not impose them on a debate or forum with participants who had not opted-in explicitly to these rules, nor use them to exclude any participant.
I suspect that if Crocker's rules were mandatory for participation in something, there would be a large number of people who would be pushed into accepting them. I don't think this would actually improve anything. Lots of people invoking Crocker's rules is a symptom of a healthy discourse, not a cause of it. Personally, I know that when I invoke Crocker's rules I have a much smaller knee-jerk reaction to criticism. LessWrong can already be a blunt place at times, probably more than is optimal.
I probably have 50 pages of random notes and thoughts that would be appropriate to post here, but haven't. Some are things I started to write specifically for LW, but never finished polishing. I suspect both the community and I would benefit from the discussion, but honestly it takes an order of magnitude more time for me to get something to a state where I would be willing to post it here. That's probably twice as much time as for me to be willing to post something on Facebook. I get diminishing returns from rereading the same thing over and over again, but it's much more difficult to achieve activation energy here. I suspect that difference is mostly due to the subjective feel of the group.
By "manufactured values" I meant artificial values coming from nurture rather than innate human nature. Obviously there are things we give terminal value, and things we give instrumental value. I meant to refer to a subset of our terminal values which we were not born with. That may be a null set, if it is impossible to manufacture artificial values from scratch or from acquired tastes. Even if this is the case, that wouldn't imply that instrumental values could not be constructed from terminal values as we learn about the world. There are 4 possible categories, and I meant only to refer to the last one:
Innate terminal values: “Being generous is innately good, and those who share are good people.” (Note: “generosity admired” is on the list of human universals, so it’s likely to be an innate value we are born with.)
Innate instrumental values: N/A (I don’t think there is anything in this category, because innate human values in babies precede the capacity to reason and develop instrumental values. Maybe certain aesthetic values don’t express themselves until a baby first opens its eyes, and so there could be reasoned instrumental values which are more “innate” than aesthetic values.)
learned instrumental values: “eating spicy food is good to do because it clears your sinuses”
learned terminal values (that is, “manufactured” values): “Bacteria suffering matters, even though I have no emotional connection to them, because of these abstract notions of fairness.” Or, alternatively “Eating spicy food is a pure, virtuous activity in its own right rather than for some other reason. Those who partake are thus good people, and those who don’t are unclean and subhuman.” The former is merely extrapolated from existing values and dubbed a terminal value, while the latter arises from an artificially conditioned aesthetic.
To use a more LW-central example, those of us who favor epistemic rationality over instrumental rationality do so because true knowledge is a terminal value for us. If this value is a human universal, then that would be strong evidence that every neurotypical baby is born valuing truth, and therefore that truth-seeking is a terminal value. If only a few cultures value truth, then it would seem more plausible that truth-seeking was a manufactured terminal value or an instrumental value.
To test ideas like this, we can look at the terms on the list related to epistemic rationality: abstraction in speech & thought, classification, conjectural reasoning, interpolation, logical notions [there are several examples on the list], measuring, numerals (counting), overestimating objectivity of thought, semantics [several semantic categories are also on the list], true and false distinguished. So, either all cultures get a lot of value out of instrumental truth-seeking, or truth-seeking is an innate human value. Judging by the curiosity of children, I’m strongly inclined toward the latter. Perhaps LW users have refined and accentuated their innate human curiosity, but it certainly doesn’t seem like a manufactured value.
But it looks like you guys forced me to make my question specific enough that I could answer it empirically. I could just take each item on the list of the twelve virtues of rationality, or any other list I thought gave a good representation of LW values or intellectual values. Just cross-reference them against a couple lists of human universals and lists of traits of small children. If very small children display a value, it’s probably innate, but may be learned very early. If no infants have it but some/all adults do, it’s probably a learned value developed later in life. If it seems like it is probably a learned value, and seems subjectively to be a terminal value, then it is manufactured.
Also, to be clear, just because something is manufactured doesn’t make it a bad thing. To say so is to commit the naturalistic fallacy. However, altering one’s utility function is scary. If we are going to replace our natural impulses with more refined values, we should do so carefully. Things like the trolley problem arguably segregate people who have replaced their default values with more abstract utilitarian notions (value all lives equally, regardless of in-group or a sense of duty). Extrapolating new values from existing ones doesn’t seem as dangerous as deriving them from acquired tastes.
My possibly stupid question is: "Are some/all of LessWrong's values manufactured?"
Robin Hanson brings up the plasticity of values. Humans exposed to spicy food and social conformity pressures rewire their brain to make the pain pleasurable. The jump from plastic qualia to plastic values is a big one, but it seems plausible. It seems likely that cultural prestige causes people to rewire things like research, studying, etc. as interesting/pleasurable. Perhaps intellectual values and highbrow culture are entirely manufactured values. This seems mildly troubling to me, but it would explain why rationality and logic are so hard to come by. Perhaps the geek to nerd metamorphosis involves a more substantial utility function modification than merely acquiring a taste for something new.
"well, you're also ultimately basing yourself on intuitions for things like logic, existence of mind-independent objects, Occamian priors, and all the other viewpoints that you view as intuitively plausible, so I can jolly well use whatever intuitions I feel like too."
It's true that a priori using intuition is about as good as using an intuitive tool like inductive reasoning. However, induction has a very very strong track record. The entire history of science is one of humans starting out with certain intuitive priors, and huge numbers of them being challenged by experimental evidence. Babies learn about the world using induction, so each one of us has a mountain of evidence supporting its usefulness. Once you perform a Baysian update on that evidence, intuition looks like a much less useful predictor of future events.
Sure, you can deny the validity of induction. You can also claim that "I think therefor I am" is invalid because it isn't based on anything. (What is this "therefore" concept? What is "I"? What does it mean to "exist"?) You can even decide when to use induction versus intuition based on whim or based on which conclusion you want to “prove”. It’s just that doing so is incoherent.
Yes, I did just use an informal form of inductive reasoning, by using observations of evidence to demonstrate that induction seems valid upon reflection. Yes, that makes me feel dirty inside. But you have to start somewhere, and we don't really have any better options. It should be noted that anyone who says "intuition is a better option" learned those 5 words through induction, and probably uses induction to override intuitions every day. It is mathematically provable that no mathematical system can assert its own soundness without becoming inconsistent. We can't prove the validity of induction using induction. The best we can do is try to falsify the hypothesis "induction is valid". If looking at the track record of induction seemed to indicate that it wasn't valid, then we'd have an even worse mess on our hands, but fortunately that isn't the case. If someone can come up with a better alternative to induction, they'd better be able to demonstrate that it is better.
Some would call that faith. I'd counter that faith is belief regardless of evidence, and this is forming beliefs based on all available evidence. If that's faith, then so is every belief about anything.
the idea of people buying our product because we are EAs makes me uncomfortable.
In retrospect, I think that would make me uncomfortable too. In your position, I'd probably feel like I'd delivered an ultimatum to someone else, even if they were the one who actually made the suggestion. On the other hand, maybe a deep feeling of obligation to charity isn't a bad thing?
Would you say that you are not interested in paying more for a healthier product, not convinced that MealSquares is better for you, something else?
Based on my (fairly limited) understanding of nutrition, I suspect that any marginal difference between your products is fairly small. I suspect humans get strongly diminishing returns (in the form of increased lifespan) once we have our basic nutritional requirements met in bio-available forms and without huge amounts of anything harmful. After that, I'd expect the noise to overpower the signal. For example, perhaps unmeasured factors like my mood or eating habits change as a function of my Soylent/MealSquares choice, and I wind up getting fast food more often, or get less work done or something. Let's say it would take me a month of solid researching and reading nutrition textbooks to make a semi-educated decision of which of two good things is best. Would the added health benefit give me an additional month of life? What if I value my healthy life, here and now, far more than 1 more month spent senile in a nursing home? What if I also apply hyperbolic discounting?
I've probably done more directed health-related reading than most people. (Maybe 24 hours total, over the pasty year or so?) Enough to minimize the biggest causes of death, and have some vague idea of what "healthy" might look like. Enough to start fooling around with my own DIY soylent, even if I wouldn't want to eat that every day without more research. If someone who sounds knowledgeable sits down and does an independent review, I'd probably read it and scan the comments for critiques of the review.
Someone gave you a downvote. If it was on my behalf or on the behalf of Soylent, then for the record I thought it was funny. :)
Hmmm, that's worrying. I played with some numbers for a 5'6" male, and got this:
99 lbs yields "Your BMI is way too low to be living"
100lbs yields 74 years
150lbs yields 76 years
200lbs yields 73 years
250lbs yields 69 years
300lbs yields 69 years
500lbs yields 69 years
999lbs yields 69 years
It looks to me like they are pulling data from a table, and the table maxes out under 250lbs?
avoiding obesity
Not to be pedantic, but I thought this might be of interest: As I understand it, amount of exercise is a better predictor of lifespan than weight. That is, I would expect someone overweight but who exercises regularly to outlive someone skinny who never exercises.
For example, this life expectancy calculator outputs 70 years for a 5"6" 25 year old male who weighs 300lbs, but exercises vigorously daily. Changing the weight to 150 lbs and putting in no exercise raised the life expectancy by only 1 year. (a bit less than I was expecting, actually. I was about to significantly update, but then it occurred to me that 300 lbs isn't the definition of obesity. I knew this previously, but apparently hadn't fully internalized that.) EDIT: This calculator may not work well for weights over ~250 lbs. See comment below.
So, my top two recommendations to friends would be quit smoking and exercise regularly. I'd recommend Less Wrongers either do high intensity workouts once a can read or watch Khan Academy or listen to The week to minimize the amount of time spent on non-productive activities, or pick a more frequent but lower intensity activity they Sequences audiobook while doing. I'm not an expert or anything. That's just the impression I've gotten from my own research.
you'll help us earn money for effective giving
I realize you are in the startup phase now, and so it probably makes sense for you to put any surplus funds into growth rather than donating now. However, 2 questions:
Once you finish with your growth phase, about what percent of your net proceeds do you expect to donate?
What sorts of EA charities are you interested in?
I've been using MealSquares regularly, without realizing that that you guys were LWers or EAs. As such, I've been using mostly s/Soylent because of the cost difference. (A 400 Calorie MealSquare is ~$3, a 400 Calorie jug of Soylent 2.0 is ~$2.83, 400 Calories worth of unmixed Soylent powder is ~$1.83, and the ingredients for 400 Calories worth of DIY People Chow are ~$0.70. All these are slightly cheaper with a subscription/large purchase.)
I ask, because if you happen to be interested in similar EA causes to me, and expect to eventually donate X% of proceeds, then I should be budgeting my expenses to factor that in. If (100%-X%) * MealSquares_Cost < soylent_Cost, then I would buy much less soylent and much (/many?) more MealSquares. I'd be paying a premium to Soylent in order to add a bit more culinary variety. (Also, I realize this X isn't equal to the expected altruistic return on investment, but that would be even harder to estimate.)
Good point. It seems like we 1) value an incredibly diverse assortment of things, and 2) value our freedom to fixate on any particular one of those things. So, any future which lacks some option we now have will be lacking. Because at some point we have to choose one future over another, perhaps we will always have a tiny bit of nostalgia. (Assuming that the notion of removing that nostalgia from our minds is also abhorrent.)
I'll also note that after a bit more contemplation, I've shifted my views from what I expressed in the second paragraph of my comment above. It seems plausible that certain classes of problems tickle a certain part of our brain. Visual stimuli excite our visual cortex, so maybe Rubik's Cubes excite the parts of our brain involved in spatial reasoning. It seems plausible, then, that we could add entire new modules to our minds for solving entire new classes of problems. Perhaps neuroplasticity allows us to already do this to a degree, but it also seems likely that a digital mind would be much less restricted in this regard.
A world without complex novelty would be lacking. But so would a world without some simple pleasures. There are people who really do enjoy woodworking. I can't picture a utopia where no one ever whittles. And a few of them will fancy it enough to get really, really good at it, for pretty much the same reason that there are a handful of devoted enthusiasts. Even without Olympic competitions and marathons, I'd bet there would still be plenty of runners, who did so purely for it's own sake, rather than to get better or to compete, or for novelty. Given an infinite amount of time, everyone is likely to spend a great deal of time on such non-novel things. So, what's most disturbing about carving 162,329 table legs is that he altered his utility function to want to do it.
(As best I can grasp the Law, there are insights you can't understand at all without having a brain of sufficient size and sufficient design. Humans are not maximal in this sense, and I don't think there should be any maximum—but that's a rather deep topic, which I shall not explore further in this blog post. Note that Greg Egan seems to explicitly believe the reverse—that humans can understand anything understandable—which explains a lot.)
Perhaps I'm missing something, but it seems to me that any mind capable of designing a turning-complete computer can, in principle, understand any class of problem. I say "class of problem", because I doubt we can even wrap our brains around a 10x10x10x10 Rubik's Cube. But we are aware of simpler puzzles of that class. (And honestly, I'm just using an operational definition of "classes of problem", and haven't fleshed out the notion.) There will always be harder logic puzzles, riddles, and games. But I'm not sure there exist entirely new classes of problems, waiting to be discovered. So we may well start running out of novelty of that type after a couple million years, or even just a couple thousand years.
I absolutely love the poem. Unfortunately every reading I've ever heard is painfully bad, so maybe it isn't a great choice for a spoken piece. The exception is this scene from Interstellar: [Trigger warning? Or is it just me?]
I'm a fan of both versions of Ozymandias, but here's Shelley's version:
I met a traveller from an antique land
Who said: "Two vast and trunkless legs of stone
Stand in the desert. Near them, on the sand,
Half sunk, a shattered visage lies, whose frown,
And wrinkled lip, and sneer of cold command,
Tell that its sculptor well those passions read
Which yet survive, stamped on these lifeless things,
The hand that mocked them and the heart that fed:
And on the pedestal these words appear:
'My name is Ozymandias, king of kings:
Look on my works, ye Mighty, and despair!'
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare
The lone and level sands stretch far away."
Ah, that's the definition about which we were talking past each other. I certainly wouldn’t say that "Reiki might work, and until we test it we just don't know!" Perhaps it "works" somewhat through the placebo effect, but even in the unlikely event of a study showing some random placebo controlled health benefit, it would still be astronomically unlikely that ki was the mechanism. (That's not to say that no one will look at the real mechanism after the fact, and try to pick out some superficial similarity to the idea of "ki".)
But that’s beside the point. For hypotheses that are worth our time to test, we test them precisely because it’s an open question. Until we take the data, it remains an open question. (at least for certain definitions of “open question”) I think that’s the point the author was trying to get at with his infeasible historical example.
In passing, he gestured vaguely at a vague conception of science. I guess that doesn't qualify as an argument, so perhaps there is no argument to steelman. But I think that the vague conception of science he was trying to gesture toward does correspond to a real thing that scientists sometimes do.
In the map-territory analogy, this might correspond to a fuzzy or blank region of the map. A scientifically minded person might well say "One reasonable hypothesis is that the Earth is flat the blank region looks like nearby regions, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question."
But here's the idea I think the author was trying to gesture at. In my experience, most people are way too eager to try and solve problems they don't fully understand. I've often heard scientists and engineers caution against this, but the most notable quote is from the rocket scientist Wernher Von Braun: "One good test is worth a thousand expert opinions". I've seen people like Bill Nye repeat this, and seen plenty of science-themed reminders that test results are often surprising, since the world is often much more complex that we give it credit for.
As for the historical commentary, I completely agree. The scenario isn't historically plausible. The scientific revolution would have had to happen earlier just to produce someone capable of saying the quote, and society would have had to somehow go through a scientific revolution without noticing that the earth was round.
Yeah, but if you steel-man it, I think he was trying to make something similar to a map-territory distinction. It's often useful to make a distinction between the data and our best interpretation of the data. Some conclusions don't require much extrapolation, but others require a great deal.
On LW we happily discuss with very long inferential distances, and talk about regions of hypothesis space with high densities of unknown unknowns. Most scientists, however, work over much smaller inferential distances, with the intent of meticulously build up a rock solid body of knowledge. If things are "open questions" until they are above a confidence interval of, say, 0.99, then just about everything we discuss here is an open question, as the quote suggests.
Using a historical example which happens to be false just complicates things. If I recall, philosophers first hypothesized a round earth around 600 BCE, but didn't prove it experimentally until 300 BCE.
Try thinking of it as a case study, not a comprehensive literature review. I didn't really take anything in there as claiming that if I install Musk's mental software then I will succeed at anything I try. The author explicitly mentions several times that Musk thought SpaceX was more likely to fail than succeed. Similarly, there's bits like this:
Likewise, when an artist or scientist or businessperson chef reasons independently instead of by analogy, and their puzzling happens to both A) turn out well and B) end up outside the box, people call it innovation and marvel at the chef’s ingenuity.
It makes a lot more sense if you read it as a case study. He's positing a bunch of hypotheses, some of which are better worded than others. If you steel-man the ones with obvious holes, most seem plausible. (For example, one of the ones that really annoyed me was the way he worded a claim that older children are less creative, which he blamed on schooling but made no mention of a control group.) But the thing was already pretty long, so I can excuse some of that. He's just hypothesizing a bunch of qualities that are necessary but not sufficient.
I agree that OP was leaning a bit heavy on the advertising methods, and that advertising is almost 100% appeal to emotion. However, I'm not sure that 0% emotional content is quite right either. (For reasons besides argument to moderation.) Occasionally it is necessary to ground things in emotion, to some degree. If I were to argue that dust specs in 3^^^3 people's eyes is a huge amount of suffering, I’d likely wind up appealing to empathy for that vastly huge unfathomable amount of suffering. The argument relies almost exclusively on logic, but the emotional content drives the point home.
However, maybe a more concrete example of the sorts of methods EAs might employ will make it clearer whether or not they are a good idea. If we do decide to use some emotional content, this seems to be an effective science-based way to do it: http://blog.ncase.me/the-science-of-social-change/
Aside from just outlining some methods, the author deals briefly with the ethics. They note that children who read George Washington's Cherry Tree were inspired to be more truthful, while the threats implicit in Pinocchio and Boy Who Cried Wolf didn’t motivate them to lie less than the control group. I have no moral problem with showing someone a good role model, and setting a good example, even if that evokes emotions which influence their decisions. That’s still similar to an appeal to emotion, although the Aristotelian scheme the author mentions would classify it as Ethos rather than Pathos. I’m not sure I’d classify it under Dark Arts. (This feels like it could quickly turn into a confusing mess of different definitions for terms. My only claim is that this is a counterexample, where a small non-rational component of a message seems to be permissible.)
It seems worth noting that EAs are already doing this, to some degree. Here are a couple EA and LW superheroes, off of the top of my head:
Norman Borlough saved a billion lives from starvation by making sweeping improvements in crop yields using industrial agriculture. https://80000hours.org/2011/11/high-impact-science/
Viktor Zhdanov convinced the World Health Assembly, by a margin of only 2 votes, to eradicate Smallpox, saving perhaps hundreds of millions of lives. https://80000hours.org/2012/02/in-praise-of-viktor-zhdanov/
Stanislav Petrov day was celebrated here a bit over a month ago, although there are others who arguably averted closer cold war near-misses, but on days less convenient to make a holiday out of.
One could argue that we should only discuss these sorts of people purely for how their stories inform the present. However, if their stories have an aspirational impact, then it seems reasonable to share that. I’d have a big problem if EA turned into a click-maximizing advertising campaign, or launched infomercials. I agree with you there. There are some techniques which we definitely shouldn’t employ. But some methods besides pure reason legitimately do seem advisable. But guilting someone out of pocket change is significantly different from acquiring new members by encouraging them to aspire to something, and then giving them the tools to work toward that common goal. It’s not all framing.
Guilty. I've spent most of my life trying to articulate and rigorously define what our goals should be. It takes an extra little bit of cognitive effort to model others as lacking that sense of purpose, rather than merely having lots of different well-defined goals.
(EDIT, to avoid talking past each other: Not that people don't have any well defined sub-goals, mind you. Just not well defined terminal values, and well defined knowledge of their utility function. No well-defined answers to Life, The Universe, And Everything.)
This is a good point. Perhaps an alternative target audience to "emotionally oriented donars" would be "Geeks". Currently, EA is heavily focused on the Nerd demographic. However, I don't see any major problems with branching out from scientists to science fans. There are plenty of people who would endorse and encourage effectiveness in charities, even if they suck at math. If EA became 99.9% non-math people, it would obviously be difficult maintain a number crunching focus on effectiveness. However this seems unlikely, and compared to recruiting "emotionally-oriented" newbies it seems like there would be much less risk of losing our core values.
Maybe "Better Giving Through SCIENCE!" would make a better slogan than "Be A Superdonor"? I've only given this a few minutes of thought, so feel free to improve on or correct any of these ideas.
Excellent point. Most people aren't trying and failing to achieve their dreams. We aren’t even trying. We don’t have well-articulated dreams, so trying isn’t even a reasonable course of action until we have a clear objective. I'd guess that most adults still don't know what they want to be when they grow up, and still haven't figured it out by the time they retire.
So, all arguments which do not make different predictions are extensionally equal, but are not intensional. From the Wikipedia page:
Consider the two functions f and g mapping from and to natural numbers, defined as follows:
To find f(n), first add 5 to n, then multiply by 2.
To find g(n), first multiply n by 2, then add 10.
These functions are extensionally equal; given the same input, both functions always produce the same value. But the definitions of the functions are not equal, and in that intensional sense the functions are not the same.
That provided me with some perspective. I'd only been thinking of cases where we imposed limitations, such as those we use with Alcohol and addictive drugs. But, as you point out, there are also regulations which push us toward immediate gratification, rather than away. If, after much deliberation, we collectively decide that 99% of potential values are long term, then perhaps we'd wind up abolishing most or all such regulations, assuming that most System 2 values would benefit.
However, at least some System 2 values are likely orthogonal to these sorts of motivators. For instance, perhaps NaNoWriMo participation would go down in a world with fewer social and economic safety nets, since many people would be struggling up Maslow's Hierarchy of Needs instead of writing. I'm not sure how large of a fraction of System 2 values would be aided by negative reinforcement. There would be a large number of people who would abandon their long-term goals in order to remove the negative stimuli ASAP. If the shortest path to removing the stimuli gets them 90% of the way toward a goal, then I'd expect most people to achieve the remaining 10%. However, for goals that are orthogonal to pain and hunger, we might actually expect a lower rate of achievement.
If descriptive ethics research shows that System 2 preferences dominate, and if the majority of that weighted value is held back by safety nets, then it'll be time to start cutting through red tape. If System 2 preferences dominate, and the majority of moral weight is supported by safety nets, then perhaps we need more cushions or even Basic Income. If our considered preference is actually to "live in the moment" (System 1 preferences dominate) then perhaps we should optimize for wireheading, or whatever that utopia would look like.
More likely, this is an overly simplified model, and there are other concerns that I'm not taking into account but which may dominate the calculation. I completely missed the libertarian perspective, after all.
Sounds like WoW is optimized for System 1 pleasures, and you explicitly reject this. I think that brings up an important point: How can we build a society/world where there are strong optimization forces to enable people to choose System 2 preferences? Once such a world iterated on itself for a couple generations, what might it look like?
I don’t think this would be a world with no WoW-like activities, because a world without any candy or simple pleasures strikes me as deeply lacking. My System 2 seems to place at least a little value on System 1 being happy. So I’d guess the world would just have many fewer of such activities, and be structured in such a way as to make it easy to avoid choices we’d regret the next day.
If this turns out to a physically impossible problem to overcome for some reason, then I could imagine a world with no System 1 pleasures, but such a world would be deeply lacking, even if that loss was more than made up for by gains in our System 2 values.
As a side note, it'd be an interesting question how much of the theoretical per capita maximum value falls into which categories. An easier question is how much of our currently actualized value is immediate gratifications. I'd expect that to be heavily biased toward System 1, since we suffer from Akrasia, but it might still be informative.
I've recently started using RSS feeds. Does anyone have LW-related feeds they'd recommend? Or for that matter, anything they'd recommend following which doesn't have an RSS feed?
Here's my short list so far, in case anyone else is interested:
Less Wrong Discussion
Less Wrong Main (ie promoted)
Slate Star Codex
Center for the Study of Existential Risk
Future of Life Institute [they have a RSS button, but it appears to be broken. They just updated their webpage, so I'll subscribe once there's something to subscribe to.]
Global Priorities Project
80,000 Hours
SpaceX [an aerospace company, which Elon Musk refuses to take public until they've started a Mars colony]
These obviously have an xrisk focus, but feel free to share anything you think any Less-Wrongers might be interested in, even if it doesn't sound like I would be.
For anyone looking to start using RSS, I'd recommend using the Bamboo Feed Reader extension in FireFox, and deleting all the default feeds. I started out using Sage as a feed aggregator, but didn't like the sidebar style or the tiled reader.
Ah, thanks for the explanation. I interpreted the statement as you trying to demonstrate that number of nuclear winters / number of near misses = 1/100. You are actually asserting this instead, and using the statement to justify ignoring other categories of near misses, since the largest will dominate. That's a completely reasonable approach.
I really wish there was a good way to estimate the accidents per near miss ratio. Maybe medical mistakes? They have drastic consequences if you mess up, but involve a lot of routine paperwork. But this assumes that the dominant factors in the ratio are severity of consequences. (Probably a reasonable assumption. Spikes on steering wheels make better drivers, and bumpers make less careful forklift operators.) I'll look into this when I get a chance.
Excellent start and setup, but I diverge from your line of thought here:
We will use a lower estimate of 1 in 100 for the ratio of near-miss to real case, because the type of phenomena for which the level of near-miss is very high will dominate the probability landscape. (For example, if an epidemic is catastrophic in 1 to 1000 cases, and for nuclear disasters the ratio is 1 to 100, the near miss in the nuclear field will dominate).
I'm not sure I buy this. We have two types of near misses (biological and nuclear). Suppose we construct some probability distribution for near-misses, ramping up around 1/100 and ramping back down at 1/1000. That's what we have to assume for any near-miss scenario, if we know nothing additional. I'll grant that if we roll the dice enough times, the 1/100 cases will start to dominate, but we only have 2 categories of near misses. That doesn't seem like enough to let us assume a 1/100 ratio of catastrophes to near misses.
Additionally, there does seem to be good reason to believe that the rate of near misses has gone down since the cold war ended. (Although if any happened, they'd likely still be classified.) That's not to say that our current low rate is a good indicator, either. I would expect our probability of catastrophe to be dominated by the probability of WWIII or another cold war.
We had 2 world wars in the first 50 years of last century, before nuclear deterrence substantially lowered the probability of a third. If that's a 10x reduction, then we can expect 0.4 a century instead of 4 a century. If there's a 100x reduction, then we might expect 0.04 world wars a century. Multiply that by the probability of nuclear winter given WWIII to get the probability of disaster.
However, I suspect that another cold war is more likely. We spent ~44 of the past 70 years in a cold war. If that's more or less standard, then on average we might expect to spend 63% of any given century in a cold war. This can give us a rough range of probabilities of armageddon:
1 near miss a year spent in cold war 63 years spent in cold war per century 1 nuclear winter per 100 near misses = 63% chance of nuclear winter per century
0.1 near miss a year spent in cold war 63 years spent in cold war per century 1 nuclear winter per 3000 near misses = 0.21% chance of nuclear winter per century
For the record, this range corresponds to a projected half life between roughly 1 century and ~100 centuries. That's much more broad then your 50-100 year prediction. I'm not even sure where to start to guesstimate the risk of an engineered pandemic.
The problem here of course is how selective to be about rules to let into this protected level
Couldn't this be determined experimentally? Ignore the last hundred years or so, or however much might influence our conclusion based on modern politics. Find a list of the people who had a large counterfactual impact on history. Which rules lead to desirable results?
For example, the trial of Socrates made him a martyr, significantly advancing his ideas. That's a couple points for "die for the principle of the matter" as an ethical injunction. After Alexander the great died, anti-Macedonian sentiment in Athens caused Aristotle to flee, saying "I will not allow the Athenians to sin twice against philosophy". Given this, perhaps Socrates's sacrifice didn't achieve as much as one might think, and we should update a bit in the opposite direction. Then again, Aristotle died a year later, having accomplished nothing noteworthy in that time.
All the happiness that the warm thought of an afterlife ever produced in humanity, has now been more than cancelled by the failure of humanity to institute systematic cryonic preservations after liquid nitrogen became cheap to manufacture. And I don't think that anyone ever had that sort of failure in mind as a possible blowup, when they said, "But we need religious beliefs to cushion the fear of death." That's what black swan bets are all about—the unexpected blowup.
That's a fantastic quote.
Today, October 27th, is the 53rd anniversary of the day Vasili Arkhipov saved the world. I realize Petrov Day was only a month ago, and there was a post then. Although I appreciate our Petrov ceremony, I personally think Arkhipov had a larger counterfactual impact than Petrov, (since nukes might not have been launched even if Petrov hadn't been on shift at the time) and so I'd like to remember Vasili Arkhipov as well.
Donald E. Brown's list of human universals is a list of psychological properties which are found so commonly that anthropologists don't report them.
I've looked for that link before, and couldn't find it. It's closely related to Moral Foundations Theory, which is basically 6 categories for features of morality which are found in every culture.
Interesting point. It seems like human morality is more than just a function which maximizes human prosperity, or minimizes human deaths. It is a function which takes a LOT more into account than simply how many people die.
However, it does take into account its own biases, at least when it finds them displeasing, and corrects for them. When it thinks it has made an error, it corrects the part of the function which produced that error. For example, we might learn new things about game theory, or even switch from a deontological ethical framework to a utilitarian one.
So, the meta-level question is which of our moral intuitions are relevant to the trolley problem. (or more generally, what moral framework is correct.) If human deaths can be shown to be much more morally important than other factors, then the good of the many outweighs the good of the few. If, however, deontological ethics is correct, then the ends don't justify the means.
Individual wealth has diminishing returns on investment. The marginal utility of each extra dollar of income is less. There's reason to believe that we'll have to slowly shift the focus of our efforts elseware, if we want to continue making equally huge strides forward.
We hit the UN's old goal of having extreme poverty level from 1990. We even did it 5 years ahead of the 2015 target date, which is fantastic. But if we want to hit the next set of goals, we'll need more than just more economic growth. For example, this TED talk indicates that all of the UN's Global Goals can be expressed roughly as an increase in global Social Progress Index from 61 to ~75. However, if we rely entirely on continued economic growth and don't have any social change, then he claims we will only move from 61 to ~62.4.
As an asside, I find the Social Progress Index to be an interesting metric. It's an equally weighted composit of "Basic Human Needs" (such as nutrition and basic medicine), "Foundations of Wellbeing" (such as access to education and information), and "Opportunit" (such as personal rights and tollerence).
Link to the sleep advice: http://lesswrong.com/lw/mvf/a_very_long_list_of_sleep_maintenance_suggestions/
Another way of saying "below 1/100,000 chance of aliens" is "above 99.999% chance of natural causes". That seams awefully certain of the unlikelyness of aliens. I'm pretty sure it's not aliens, but I'm not that confident. I'd happily lose a dollar in that bet, if someone wanted to wager $100,000 against it.
I'd be interested in leaving a will to an existential risk related group, although not necessarily MIRI. The Future of Humanity Institute currently looks like one of the best options, although I haven't researched this exhaustively. (They get most of their funding from academic research grants, but more flexible donations allow them to focus on the activities they think are most important.)
My current thought is to download a will template from docracy.com and just update it every year or so until I settle on one organization. Most of those wills seem to be built around leaving money to family, so I'm not confident they are optimized for tax efficient charitable donations. Does anyone have any better ideas, or do I need to get researching on this one?
After the first three words, I assumed you were going to point out that the entire planet gets much hotter and colder as the zodiac shifts each year. The rising and setting of the sun has a similarly large effect, and the moon is also connected with tides.
Historical astrology was the precursor to modern astronomy, and was highly complex. There was likely a relatively strong tie to the course of history. If an astrological event associated with regime changes occurs, people are much more likely to revolt, because they are much less worried about being punished for a failed revolt.
Perhaps this did generate some traffic, but LessWrong doesn't have adds. And any publicity this generated was bad publicity, since Roko's argument was far too weird to be taken seriously by almost anyone.
It doesn't look like anyone benefited. Eliezer made an ass of himself. I would guess that he was rather rushed at the time.
Thanks for the link to Harold Hering's article. I just read it, as well as the one on Stanislav Petrov.
Vasili Arkhipov's Wikipedia article is also worth reading. Although most Russian nuclear submarines required only the captain's order to launch, he was the only one of the three officers on his sub to vote against launching their nukes during the Cuban Missile Crisis. Due to his position as Flotilla commander, he was able to win the argument with the submarine's captain, who wanted to launch.
the difference between "because it is the human one" and "because it is h-right" sounds a lot less convincing.
If I see a toddler in the path of a boulder rolling downhill, I don't ask myself "should I help the bolder, or the toddler?" and conclude "the toddler, because it is the human one."
If I were to even pause and ask myself a question, it would be "what should I do?" and the answer would be "save the toddler, because it is h-right".
Perhaps h-right is "just the human perspective", but that's not the reason I save the toddler. Similarly, the bolder rolls downhill because F=G(m1m2)/r^2, not because it is what boulders do. It is what boulders do, but that's different from the question of why they do what they do.