Open Thread June 2010, Part 3

Interesting. I suspect that sick systems are actually highly competitively-fit, and while people who opt-out of them may be happier, those people will propagate themselves less, and therefore will be overwhelmed by Azathothian forces.

Is there any way to combat Azathoth aside from forming a singleton?

My wealth has come from a combination of living in America, some lucky genes, and compound interest. Both my children and I won what I call the ovarian lottery. (For starters, the odds against my 1930 birth taking place in the U.S. were at least 30 to 1. My being male and white also removed huge obstacles that a majority of Americans then faced.)

My luck was accentuated by my living in a market system that sometimes produces distorted results, though overall it serves our country well. I've worked in an economy that rewards someone who saves the lives of others on a battlefield with a medal, rewards a great teacher with thank-you notes from parents, but rewards those who can detect the mispricing of securities with sums reaching into the billions. In short, fate's distribution of long straws is wildly capricious.

Wow. That is some seriously clear thinking. Too bad Mr. Buffet isn't here to get the upvote himself, so I upvoted you instead. ;-)

At the same time the degree of confidence that many posters have about their beliefs in the significance of Less Wrong and SIAI is unsettling to me. A number of posters write as though they're sure that what Less Wrong and SIAI are doing are the most important things that any human could be doing. It seems very likely to me that what Less Wrong and SIAI are doing is not as nearly important (relative to other things) as such posters believe

I feel like an elephant in the room is the question of whether the reason that those of you who believe that Less Wrong and SIAI doing things of the highest level of importance is because you're a part of these groups (*).

You know what... I'm going to come right out and say it.

A lot of people need their clergy. And after a decade of denial, I'm finally willing to admit it - I am one of those people.

The vast majority of people do not give their 10% tithe to their church because some rule in some "holy" book demands it. They don't do it because they want a reward in heaven, or to avoid hell, or because their utility function assigns all such donated dollars 1.34 points of utility up to 10% of gross income.

They do it because they want their priests to kick more ass than the OTHER group's priests. OUR priests have more money, more power, and more intellect and YOUR sorry-ass excuse for a holy-man. "My priest bad, cures cancer and mends bones; your priest weak, tell your priest to go home!"

So when I give money to the SIAI (or FHI or similar causes) I don't do it because I necessarily think it's the best/most important possible use of my fungible resources. I do it because I believe Eliezer & Co are the most like-me actors out there who can influence the future. I do it because of all the people out there with the ability to alter the flow of future events, their utility function is the closest to my own, and I don't have the time/energy/talent to pursue my own interests directly. I want the future to look more like me, but I also want enough excess time/money to get hammered on the weekends while holding down an easy accounting job.

In short - I want to be able to just give a portion of my income to people I trust to be enough like me that they will further my goals simply by pursuing their own interests. Which is to say: I want to support my priests.

And my priests are Eliezer Yudkowsky and the SIAI fellows. I don't believe they leach off of me, I feel they earn every bit of respect and funding they get. But that's besides the point. The point is that even if the funds I gave were spent sub-optimally, I would STILL give them this money, simply because I want other people to see that MY priests are better taken care of than THEIR priests.

The vatican isn't made out of gold because the pope is greedy, it's made out of gold because the peasants demand that it be so. And frankly, I demand that the vatican be put to fucking shame when it compares itself us.

Standard Disclaimer, but really... some enthusiasm is needed to fight Azathoth.

Comment on markup: I saw the first version of your comment, where you were using "(*)" as a textual marker, and I see you're now using "#" because the asterisks were messing with the markup. You should be able to get the "(*)" marker to work by putting a backslash before the asterisk (and I preferred the "(*)" indicator because that's more easily recognized as a footnote-style marker).

Feels weird to post an entire paragraph just to nitpick someone's markup, so here's an actual comment!

From what I've read (several hundred assorted threads), I feel like an elephant in the room is the question of whether the reason that those of you who believe that Less Wrong and SIAI doing things of the highest level of importance is because you're a part of these groups

Let me try and rephrase this in a way that might be more testable/easier to think about. It sounds like the question here is what is causing the correlation between being a member of LW/SIAI and agreeing with LW/SIAI that future AI is one of the most important things to worry about. There are several possible causes:

1. group membership causes group agreement (agreement with the group)
2. group agreement causes group membership
3. group membership and group agreement have a common cause (or, more generally, there's a network of causal factors that connect group membership with group agreement)
4. a mix of the above

And we want to know whether #1 is strong enough that we're drifting towards a cult attractor or some other groupthink attractor.

I'm not instantly sure how to answer this, but I thought it might help to rephrase this more explicitly in terms of causal inference.

I'm not aware of anyone here who would claim that LW is one of the most important things in the world right now but I think a lot of people here would agree that improving human reasoning is important if we can have those improvements apply to lots of different people across many different fields.

There is a definite group of people here who think that SIAI is really important. If one thinks that a near Singularity is a likely event then this attitude makes some sense. It makes a lot of sense if you assign a high probability to a Singularity in the near future and also assign a high probability to the possibility that many Singularitarians either have no idea what they are doing or are dangerously wrong. I agree with you that the SIAI is not that important. In particular, I think that a Singularity is not a likely event for the foreseeable future, although I agree with the general consensus here that a large fraction of Singularity proponents are extremely wrong at multiple levels.

Keep in mind that for any organization or goal, the people you hear the most about it are the people who think that it is important. That's the same reason that a lot of the general public thinks that tokamak fusion reactors will be practical in the next fifty years: The physicists and engineers who think that are going to loudly push for funding. The ones who don't are going to generally just go and do something else. Thus, in any given setting it can be difficult to estimate the general communal attitude towards something since the strongest views will be the views that are most apparent.

#: I feel like an elephant in the room is the question of whether the reason that those of you who believe that Less Wrong and SIAI doing things of the highest level of importance is because you're a part of these groups.

Even if the most ardent Less Wrong and SIAI supporters are mostly right about their beliefs, # is almost certainly at least occasionally present and I think that the community would benefit from a higher level of vigilance concerning the possibility #.

# refers to a pattern of incorrect (intuitive) reasoning. This pattern is potentially dangerous specifically because it leads to incorrect beliefs. But if you are saying that there is no significant distortion in beliefs (in particular about the importance of Less Wrong or SIAI's missions*), doesn't this imply that the role of this potential bias is therefore unimportant? Either # isn't important, because it doesn't significantly distort beliefs, or it does significantly distort beliefs and therefore important.

* Although I should note that I don't remember there being a visible position about the importance of Less Wrong.

There are a few scattered posts in Eliezer's sequences which do not, I believe, have strong dependencies (I steal several from the About page, others from Kaj_Sotala's first and second lists) - I separate out the ones which seem like good introductory posts specifically, with a separate list of others I considered but do not think are specifically introductory.

Introductions:

• Absence of Evidence is Evidence of Absence
• Feeling Rational
• Taboo Your Words
• Back Up and Ask Whether, Not Why
• Expecting Short Inferential Distances
• Knowing About Biases Can Hurt People
• Burdensome Details
• The Proper Use of Humility
• Mysterious Answers to Mysterious Questions
• The Bottom Line

Not introductions, but accessible and cool:

• Outside the Laboratory
• Beyond the Reach of God
• Conjunction Controversy

That is usually called "collective intelligence":

http://en.wikipedia.org/wiki/Collective_intelligence

Calling it "synthetic Intelligence" would be bad, IMO.

Could you expand on what would be included and excluded from Synthetic Intelligence?

Would a free market count?

It can mostly be ignored, but uFAI affects physically-nearby aliens who might have developed a half-Friendly AI otherwise. (But if they could have, then they have counterfactual leverage in trading with your uFAI.) No reason to suspect that those aliens had a much better shot than we did at creating FAI, though. Creating uFAI might also benefit the aliens for other reasons... that I won't go into, so instead I will just say that it is easy to miss important factors when thinking about these things. Anyway, if the nanobots are swarming the Earth, then launching uFAI does indeed seem very reasonable for many reasons.

That is, if you have an option of trading Doom for UFAI, while forsaking only negligible probability of FAI, you should take it.

Fascinating! Do you still agree with what you wrote there? Are you still researching this issues and do you plan on writing a progress report or an open problems post? Would you be willing to write a survey paper on decision theoretic issues related to acausal trade?

I tend to agree that we frequently would do better to make do with an atlas of charts rather than seeking the One True Map. But I'm not sure I like the differential geometry metaphor. It is not the location on the globe which makes use of one chart more fruitful than another. It is the question of scale, or as a computer nerd might express it, how zoomed in you are. And I would prefer to speak of different models rather than different maps.

For example, at one level of zoom, we see the universe as non-deterministic due to QM. Zoom out a bit and you have billiard-ball atoms in a Newtonian billiard room. Zoom out a bit more and find non-deterministic fluctuations. Out a bit more and you have deterministic chemical thermodynamics (unless you are dealing with a Brusselator or some such).

But I would go farther than this. I would also claim that we shouldn't imagine that these maps (as you zoom in) necessarily become better and better maps of the One True Territory. We should remain open to the idea that "It's maps (or models, or turtles) all the way down".

But I wonder if we are too concerned with the One True Map as opposed to an atlas of pretty good maps.

What's an example of people doing this?

You might be interested in How to Lie with Maps.

I'm not sure I see what the problem is. Capitalism works? It makes it seem like this system is unsustainable or bound to collapse, but I'm not sure I see how two and two fit together. I am particularly confused with this quote:

Obviously, the “pyramid paradigm can’t continue forever,” says Susan Gerbi, chair of molecular biology at Brown University and one of the relatively small number of scientists who have expressed serious concern about the situation. Like any Ponzi scheme, she fears, this one will collapse when it runs out of suckers — a stage that appears to be approaching. “We need to have solutions for some new steady-state model” that will limit the production of new scientists and offer them better career prospects, she adds.

First of all, how is it a ponzi scheme that is bound to collapse? Also limiting the number of scientists is not going to make the system better, except that maybe individuals will have less competition = more opportunities, which is not a benefit to the whole system, just to the individual.

EDIT: Fixed spelling.

Two counters to the majoritarian argument:

First, it is being mentioned in the mainstream - there was a New York Times article about it recently.

Secondly, I can think of another monumental, civilisation-filtering event that took a long time to enter mainstream thought - nuclear war. I've been reading Bertrand Russel's autobiography recently, and am up to the point where he begins campaigning against the possibility of nuclear destruction. In 1948 he made a speech to the House of Lords (UK's upper chamber), explaining that more and more nations would attempt to acquire nuclear weapons, until mutual annihilation seemed certain. His fellow Lords agreed with this, but believed the matter to be a problem for their grandchildren.

Looking back even further, for decades after the concept of a nuclear bomb was first formulated, the possibility of nuclear was was only seriously discussed amongst physicists.

I think your second point is stronger. However, I don't think a single AI rewiring itself is the only way it can go FOOM. Assume the AI is as intelligent as a human; put it on faster hardware (or let it design its own faster hardware) and you've got something that's like a human brain, but faster. Let it replicate itself, and you've got the equivalent of a team of humans, but which have the advantages of shared memory and instantaneous communication.

Now, if humans can design an AI, surely a team 1,000,000 human equivalents running 1000x faster can design an improved AI?

The information content of a mind cannot exceed the amount of information necessary to specify a representation of that same mind.

If your argument is based on information capacity alone, it can be knocked down pretty easily. An AI can understand some small part of its design and improve that, then pick another part and improve that, etc. For example, if the AI is a computer program, it has a sure-fire way of improving itself without completely understanding its own design: build faster processors. Alternatively you could imagine a population of a million identical AIs working together on the problem of improving their common design. After all, humans can build aircraft carriers that are too complex to be understood by any single human. Actually I think today's humanity is pretty close to understanding the human mind well enough to improve it.

The information content of a mind cannot exceed the amount of information necessary to specify a representation of that same mind. Therefore, while the AI can understand in principle that it is made up of transistors etc., its self-representation necessary has some blank areas.

This is strictly true if you're talking about the working memory that is part of a complete model of your "mind". But a mind can access an unbounded amount of externally stored data, where a complete self-representation can be stored.

A Turing Machine of size N can run on an unbounded-size tape. A von Neumann PC with limited main memory can access an unbounded-size disk.

Although we can only load a part of the data into working memory at a time, we can use virtual memory to run any algorithm written in terms of the data as a whole. If we had an AI program, we could run it on today's PCs and while we could run out of disk space, we couldn't run out of RAM.

I'd just forget the majoritarian argument altogether, it's a distraction.

The second question does seem important to me, I too am skeptical that an AI would "obviously" have the capacity to recursively self-improve.

The counter-argument is summarized here, whereas we humans are stuck with an implementation substrate which was never designed for understandability, an AI could be endowed with both a more manageable internal representation of its own capacities and a specifically designed capacity for self-modification.

It's possible - and I find it intuitively plausible - that there is some inherent general limit to a mind's capacity for self-knowledge, self-understanding and self-modification. But an intuition isn't an argument.

Stephen Hawking, Martin Rees, Max Tegmark, Nick Bostrom, Michio Kaku, David Chalmers and Robin Hanson are all smart people who broadly agree that >human AI in the next 50-100 years is reasonably likely (they'd all give p > 10% to that with the possible exception of Rees). On the con side, who do we have? To my knowledge, no one of similarly high academic rank has come out with a negative prediction.

Edit: See Carl's comment below. Arguing majoritarianism against a significant chance of AI this century is becoming less tenable, as a significant set of experts come down on the "yes" side.

It is notable that I can't think of any very reputable nos. The ones that come to mind are Jaron Lanier and that Glenn Zorpette.

Re: "can a mind understand itself?"

That is no big deal: copy the mind a few billion times, and then it will probably collectively manage to grok its construction plans well enough.

Another argument against the difficulties of self-modeling point: It's possible to become more capable by having better theories rather than by having a complete model, and the former is probably more common.

It could notice inefficiencies in its own functioning, check to see if the inefficiencies are serving any purpose, and clean them up without having a complete model of itself.

Suppose a self-improving AI is too cautious to go mucking about in its own programming, and too ethical to muck about in the programming of duplicates of itself. It still isn't trapped at its current level, even aside from the reasonable approach of improving its hardware, though that may be a more subtle problem than generally assumed.

What if it just works on having a better understanding of math, logic, and probability?

In addition to theoretical objections, I think the majoritarian argument is factually wrong. Remember, 'future is here, just not evenly distributed'.

http://www.google.com/trends?q=singularity shows a trend

http://www.nytimes.com/2010/06/13/business/13sing.html?pagewanted=all - this week in NYT. Major MSFT and GOOG involvement.

http://www.acceleratingfuture.com/michael/blog/2010/04/transhumanism-has-already-won/

Of course, it could just add complexity and hope that it works, but that’s just evolution, not intelligence explosion.

The critical aspect of a "major-impact intelligence-explosion singularity" isn't the method for improvement but the rate of improvement. If computer processing power continues to grow at an exponential rate, even an inefficiently improving AI will have the growth in raw computing power behind it.

So: do you know any counterarguments or articles that address either of these points?

I don't have any articles but I'll take a stab at counterarguments.

A Majoritarian counterargument: AI turned out to be harder and further away than originally thought. The general view is still tempered by the failure of AI to live up to those expectations. In short, the AI researchers cried "wolf!" too much 30 years ago and now their predictions aren't given much weight because of that bad track record.

A mind can't understand itself counterargument: Even accepting as a premise that a mind can't completely understand itself, that's not an argument that it can't understand itself better than it currently does. The question then becomes which parts of the AI mind are important for reasoning/intelligence and can an AI understand and improve that capability at a faster rate than humans.

Re: "If nothing else, you might think that someone could exaggerate the threat of the singularity and use it to scare people into giving them government funds. But we don’t even see that happening."

? I see plenty of scaremongering around machine intelligence. So far, few governments have supported it - which seems fairly sensible of them.

How do we know that governments aren't secretly working on AI?

Is it worth speculating about the goals which would be built into a government-designed AI?

Regarding majoritarianism:

imminent, so important, and furthermore so sensitive to initial conditions To a risk-neutral consequentialist or to an elected or corporate official?

Crash programs in basic science because of speculative applications are very uncommon. Decades of experimentation with nuclear fission only brought a crash program with the looming threat of the Nazis, and after a practical demonstration of a chain reaction.

Over the short time spans over which governments make their plans, the probability of big advances in AI basic science coming is relatively small, even if substantially over the longer term. So you get all the usual issues with attending to improbable (in any given short period) dangers that no one has recent experience with. Note things like hurricane Katrina, the Gulf oil spill, etc. The global warming effects of fossil fuel use have been seen as theoretically inevitable since at least the Eisenhower administration, and momentum for action has only gotten mobilized after a long period of actual warming providing pretty irrefutable (and yet widely rejected anyway!). evidence.

Thanks a lot for the link. I remember Eliezer arguing with Robin whether AI will advance explosively by using few big insights, or incrementally by amassing encoded knowledge and many small insights. Watson seems to constitute evidence in favor of Robin's position as it has no single key insight:

Ferrucci says his team will continue to fine-tune Watson, but improving its performance is getting harder. “When we first started, we’d add a new algorithm and it would improve the performance by 10 percent, 15 percent,” he says. “Now it’ll be like half a percent is a good improvement.”

Direct link to printable (readable) single-page version of the article.

Yes, in that it could be open-ended in any "direction" independent of the delusion. However, that might require contrived initial conditions or cognitive architecture. You might also find the delusion becoming neutralized for all practical purposes, e.g. the delusional proposition is held to be true in "real reality" but all actual actions and decisions pertain to some "lesser reality", which turns out to be empirical reality.

ETA: Harder question: are there thinking systems which can know that they aren't bounded in such a way?

Funny, I've been entertaining the same idea for a few weeks.

Every time I read statements like "... and then I update the probabilities, based on this evidence ...", I think to myself: "I wish I had the time (or processing power) he thinks he has. ;)"

Sure

Thanks for the link.

If a machine could write a Mozart sonata every bit as good as the originals, then what was so special about Mozart?

Mozart developed the Mozart sonata.

Great article. Thanks for the link!

Good music isn't about good music. It's about which music authorities have approved of it.

Wade's breakthrough came after his real-life child was born. The duties of fatherhood limited the time he could spend playing the game, so he replaced the "computer" with a much simpler pattern called an "instruction tape", made up of smaller patterns known as "gliders". By placing these at precise intervals, he created a program that feeds into the constructor and dictates its actions, much like the punched rolls of tape once used to control the first computers.

One of Eliezer's posts talks about realizing that conventional science is content with an intolerably slow pace. Here we have an example of less time leading to a better solution.

Apparently it doesn't replicate itself any more than a glider does; the old copy is destroyed as it creates a new copy.

Now I'm wondering what screen resolution and how many potions of longevity would be required to evolve intelligent life while playing ADOM.

Note that this sort of thing has been done a bit before. See for example this analysis.

Edit: The linked analysis has a lot of problems. See discussion below.

If I moved into that apartment instead, would that help or hurt the country's economy as a whole?

Good question, not because it's hard to answer, but because of how pervasive the wrong answer is, and the implications for policy for economists getting it wrong.

• If your parents prefer you being in their apartment to the forgone income, they benefit; otherwise they don't.

• If you prefer being in their apartment to the alternative rental opportunities, you benefit; otherwise, you don't.

• If potential renters or the existing ones prefer your parents' unit to the other rental opportunities and they are denied it, they are worse off; otherwise, they aren't.

ANYTHING beyond that -- anything whatsoever -- is Goodhart-laden economist bullsh**. Things like GDP and employment and CPI were picked long ago as a good correlate of general economic health. Today, they are taken to define economic health, irrespective of how well people's wants are being satisfied, which is supposed to be what we mean by a "good economy".

Today, economists equate growing GDP -- irrespective of measuring artifacts that make it deviate from what we want it to measure -- with a good economy. If the economy isn't doing well enough, well, we need more "aggregate demand" -- you see, people aren't buying enough things, which must be bad.

Never once has it occurred to anyone in the mainstream (and very few outside of the mainstream) that it's okay for people to produce less, consume less, and have more leisure. No, instead, we have come to define success by the number of money-based market exchanges, rather than whether people are getting the combination of work, consumption, and leisure (all broadly defined) that they want.

This absurdity reveals itself when you see economists scratching their heads, thinking how we can get people to spend more than they want to, in order to help the economy. Unpack those terms: they want people to hurt themselves, in order to hurt less.

Now, it's true there are prisoner's dilemma-type situations where people have to cooperate and endure some pain to be better off in the aggregate. But the corresponding benefit that economists expect from this collective sacrifice is ... um ... more pointless work that doesn't satisfy real demand .. but hey, it keeps up "aggregate demand", so it must be what a sluggish economy needs.

Are you starting to see how skewed the standard paradigm is? If people found a more efficient, mutualist way to care for their children rather than make cash payments to day care, this would be regarded as a GDP contraction -- despite most people being made better off and efficiency improving. If people work longer hours than they'd like, to produce stuff no one wants, well, that shows up as more GDP, and it's therefore "good".

How the **** did we get into this mindset?

Sorry, [/another rant].

An interesting question. Here are some initial thoughts:

In terms of broad economic aggregates, it won't make any difference. If you rent the room off your parents for a market rate, GDP is exactly unaffected, people are paying the same money to different people. If you rent it for less than market rate, GDP is lower, but this reflects deficiencies in measured GDP since GDP uses market prices as a proxy for the value of a transaction (this is fine for the most part, but doing your child a favour is an exception conventional methodology can't deal with). So from a macroeconomic perspective I'd say it's a wash either way.

Microeconomically, there could be some efficiencies in you renting from your parents. If they trust you more than a random stranger (and let's hope they do) they will spend less time monitoring your behaviour (property inspections and the like) than they would a random stranger, but the value of your familial relationship should constrain you from taking advantage of that lax monitoring in the way a stranger would. This mean that your parents save time (which makes their life easier) and no one should be worse off (I assume the current tenant of their room would find adequate accommodation elsewhere).

However, one note of caution. If you were to get into a dispute of some sort with your parents over the tenancy, this could damage your relationship with your parents. If you value this relationship (and I assume you do), this is a potential downside that doesn't exist under the status quo. Also, some people might see renting from your parents as little different to living with your parents which (depending on your age) may cost you status in your day-to-day life (even if you pay a market rate). If you value status, you should be aware of this drawback.

So in summary, the most efficient outcome depends on three variables: 1) How much time and effort do your parents spend monitoring their tenant at the moment? 2) How likely is it that your relationship with them could be strained as a result of you living there? 3) How many friends / acquaintances / colleagues do you have that would think less of your for renting from your parents (and how much do you care)?

I hope that helps.

I think that a majority of economists agree that in many downturns, it helps the economy if people, on the margin, spend a little more. This justifies Keynesian stimulus. Therefore, the economy would be helped if your choice increases the total amount of money changing hands, presumably if you rent the apartment for $X when X>Y. My impression is that in good economic times, marginal spending is not considered to improve economic welfare.

Coming back to this question after a few years, I was able to find a surprisingly simple Econ 101 answer in five minutes. To zeroth order, there's no change because the amount of goods and services in the economy stays the same. To first order, allowing a deal to be freely made usually increases total value in the economy, not just the value for those making the deal; so this deal is good for the economy iff both sides agree to it.

That sidesteps all complications like "the parents are happy to help their child", "the apartment might have facilities that the child doesn't need", etc. I guess reading an econ textbook has taught me to look for ways to estimate the total without splitting it up.

Here's another question to chew on:

Suppose you're in a country that grows and consumes lots of cabbages, and all the cabbages consumed are home-grown. Suppose that one year people suddenly, for no apparent reason, decide that they like cabbages a lot more than they used to, and the price doubles. But at least to begin with, rates of production remain the same throughout the economy. Does this help or harm the economy, or have no effect?

In one sense it 'obviously' has no effect, because the same quantities of all goods and services are produced 'before' and 'afterwards'. So whether we're evaluating them according to the 'earlier' or the 'later' utility function, the total value of what we're producing hasn't changed. (Presumably the prices of non-cabbages would decline to some extent, so it's at least consistent that GDP wouldn't change, though I still can't see anything resembling a mathematical proof that it wouldn't.)

would that help or hurt the country's economy as a whole?

What exact metric do you have in mind?

I recommend going to an econ textbook for good questions.

My (admittedly simple-minded) answer would be "other things being equal it has no effect at all".

Each day you and your parents do whatever it is you do, creating a given amount of wealth (albeit perhaps in such a way that it's impossible to say exactly how much of this wealth you personally created, rather than your colleagues, or the equipment you use). Then a bunch of wealth gets redistributed in a funny way (through wages and rents being paid). But changing the way that wealth is redistributed doesn't affect the 'total rate of wealth-generation' which is what GDP is trying (sometimes unsuccessfully, as James_K says) to measure. In just the same way, getting a pay rise doesn't in itself help the economy (but it may have been caused by you doing more valuable work, which does help).

There is other information you want to consider. Tax rates for example, and whether or not the economy is in the sort of downturn that would benefit from stimulus or not.

Regardless, the effects on aggregate supply and demand will be tiny. How much you and your parents value these alternatives is what matters most.

which great things LWers might have tried and discarded for no particular reason.

That's kind of hard - if they were so great, how could we remember they were great and also not immediately reinstate them?

Gawande on the need to develop competent systems for delivering medical care

(Closing parenthesis.)

I was wondering indeed. That was surreal.

Such plans work in societies with rule of law, and fail miserably in societies that are clan based and tribal. A quarter of Afghanistan's GDP may go to bribes and shakedowns. A more honest description from NPR would be that historically, mineral wealth when controlled by deeply corrupt governments like Afghanistan's, is primarily used for graft and nepotism, benefiting a few elites in government and industry while funding the oppression of everyone else.

In other words, Afghanistan is more like Nigeria than Norway.

I'm currently reading Thomas Schelling's Strategy of Conflict and it sounds like what you're looking for here. From this Google Books Link to the table of contents you can sample some chapters.

"I know [he was involved] because my brother confessed to me that he had killed Meredith and he asked me to hide a blood-stained knife and set of keys," he said, according to an attachment to Knox's appeal documents.

"I had everything under a little wall behind my house," he said. "I am happy to stand up in court and confirm all this and wrote to the court several times to tell them but was never questioned."

Should be easy to test his claims...

We "can't simply investigate in the course of a trial every claim that comes up," Mignini told CNN.

I sometimes wonder, is the Italian judicial system really that lousy or is there some sort of linguistic or cultural barrier there.

Slashdot threads have a bad enough signal to noise ratio as is. Please don't do that sort of thing.

You were arguing against your real opinion as a 5th columner? May I ask why?

(Well done, by the way, in a technical sense. Just the right amount of character assassination: "Sollecito and Knox were known to be practitioners of dangerous sex acts.")

Just don't kill the younglings, Anakin!

FYI, this was discussed previously here

Calling them "dark arts" is itself a tactic for framing that only affects the less-rational parts of our judgement.

A purely rational agent will (the word "should" isn't necessary here) of course use rhetoric, outright lies, and other manipulations to get irrational agents to behave in ways that further it's goals.

The question gets difficult when there are no rational agents involved. Humans, for instance, even those who want to be rational most of the time, are very bad at judging when they're wrong. For these irrational agents, it is good general advice not to lie or mislead anyone, at least if you have any significant uncertainty on the relative correctness of your positions on the given topic.

Put another way, persistent disagreement indicates mutual contempt for each others' rationality. If the disagreement is resolvable, you don't need the dark arts. If you're considering the dark arts, it's purely out of contempt.

Dark arts, huh? Sometime ago I put forward the following scenario:

Bob wants to kill a kitten. The FAI wants to save the kitten because it's a good thing according to our CEV. So the FAI threatens Bob with 50 years of torture unless Bob lets the kitten go. The FAI has two distinct reasons why threatening Bob is okay: a) Bob will comply and there will be no need to torture him, b) the FAI is lying anyway. Expected utility reasoning says the FAI is doing the Right Thing. But do we want that?

(Yes, this is yet another riff on consequentialism, deontologism and lying. Should FAIs follow deontological rules? For that matter, should humans?)

Lately I've been wondering if a rational agent can be expected to use the dark arts when dealing with irrational agents.

Yes.

For example: if a rational AI (not necessarily FAI) had to convince a human to cooperate with it, would it use rhetoric to leverage the human biases against it?

Yes. (When we say 'rational agent' or 'rational AI' we are usually referring to "instrumental rationality". To a rational agent words are simply symbols to use to manipulate the environment. Speaking the truth, and even believing the truth are only loosely related concepts.

Would a FAI?

Almost certainly, but this may depend somewhat on who exactly it is 'friendly' to and what that person's preferences happen to be.

Sounds a little like Shalmaneser.

And now it's time for the Daily Double!

In the video, I didn't understand whether that series of wrong answers was staged or actually happened.

Very impressive though. Class.

The first one fails terribly. I've had dreams where I've thought I've proven some statement I'm thinking about and when waking up can remember most of the "proof" and it is clearly incoherent. No, subconscious, the fact that Martin van Buren was the 8th President of the United States does not tell me anything about zeros of L-functions. (I've had other proofs that were valid though so I don't want the subconscious to stop working completely).

The second one seems more viable. May I suggest using something like electromagnetic stimulation of specific areas of the brain rather than deliberately damaging sections? For that matter, the fact that drugs can alter thought processes not just perception also strongly argues against being a brain in the vat by the same sort of logic.

How to check that you aren't a brain in a vat: inflict some minor brain damage on yourself. If it influences your mind's workings as predicted by neurology, now you know your brain is physically here, not in a vat somewhere.

No, there's no way of knowing that you're not being tricked. If your perception changes and your perception of your brain changes, that just means that the vat is tricking the brain to perceive that.

The "brain in the vat" idea takes its power from the fact that the vat controller (or the vat itself) can cause you to perceive anything it wants.

How to check that you aren't dreaming: make up a random number that's too large for you to factor in your head, factor it with a computer,

Do you have access to the computer software of your choice in your dreams? That sounds unusually vivid to me, maybe even lucid. I'm lucky if I can find a working pen and a desk that obeys the laws of physics in my dreams.

How to check that you aren't dreaming: make up a random number that's too large for you to factor in your head, factor it with a computer, then check the correctness by pen and paper. If the answer fits, now you know the computing hardware actually exists outside of you.

A similar method was used by Solaris protagonist to check if he isn't hallucinating.

How to check that you aren't dreaming: make up a random number that's too large for you to factor in your head, factor it with a computer, then check the correctness by pen and paper. If the answer fits, now you know the computing hardware actually exists outside of you.

The trick, then, is to instill in yourself a habit of checking whether you are asleep regularly (ie. even when you are awake). A habit of thinking "am I awake, let me check" is the hard part and without that habit your sleeping mind isn't likely to question itself. Literature on lucid dreaming talks a lot about such tests. In fact, combined with 'write dreams down as soon as you wake up' and 'consume X substance" it more or less summarizes the techniques.

I think "unusually clever" should be "sufficiently clever" in your caveat. I have very wide error bars on what I think would be usual, but I suspect that it's almost guaranteed to defeat those tests if it's defeated the overall test you've already applied of "have only memories of experiences consistent with a believable reality".

In which case both questions are indeed hopeless.

Notice that many of the incidents mentioned at your link don't involve nuclear bombs at all: many involve leaks at research facilities and power stations. Here's a chronological list of radiation incidents that caused injury from the start of the 20th century onwards. The vast majority don't involve nuclear bombs.

Historically, unless you were in Hiroshima or Nagasaki, you would have been less likely to die from a nuclear bombing than you would have been to die from a radiation leak, picking up a lost radioactive source without recognizing it (or living with someone who's brought one into your home), being poisoned with radiation by a coworker, or medical overexposure. (Note also that the list is surely incomplete.) It is possible that this trend will reverse in the future, but it's not obvious that it will.

More generally, gwern sounds about right to me on the subject of terrorists putting together their own nuke. (Or hauling one up from the bottom of the ocean.)

I am not. To even suggest that that this is a possibility anywhere near the level of a sovereign actor giving terrorists nukes is to dramatically overestimate terrorist groups' technical competence, and also ascribe basic instrumental rationality to them (a mistake; see my Terrorism is not about Terror).

Even if a terrorist could marshal the interest, assemble in one place the millions necessary, and actually hire a world-class submersible and in the scant days they can afford, find the wreckage of a bomb, it would probably be useless. US nukes are designed to failsafe, so if the wiring has corroded, or the explosives are misaligned? And that's ignoring issues with radioactive decay. (Was the bomb a tritium-pumped H-bomb? Well, given tritium's extremely short half-life, I'm afraid that bomb is now useless.)

I don't know the ins and the outs of the Summers case, but that article has a smell of straw man. Especially this (emphasis mine) :

You see, there's a shifty little game that proponents of gender discrimination are playing. They argue that high SAT scores are indicative of success in science, and then they say that males tend to have higher math SAT scores, and therefore it is OK to encourage more men in the higher ranks of science careers…but they never get around to saying what their SAT scores were. Larry Summers could smugly lecture to a bunch of accomplished women about how men and women were different and having testicles helps you do science, but his message really was "I have an intellectual edge over you because some men are incredibly smart, and I am a man", which is a logical fallacy.

From what I understand (and a quick check on Wikipedia confirms this), what got Larry Summers in trouble wasn't that he said we should use gender as a proxy for intelligence, but merely that gender differences in ability could explain the observed under-representation of women in science.

The whole article is attacking a position that, as far as I know, nobody holds in the West any more : that women should be discriminated against because they are less good at science.

Well, he also seems to be attacking a second group that does exist (those that say that there are less women in science because they are less likely to have high math ability), mostly by mixing them up with the first, imaginary, group.

I could have sworn that I'd seen this posted somewhere before, for example in this thread. Maybe it was on Stumbelupon...

Hmm... I don't know about recent comments, I just go to the posts I'm following. Hit control+F and then type (or copy/paste) "load more comments" and go through and hit each one. Then erase it and type the current date or yesterday's date in the formate "date month" (18 June) and it will highlight all of those comments (if you use youtube a lot, you might already use this method on the "see all comments" page except you have to type "hour" or "minute" instead of an exact time which is actually more convenient.) When you're done checking all of the new comments you can erase that and put in "continue this thread" (is that right, I forgot what it is exactly.)

Hope that helps.

Use the RSS feed that appears on the recent comments page. I use reader.google.com to read my RSS feeds. This will allow you to scroll back in bulk using just the scrollbar then read at leisure. It also shows comments as 'read' or 'unread' based on where you are up to.

The only measure I know of that might make it more efficient to catch up on recent comments is for you to go to your preferences page, and where it says "Display 50 comments by default," change the "50" to some larger number. I have been using "200" on a very slow (33.6 K bits/sec) connection.

Are there periods in your life when you read or at least skim every comment made on Less Wrong? The reason I ask is that I am a computer programmer, and every now and then I imagine ways of making the software behind Less Wrong easier to use. To do that effectively, I need to know things about how people use Less Wrong.