Open thread, Aug. 03 - Aug. 09, 2015

Upvoted for good cultural standards.

This is super interesting. Is this based on UDT?

Also in terms of reinforcement learning, if you work quickly, you will get more rewards per unit of time, and the rewards will be closer in time to the start the work (shorter time delay means better reinforcement).

Very good article. "Me" studies refers to, basically, studying yourself--which gets into topics of identity politics. (Instead of just studying your life, you might, say, study the history of your racial group in your country.) But the core of it is a simple model of how discussions get radicalized when people are studying oppression. The post-script is also fascinating reading, because someone objects to a minor comment in the first post in a way that highlights the underlying dynamics (see the first comment, possibly by the author of the email).

The next post on the same topic, On the problem of normative sociology, is also well worth reading.

(On an unrelated, but still rationalist, topic, see his post on the implications of psychology on consumerism / climate change.)

Great share

Even when a law professor thinks that those purchases should be illegal, I find it hard to imagine that the legal system moves against index funds.

look into something called a kanban board. Consider segments of:

• "tasks to do"
• "single task doing now"
• "single next task"
• "tasks awaiting external imput"
• "completed tasks"

Where a task X seems hard, break it down into smaller tasks, (task of break X down to smaller tasks)

As a bonus: make an estimate of how long each task will take. After completing - compare your predicting ability and update your time-guess methods.

I believe there is a misunderstanding in the word where if you taboo you might get:

= imagination

= describe with a human brain

= develop all the requirements for it to be feasible and the understanding of how to make it so within one human brain.

where for C3 if we can conceive3 it, it is logically possible.

The truth is usually simple, but arguments about it are allowed to be unboundedly complicated :P

Which is to say, I bet Chalmers has heard this argument before and formulated a counterargument, which would in turn spawn a counter-counterargument, and so on. So have you "proven" anything in a publicly final sense? I don't think so.

Doesn't mean you're wrong, though.

My default assumption is that if someone smart says something that sounds obviously false to me, either they're giving their words different meanings than I am, or alternatively the two-sentence version is skipping a lot of inferential steps.

Compare the cautionary tale of talking snakes.

Am I being a hubristic idiot for thinking I can disprove a famous philosopher so casually?

You certainly are appropriating higher status than you deserve from the academic point of view. :P

If you are, then so am I, because that was also my immediate reaction on hearing this conceivability argument.

One of the stronger factors influencing the frequency of wars is the ratio of young men to older men. Life extension would change that ratio to imply fewer wars. See http://earthops.org/immigration/Mesquida_Wiener99.pdf.

Stable regimes seem to have less need for oppression than unstable ones. So while I see some risk that mild oppression will be more common with life extension, I find it hard to see how that would increase existential risks.

Radical life extension might lead to overpopulation and wars that might escalate to existential risk level danger.

Is there anything that can't somehow be spun into increasing existential risk? The biggest existential risk is being alive at all in the first place.

For story purposes, using a multi-tiered variant of utilitarianism based on social distance could lead to some interesting results. If the character were to calculate his utility function for a given being by something Calculated Utility = Utility / (Degrees of Separation from me)^2, it would be really easy to calculate, yet come close to what people really use. The interesting part from a fictional standpoint could be if your character rigidly adheres to this function, such that you can manipulate your utility in their eyes by becoming friends with their friends. (e.g. The utility for me to give a random stranger $10 is 0 (assuming infinite degrees of separation), but if they told me they were my sister's friend, it may have a utility of $10/(2)^2, or $2.50) It could be fun to play around with the hero's mind by manipulating the social web.

How badly could a reasonably intelligent follower of the selfish creed, "Maximize my QALYs", be manhandled into some unpleasant parallel to a Pascal's Mugging?

They'd be just as subject to it as anyone else. It's just that instead of killing 3^^^3 people, they threaten to torture you for 3^^^3 years. Or offer 3^^^3 years of life or something. It comes from having an unbounded utility function. Not from any particular utility function.

Does anybody else get the sense that in terms of karma, anecdotes seem to be more popular than statistical analysis when rating comments?

Are you basing this observation on anecdotes or on statistical analysis? :-P

Bikeshed effect

I get the opposite sense.

Isn't "karma" just a fancy word for "how much I like this post/comment"? I mean at registration I didn't get questions like:

• (1) How much of your income have you donated to charity? [1]
• (2) What is more dangerous to humanity? God's rage, or an out-of-control AI? [2]
• (3) Which of the sequences is your favorite? Name two. [3]

• [1] You must enter a number above 100%
• [2] AI means artificial intelligence, but you're supposed to know that.
• [3] "Favorite" means one, and it asks for two; clearly the write answer is to point out the grammatical error.

So clearly not everyone here is as rational as they should be, or at least LW-rational.

Moreover, that's only by a per-person basis. On comments with more upvotes, it would probably be the community's outlook on the comment/post. Same with downvotes. Just a matter of scale, really.

Looks like the popular depression and anxiety tests are pretty valid but also non-overlap between test items indicates that they're overly broad terms.

The fact that you don't understand overlap as a layperson doesn't indicate that a test doesn't test for a real thing. On the other hand the DSM-V categories are likely not the best possible label. That even the NIH opinion who declared that they are willing to fund studies that don't use them and try to find new categories.

If you want to dig deeper in how to think about such terms "How to think straight about Psychology" by CFAR advisor and professor of psychology Keith Stanovich is a good read.

To my knowledge - the line "up to three lives" is quoted because a blood sample can be separated into 3 parts? or 3 samples, to help with different problems.

What is not mentioned often is the shelf-life for blood products. 3 months on the shelf and that pint is in the medical-waste basket. AKA zero lives saved.

And further, if a surgery goes wrong and they need multiple transfusions to stabilise a person the lives saved goes into fractional numbers. (0.5, 0.33, 0.25...) But those numbers are not pretty.

Further; if someone requires multiple transfusions over their life; to save their life multiple times...

There are numbers less than 1 (0); there are numbers smaller than a whole; and (not actually a mistake made here) real representative numbers don't often fall to a factor of 5 or 10. (5, 10, 50, 100, 1000).

Anyway if you are healthy and able to spare some blood then its probably a great thing to do.

Ike's article linked does start to cover adverse effects of blood donation; I wonder if a study has been made into it.

(http://www.ihn-org.com/wp-content/uploads/2014/04/Side-effects-of-blood-donation-by-apheresis-by-Hans-Vrielink.pdf comes as a source from wikipedia on prevelance of adverse effects) (oh shite thats a lot more common than I expected.

The risk I see is that donating blood temporarily disables you by a small amount. I would call it akin to being a little tipsy; a little sleep deprived, or a little drowsy; or a little low in blood pressure (oh wait yea). Nothing bad happens by being a little drowsy, or a little sleep deprived. It really depends on the whole-case of your situation as to whether something bad happens. (See: swiss cheese model: https://en.wikipedia.org/wiki/Swiss_cheese_model )

The important question to ask is - can you take it? If yes; then go right ahead. If you are already under pressure from the complexities of life in such a way that you might be adversely burdening yourself to donate blood; Your life is worth more (even for the simple reasoning that you can donate in the future when you are more up to it)

http://acesounderglass.com/2015/04/07/is-blood-donation-effective-yes/

I assume the effectiveness of blood donation is affected by whether someone has a rare blood type.

A core idea of EA is the marginal value of a donation. The marginal value of an additional person donating blood is certainly less than a live saved.

And as a corollary, should I move my charitable giving to bribing people to donate blood whenever there is a shortage?

Certainly not. Finding funding to have enough blood donations isn't a problem. Our medical system has enough money to pay people in times of shortage.

But it doesn't want to pay people. The average quality of blood of people who have to be bribed is lower than the average quality of people who donate blood to help their fellow citizens.

I don't think he suggests bayesian networks (which, to me, mean the causal networks of Pearl et al). Rather, he is literally suggesting trying to learn by Bayesian inference. His comments about nonlinearity I think are just to the effect that one shoudn't have to introduce nonlinearity with sigmoid activation functions, one should have nonlinearity naturally from Bayesian updates. But yeah, I think it's quite impractical.

E.g. suppose you wanted to build an email spam filter, and wanted P(spam). A (non-naive) Bayesian approach to this classification problem might involve a prior over some large population of email-generating processes. Every time you get a training email, you update your probability that a generic email comes from a particular process, and what their probability was of producing spam. When run on a test email, the spam filter goes through every single hypothesis, evaluates its probability of producing this email, and then takes a weighted average of the spam probabilities of those hypotheses to get its spam / not spam verdict. This seems like too much work.

There are reasons very little work is going into it - the concept makes very little sense compared to manipulating biological systems or making systems that work similar to biological systems. See http://www.sciencemag.org/content/347/6227/1221.short or this previous post of mine: http://lesswrong.com/lw/hs5/for_fai_is_molecular_nanotechnology_putting_our/97rl

I'm roughly your age, and have been working for 10 years at the same company (in the Seattle area, not SV, but we have offices there). Unlike your position, it's never been boring - I've been able to work on both immediate-impact and far-reaching topics, and there's always more interesting things coming down the pike.

I mostly want to address the age-ism and cultural oddities issue. It definitely exists, and is worse in California than other places. However, it's a topic that can't be analyzed by averages or aggregates over a geographic area. It varies so much by company, by position/role within a company, and by individual interaction with the nearby-team cultures that you really can't decide anything based on the region. This is especially true for remote work - your cultural experience will be far different from someone living there.

So, the questions you should be asking are about the expectations for your specific interaction with the employer and coworkers, rather than about the general HR-approved culture spiel you'll get if you ask generally.

Yes, as Dagon says it is very company specific. Is there a way that you could talk to people who already work at the company who are not the people who are involved in the hiring process? If you are on Linked In, perhaps you could find out if you have some connections who would talk to you informally over the phone or in person.

Even though you would be working remotely, it may be worth it to go visit the place in person to get a feel for things and observe things that they wouldn't tell you explicitly, before making a decision of this magnitude.

Also, read the company's annual report. There are clues to its culture in there, and numbers that will help make sense of the company and the direction it is likely to take in the near future. Not enough people read the annual report when applying for a position at a company or evaluating an offer.

I've made a fair number of predictions about things I need to do on PredictionBook, and I don't think it has had much any effect on my motivation. Boosting your productivity might be possible if you make optimistic predictions if you are strongly motivated to be well calibrated.

Another possible use of PredictionBook for motivation is getting a more objective view on whether you might complete a task by a certain date. If others think you are overconfident, then you could put in place additional things to ensure you complete the task.

Another idea: Self-fulfilling prophecies

This seems to be the general name for the phenomena of a prediction causing itself to be fulfilled. I don't have time to read the Wikipedia entry right now, but I suspect it'll offer some ideas about how to use predictions to your own advantage. Let me know if you think of anything good. I'll post a reply here if I do.

When you write the predictions, do you simply add optimism without changing the processes to reach a conclusion, or do you try to map out the "how" of making an outcome match more optimistic outcomes?

Now after I post this, I see there has been a brief discussion of analysis of competing hypotheses before on LessWrong, from which you can find an open source software for the methodology (GitHub).

I also see there are other softwares for this methodology, but none of these seem quite like what I want. I'll have to look closer.

Any other systems would be of interest to me. This is a good system for comparing competing hypotheses, but does nothing for the management of non-competing hypotheses (which could still be related).

Is "which is more effective" even a useful question to ask?

Suppose it was found that the most effective way to deal with people's fears of terrorism is to ban Islam. Should we then ban Islam?

(Also, if you will do X when doing X is most effective, that creates incentives for people who want X to respond unusually strongly to doing X. You end up creating utility monsters.)

In general the effectiveness of awareness raising programs intended to shift public perception of a risk is low.

I'm afraid that nobody knows, but you will have to dig into sociological studies to find out for sure.

I just want to offer you a different perspective, a parameter that might affect your investigation. It might be possible that cultural and economical influences affect the general level of anxiety in a population, so that even if you just ban a stimulus (say gun ownership) anxiety will just find another object of focus.

One answer could be that people don't really think that a superintelligence is possible. It doesn't even enter in their model of the world.

FWIW, I have been a long time reader of SF, have long been a believer of strong AI, am familiar with friendly and unfriendly AIs and the idea of the singularity, but hadn't heard much serious discussion on development of superintelligence. My experience and beliefs are probably not entirely normal, but arose from a context close to normal.

My thought process until I started reading LessWrong and related sites was basically split between "scientists are developing bigger and bigger supercomputers, but they are all assigned to narrow tasks -- playing chess, obscure math problems, managing complicated data traffic" and "intelligence is a difficult task akin to teaching a computer to walk bipedally or recognize complex visual images, which will teke forever with lots of dead ends". Most of what I had read in terms of spontaneous AI was fairly silly SF premises (lost packets on the internet become sentient!) or in the far future, after many decades of work on AI finally resulting in a super-AI.

I also believe that science reporting downplays the AI aspects of computer advances. Siri, self-driving cars, etc. are no longer referred to as AI in the way they would have been when I was growing up; AI is by definition something that is science fiction or well off in the future. Anything that we have now is framed as just an interesting program, not an 'intelligence' of any sort.

If you're not reading about futurism, it's unlikely to come up. There aren't any former presidential candidates giving lectures about it, so most people have never heard of it. Politics isn't about policy as Robin Hanson likes to say.

It's complicated, but maybe we could make some approximations. For example: "list ten things you care about, create a metric for each of them", providing a list of what people usually care about.

What purpose would such a measure serve? And do you try to find a universal measure or one that is individual for every person? Because different people have different goals, you could try to measure how well reality aligns with their goals, but then you just select for people who can accurately predict what they can achieve.

I have a definition of success. For me, it's very simple. It's not about wealth and fame and power. It's about how many shining eyes I have around me.

--Benjamin Zander

Bhutan's Gross National Happiness Index, and various indices inspired by it, attempt to measure this in populations. https://en.wikipedia.org/wiki/Happiness_economics

wouldn't it be sensible to define a measure of 'life success'?

"He who dies with the most toys wins" :-P

I think the most mathy (and thus, best :P) way to go about this is to think of the properties that these "utility" objects have, and just define them as objects with those properties.

For starters, you can compare them for size - The relationship is either bigger, or smaller, or the same. And you can do an operation to them that is a weighted sum - if you have two utilities that are different, you can do this operation to them and get a utility that's in between them, with a third parameter (the probability of one versus the other) distinguishing between different applications of this operation.

Actually, I think this sort of thing is pretty much what Savage did.

Seems to be an established conversation around this point, see: https://en.wikipedia.org/wiki/Ordinal_utility https://en.wikipedia.org/wiki/Cardinal_utility

"The idea of cardinal utility is considered outdated except for specific contexts such as decision making under risk, utilitarian welfare evaluations, and discounted utilities for intertemporal evaluations where it is still applied. Elsewhere, such as in general consumer theory, ordinal utility with its weaker assumptions Is preferred because results that are just as strong can be derived."

Or you could go back to the original Theory of Games proof, which I believe was ordinal- it's going to depend on your axioms. In that document, Von Neumann definitely didn't go so far as to treat utility as simply an integer.

It's a tempting thought. But I think it's hard to make the math work that way.

I have a lovely laptop here that I am going to give you. Suppose you assign some utility U to it. Now instead of giving you the laptop, I give you a lottery ticket or the like. With probability P I give you the laptop, and with probability 1 - P you get nothing. (The lottery drawing will happen immediately, so there's no time-preference aspect here.) What utility do you attach to the lottery ticket? The natural answer is P * U, and if you accept some reasonable assumptions about preferences, you are in fact forced to that answer. (This is the basic intuition behind the von Neumann-Morgenstern Expected Utility Theorem.)

Given that probabilities are real numbers, it's hard to avoid utilities being real numbers too.

The von Neumann-Morgenstern theorem say that if you are uncertain about the world then you can denominate your utility in probabilities. Since probabilities are real numbers, so are utilities.

There are various ways to get infinite and infinitesimal utility. But they don't matter in practice. Everything but the most infinite potential producer of utility will only matter as a tie breaker, which will occur with probability zero.

Cardinal numbers also wouldn't work well even as infinite numbers go. You can't have a set with half an element, or with a negative number of elements. And is there a difference between a 50% chance of uncountable utilons and a 100% chance?

I don't think that non-additivity is the only thing that matters about utilons: sometimes they do add, after all.

Besides that, yes, infinite cardinal numbers can have the property you cite: since for them
X + Z = max(X,Z)
if X < Y and Z < Y, it results
X + Z < Y

Meh. The assumption that bans won't work seems to miss most of the subtlety of reality, which can vary between the failure of U.S. prohibition of alcohol to Japan's two gun-related homocides per year.

Trying to summarize here:

The open letter says: "If we allow autonomous weapons, a global arms race will make them much cheaper and much more easily available to terrorists, dictators etc. We want to prevent this, so we propose to outlaw autonomous weapons."

The author of the article argues, that the technology gets developed either way and will be cheaply available, and then continues to say, that autonomous weapons would reduce casualties in war.

I suspect that most people agree, that (if used ethically) autonomous weapons reduce casualties. The actual question is, how much (more) damage can someone without qualms about ethics do with autonomous weapons, and can we implement policies to minimize the availability of autonomous weapons to people we don't want to have them.

I think the main problem with this whole discussion was already mentioned elsewhere: Robotics and AI experts aren't experts on politics, and don't know what the actual effects of an autonomous weapon ban would be.

Where did "pacifists" and the scare quotes around it come from?

The article's two main points are:

1 - a ban won't work
2 - properly programmed autonomous weapon (AW) could reduce causalties

So, the conclusion goes, we should totally dig AW.

Point n° 2 is the most fragile: they could very well reduce as increment causalties, depending on how they are programmed. It's also true that the availability of cheaper soldiers might make for cheaper (i.e., more affordable) wars. But point n° 1 is debatable also: after all, the ban on chemical and biological weapon has worked, sorta.

The real question that we should be asking is this

This sounds like a straw man, right there at the beginning. Stopped there.