Open Thread, Apr. 06 - Apr. 12, 2015

Interesting talk at BOAO forum : Elon Musk, Bill Gates and Robin Li (Baidu CEO). They talk about Superintelligence at around 17:00 minute.

https://www.youtube.com/watch?v=NG0ZjUfOBUs&feature=youtu.be&t=17m

• Elon is critical of Andrew Ng remark that 'we should worry about AI like we should worry about Mars overpopulation' ("I know something about mars" LOL)

• Bill Gates mentioned Nick Bostrom and his book 'Superintelligence'. His seems to have read the book. Cool.

• Later, Robin Li mentions China Brain projects, which appears to be Chinese government AGI project (anyone knows something about it? Sounds interesting...hopefully it won't end like Japans 'fifth-generation computing' in the 80s)

I'm leading a rationality training group. We're working through the most recent CFAR curriculum, but I also want to work from parts of the sequences.

Which posts in the sequences were particularly impactfull for you? Not just ones that you found interesting, but ideas that you actually implemented in your thinking about object-level stuff.

I'm particularly interested in posts that we could spin out into techniques to practice, like noticing confusion or leaving a line of retreat.

It's been more than a year since I wrote the mechanics of my recent productivity (shortly before being hired as a full-time MIRI researcher), and, as promised about a year ago, I have now posted a short update. It's been a wild year.

A lot of really stupid comments follow this article:

Tech titans’ latest project: Defy death

http://www.washingtonpost.com/sf/national/2015/04/04/tech-titans-latest-project-defy-death/

I recently stumbled across the work of Helen De Cruz. I think many of her articles will be of interest to readers of LW. Here are a few of particular interest:

• Where Philosophical Intuitions Come From
• The value of epistemic disagreement in scientific practice. The case of Homo floresiensis
• Evolved Cognitive Biases and the Epistemic Status of Scientific Beliefs

I have joined the ranks of LWers with tumblrs.

Also, interested in discussion on conversational tops and bottoms.

Is there a way to view a user's submitted articles from oldest to newest? For example, if I want to read Yvain's articles in chronological order, how would I go about doing that?

I think I’ve found a somewhat easy-to-make error that could pose a significant existential risk when making an AGI. This error can potentially be found in hierarchical planning agents, where each high-level action (HLA) is essentially its own intelligent agent that determines what lower-level actions to do. Each higher-level action agent would treat determining what lower-level action to do as a planning problem and would try to take the action that maximizes its own utility function (if its a utility-based agent) or (if it’s a goal-based agent) probability of accomplishing its goal while minimizing its cost function (UOCF).

For these agents, it is absolutely vital that each HLA’s UOCF prevents the HLA from doing anything to interfere with the highest-level action maximizing its utility function, for example by rewriting the utility function of higher-level actions or sending the higher-level actions deliberately false information. Failing to do so would result in an error that could significantly increase existential risk. To explain why, consider an agent whose highest-level action wants to maximize the number of fulfilling lives lived. In order to do this, the agent has a lower-level action whose goal is to go to a warehouse to get supplies. The cost function of this lower-level action is simply a function of, say, the amount of time it takes for the agent to reach the warehouse and the amount of money spent or money in damages done. In this situation, the lower-level action agent might realize that there is a chance that the higher-level action agent changes its mind and decides to do something other than go to the warehouse. This would cause the lower-level action to fail to accomplish its goal. To prevent this, this lower-level action may try to modify the utility function of the higher-level action to make it certain to continue trying to go to the warehouse. If this is done repeatedly by different lower-level actions, the resultant utility function could be quite different from the highest-level action’s original utility function and may pose a large existential risk. Even if the lower-level action can’t rewrite the utility function of higher-level actions, it may still sabotage the higher-level action in some other way to further its own goals, for example by sending false information to higher-level actions.

To prevent this, the utility function of the lower-level action can simply be to maximize the highest-level action’s utility, and it can see the UOCF it was provided with as a rough method of maximizing the highest-level action’s utility function. In order to make the UOCF accurately represent the highest-level action’s utility function, it would (obviously) need to place high cost on interfering with the highest-level action’s attempts to maximize its utility. Some basic ideas on how to do this is for there to be very high cost in changing the utility functions of higher-level actions or giving them deliberately false information. Additionally, the cost of this would need to increase when the agent is more powerful, as the more powerful the agent is, the greater damage a changed utility function could do. Note that although higher-level actions could learn through experience what the UOCFs of lower-level actions should be, great care would need to be taken to prevent the AGI from, when still inexperienced, accidentally creating a lower-level action that tries to sabotage higher-level actions.

Variable-score Iterated Prisoner's Dilemma?

There are a number of variants of the Iterated Prisoner's Dilemma - some with noise, some with random mutations, and so on. Different variants lead to somewhat different most-successful strategies. Does anyone know of any analysis done on IPD tourneys where each round's scoring isn't necessarily the same?

Has anyone seen a "group debate," where a group will decide on one point to put forward, and then another group will spend time coming up with a response, and then the first group will decide a response to that, and so on?

I was thinking of a conversational pattern I've seen, where one person will put forward an odd claim to a group, and then multiple members of the group will ask questions / make counterarguments to try to knock down the claim, but often in an uncoordinated way. That made me think of Kasparov Versus the World, where one player would come up with chess moves for white and a bulletin board would come up with chess moves for black, compared to one chess player simultaneously playing, say, twenty people at once, each playing a different set of black pieces. The first tends to make for much more impressive games and analysis, since one side is taking advantage of collective intelligence instead of individual mediocrity.

But while it's easy for games, it might be hard for arguments. Someone could easily model Kasparov's moves, but someone couldn't easily model potential arguments that someone else could make. (Then it seems like the various argument wikis that have been constructed are the more appropriate format.)

Why exactly did Eliezer stop writing here and started writing his new articles (on intelligent characters in fiction, angel investing etc.) on Facebook or Tumblr instead?

I don't see much brainstorming on LW. Brainstorming seems like a useful and fun thing to do.

I'm trying to wrap my mind around Hay's "Universal semimeasures", where he proves some very interesting equivalences.
In order to follow all the derivations, it would be very useful if there was a tool to draw binary trees and do some outlining of the branches, sub-trees, etc. Do you know of anything like that?

Does anyone know why a lot of work has gone in to vegetable-based imitation beef and chicken, but not into good imitation fish?

The Age of Adaline, a film about a mysteriously negligibly senescent woman:

https://www.youtube.com/watch?v=clbSd2JzAqc

Let me guess: The unaging woman finds fulfillment by pulling an Arwen or some similar nonsense.

Because Hollywood can't (yet) make films about the utter coolness where ordinary people discover this woman's power, reverse engineer it and become negligibly senescent themselves.

Self Help, CBT and quantified self Android applications

A lot of people on LW seem to hold The Feeling Good Handbook, of Dr. Burns in high regard when it comes to effective self-help. I am through the process of browsing a PDF copy, and it indeed seems like a good resource, as it is not only written in an engaging way, but also packed with various exercises, such as writing your day plan and reviewing it later while assigning Pleasure and Purpose scores to various tasks.

The problem I have with this, and any other self-help-exercise style of books is that I am simply too lazy to regularly print, draft or fill written exercise sheets. On the other hand - I have noticed that when prompted to do so by phone notification, I can usually be trusted to regularly fill in the forms of QS apps I have installed on my mobile or do exercises such as duolingo language tests.

Since the topics of CBT, depression and such seem to be quite widely discussed, I have two rather general questions I would like to ask to the community:

1) Do you know about any battle-tested mobile applications that implement CBT exercises as mentioned in the book of Dr. Burns? If so please do name them, as I would love to install one as well.

2) Do you think that creating a new mobile application to collect all Feel-Good-Hanbook exercises in one place, and remind user to do them regularly (i.e. once daily/weekly in most cases) is a good idea? Would you use such an application for yourself? I am a MSc Comp Sci student looking for some fun and useful projects to polish my android skills a bit, and I would love to work on something that might be useful to a wider community. [pollid:852]

Cold/flu, I hate having to spend a week in bed every 8 weeks or so. Any less-known tips?

For example I thought that dealing with ear pain with a cup of pan heated salt poured into a sock and held against it (increases blood circulation and somehow that helps) is pretty universal, but just yesterday it turned out an Austrian pharmacist never heard about it. It may be new to you as well. As a traditional folk remedy, it provides very quick symptomatic belief, works in about 10 mins, but if the pain returns the next day, doctor.

For the record, the known tips are: paracetamol (tylenol) but with muchas caution as it is a liver killer, or ibuprofen (I would say if you have kids, don't even keep paracetamol/tylenol in the house, ibuprofen works just as well and is safer) vit C, hot tea preferably with honey, bed, various kinds of throat lozenges.

A quote for those of you who are interested in decision theory, taken from Counterfactuals and Newcomb's Problem (sorry, though: gated link):

My second observation is closely related to the first. It is true that the state of box 2 is not causally dependent upon my act, in Newcomb's problem. But another kind of dependence relation obtains which is at least as important as causal independence and which seems to take clear precedence in the limit case. If I am completely certain that the being has predicted correctly, then the state of box 2 depends logically upon my current beliefs together with my act. That is, from the relevant propositions about the setup of the problem (including the proposition that the being has correctly predicted my choice) together with the proposition that I choose both boxes, it follows that I get $1,000. And from those relevant propositions together with the proposition that I choose only box 2, it follows that I get $1 million. (This logical dependence, of course, is the reason why the two-box choice seems patently irrational in the limit case.)

ETA: I realised that this might require context (otherwise it looks a little like I'm just randomly presenting a quote which makes a point that the community has heard often before). I thought that the quote was interesting because it's an example of philosophers who think that some concept of logical dependence should be more important than causal dependence in relation to rational choice. This view seems to be interestingly linked to the views of various people on LW.

Recently several people suggested opening a separate website for rational (or Less Irrational) discussion of political issues. If such a website is created, will you be interested in participating?

[pollid:845]

I've been working through the sequences and have been trying to apply things to my current project that has to do with remaking money. I'd appreciate any feedback. Did I miss anything major? Am I on the right track to fix my thinking?

http://www.hypercapital.info/news/2015/4/9/hypercapitalism-what-you-think-about-money-is-wrong

I'd love some feedback on the latest update on my kickstarter. Would anyone be willing to make sure I haven't misstated the math and/or concepts?

Hypercapitalism: Information Theory and Bayes

https://www.kickstarter.com/projects/hypercapital/hypercapital-experiment-testnet-apis/posts/1190876

A few times I've mentioned that nobody has adequately explained why the best method that we have of controlling humans (the market) wouldn't also work for AIs. Scott Alexander recently posted an entry... No Physical Substrate, No Problem... that is by far the best explanation that I've come across. From my perspective though his explanation is still far from adequate. The biggest problem is that there's no real recognition of the significance of garbage in, garbage out. I thoroughly explained my point here... Debugging Scott Alexander And Paul Krugman.

If your reply clearly reveals that you haven't bothered to read my thorough explanation... then this will support my suspicion that it's better to share just the link without any description of the contents. I'd prefer to trade with 1 person who has read the contents rather than trade with 100 people who have only read the description.

And of course you're certainly welcome to downvote this! But I'll stop sharing links when people stop clicking on them. In other words, I'll get the message when LessWrong completely vanishes from my blog's traffic statistics. As it stands... plenty of people still click on my links... so here I am!

1. Air Climbing
2. Visualizing The Economics of Education
3. Crazy Cable Confusion: Costless Content Creation
4. Drunk History and Economics - Sodom and Gomorrah vs Indiana

Bizarre Bundle?