Comment by aphyer on A toy model of the treacherous turn · 2016-02-13T06:42:49.218Z · score: 1 (1 votes) · LW · GW

Besides the obvious benefit of being awesome, I think there could be a more serious benefit to this. One extreme failure mode when imagining the behavior of an AI is not merely to fail to imagine it as being superintelligent but to imagine it as being less intelligent than yourself, as not doing things you could think of (a la That Alien Message). A game that consisted of you, the player, needing to come up with increasingly complicated ways to trick these 'shopkeeper' agents could illustrate this pretty neatly.

Comment by aphyer on 2014 iterated prisoner's dilemma tournament results · 2014-12-30T21:16:00.718Z · score: 2 (2 votes) · LW · GW

Something that bothers me about this tournament: I feel like a competitive tournament doesn't actually reward the kind of strategy that is meant to do well in Prisoner's Dilemna. As a (highly oversimplified) example, consider three bots who have the scores:

A: 10 B: 9 C: 2

Here, A is 'winning.' Suppose B can make a move that costs A 3 points and costs itself 1 point, leading to:

A: 7 B: 8 C: 2

B's payoff function has dropped. However, from a 'winning the tournament' approach, B has gone from 2nd to 1st, and so this outcome is now better for B. This feels wrong.

I doubt this was a really big issue here, but just on general principles I feel like competition by comparing scores is incompatible with a desire to explore the Prisoner's Dilemma, since you're turning a non-zero-sum game into a zero-sum game.

Comment by aphyer on Rationality Quotes October 2014 · 2014-10-10T00:04:02.239Z · score: 5 (7 votes) · LW · GW

The humans aren't doing what the math says. The humans must be broken.

SMBC on the Ultimatum Game

Comment by aphyer on Guardians of the Truth · 2014-05-28T14:24:19.307Z · score: 1 (1 votes) · LW · GW

I forget where I saw this (might actually have been elsewhere on LW?), but I encountered the idea that a component of the backward-reverent ages was the Roman Empire. When your civilization is built on the remnants of Roman roads that are better than anything you can make, it's forgivable to view the world as having fallen from grace.

Comment by aphyer on The Dilemma: Science or Bayes? · 2014-03-26T13:55:56.051Z · score: 0 (0 votes) · LW · GW

This is an old article, and it's possible that this question has already been asked, but I've been looking through the comments and I can't find it anywhere. So, here it is:

Why does it matter? If many-worlds is indistinguishable from the Copenhagen Interpretation by any experiment we can think of to do, how does it matter which model we use? If we ever find ourselves in a scenario where it actually does matter which one we use -- one where using the wrong model will result in us making some kind of mistake -- then we now have an experiment we can do to determine which model is correct. If we never find ourselves in such a position, it doesn't matter which model we decided on.

When phrased this way, Science doesn't seem to have such a serious problem. Saying "Traditional Science can lead to incorrect conclusions, but only about things that have no actual effect on the world" doesn't sound like such a searing criticism.

Comment by aphyer on Lone Genius Bias and Returns on Additional Researchers · 2013-11-03T02:33:26.347Z · score: 0 (0 votes) · LW · GW

Do you have any comments on the content of the article beyond this? The article makes a couple claims:

  1. The existence of Lone Genius Bias -- do you think it exists?
  2. The relevance of Lone Genius Bias -- do you think that, given Lone Genius Bias, you might be underestimating the odds of the US government developing AI and overestimating the odds of some nerd in a basement developing AI?
  3. The source of Lone Genius Bias -- do you think Lone Genius Bias comes from the small group size in the ancestral environment?

Your comment makes it clear that you disagree with the third of these. And that's a pretty fair response: post hoc evo-psych is dangerous, prone to bias, and usually wrong. But your comment ALSO seems to say that you think this entitles you to completely disregard the first two points of the article: the author's arguments for the existence of Lone Genius Bias and his arguments that it's misleading us as to where AI is likely to come from.

If you agree with the rest of the article, but dislike the evo-psych part, you can probably find a more polite way to phrase that. If you disagree with the rest of the article as well, you should be counter-arguing the rest of the article on its own merits, rather than zeroing out the one weakest point and disregarding every other point the author made.

Comment by aphyer on Schelling fences on slippery slopes · 2013-08-08T15:11:43.965Z · score: 1 (1 votes) · LW · GW

This. This this this. I feel like the "fire in a crowded theater" example is a pretty painfully outdated one. If you imagine a more crowded theater made of dry wood with no sprinklers/fire extinguishers, it becomes a lot more reasonable to expect people to panic at the thought of a fire.

Comment by aphyer on For Happiness, Keep a Gratitude Journal · 2013-07-23T00:23:55.559Z · score: 0 (0 votes) · LW · GW

One question regarding the specifics of the 'write stuff down to feel better about it' that doesn't seem to be covered:

After (to take the example from the article) being laid off, I'll probably have a rather negative view of the event. I will feel bad about it, and I will want to blame others. When I write my "feelings regarding the layoff", if they wind up as an angry rant about "my stupid worthless boss" and "my goddamn no-good backstabbing coworkers", does that still work? Or do I need to be more even-handed about this, identify things I might have done wrong and reasons I might have deserved it, admit that it was mostly my fault, etc...?

I don't have access to the books you cite as sources, so it's possible that this is covered specifically in them. But if you can actually boost "happiness, self-esteem, health, and psychological and physical well-being" by writing down an angry rant, I would find that rather surprising.

Comment by aphyer on Rationality Quotes July 2013 · 2013-07-18T05:13:14.631Z · score: 0 (0 votes) · LW · GW

Not sure I agree with that. Emphasis on "think" undercuts the point: I wouldn't say that I "think you can't jump over the moon", even though I do not have a formal proof of impossibility handy for that, I'd just say "you can't do that."

In fact, I almost like it better without the word "think" at all: "Whatever can't be done, someone will come along and do it." YMMV, though.

Comment by aphyer on Prisoner's dilemma tournament results · 2013-07-11T04:17:18.026Z · score: 2 (2 votes) · LW · GW

Sorry if this is a stupid question, but this tournament looked to me like a thinly disguised version of:

"Construct a robot that can read code and interpret what it means."

which is a Really Hard Problem.

Is that not a fair description? Was there some other way to approach the problem?

The only way I can see to go about constructing a GOOD entrant to this is to write something that can take as its input the code of the opponent and interpret what it will actually DO, that can recognize the equivalence between (say):

return DEFECT


if 1: return DEFECT return COOPERATE

and can interpret things like:

if opp_code == my_code return COOPERATE return DEFECT

And I have no idea how to go about doing that. From the fact that the winning entrants were all random, it seems safe to say that no entrants had any idea how to go about doing that either.

Am I missing something here?

Comment by aphyer on Welcome to Less Wrong! (5th thread, March 2013) · 2013-04-23T16:43:26.982Z · score: 12 (12 votes) · LW · GW

Hi, I'm Andrew, a college undergrad in computer science. I found this site through HPMOR a few years ago.

Comment by aphyer on Uncritical Supercriticality · 2011-10-09T04:51:33.859Z · score: 1 (1 votes) · LW · GW

I realize that this comment has been up for a long time, but just in defense of Buzz Aldrin: the punch was less in response to the man claiming that he was wrong, and more in response to the man being verbally abusive (don't believe everything you hear, search on Google for Buzz Aldrin Punch and you can get a video for yourselves.) There's a difference between violence being the appropriate response to reasoned argument and violence being the appropriate response to abuse/someone else's violence/etc.