The Blue-Minimizing Robot

The robot is not consequentialist, its decisions are not controlled by the dependence of facts about the future on its decisions.

The conclusion I'd draw from this essay is that one can't necessarily derive a "goal" or a "utility function" from all possible behavior patterns. If you ask "What is the robot's goal?", the answer is, "it doesn't have one," because it doesn't assign a total preference ordering to states of the world. At best, you could say that it prefers state [I SEE BLUE AND I SHOOT] to state [I SEE BLUE AND I DON'T SHOOT]. But that's all.

This has some implications for AI, I think. First of all, not every computer program has a goal or a utility function. There is no danger that your TurboTax software will take over the world and destroy all human life, because it doesn't have a general goal to maximize the number of completed tax forms. Even rather sophisticated algorithms can completely lack goals of this kind -- they aren't designed to maximize some variable over all possible states of the universe. It seems that the narrative of unfriendly AI is only a risk if an AI were to have a true goal function, and many useful advances in artificial intelligence (defined in the broad sense) carry no risk of this kind.

Do humans have goals? I don't know; it's plausible that we have goals that are complex and hard to define succinctly, and it's also plausible that we don't have goals at all, just sets of instructions like "SHOOT AT BLUE." The test would seem to be if a human goal of "PROMOTE VALUE X" continues to imply behaviors in strange and unfamiliar circumstances, or if we only have rules of behavior in a few common situations. If you can think clearly about ethics (or preferences) in the far future, or the distant past, or regarding unfamiliar kinds of beings, and your opinions have some consistency, then maybe those ethical beliefs or preferences are goals. But probably many kinds of human behavior are more like sets of instructions than goals.

Also, you misspelled my name - it's Quirinus, not Quirinius.

I'll be interested to see where you go with this, but it seems to me that saying, "look, this is the program the robot runs, therefore it doesn't really have a goal", is exactly like saying "look, it's made of atoms, therefore it doesn't really have a goal".

Goals are precisely explained (like rainbows), and not explained away (like kobolds), as the controlled variables of control systems. This robot is such a system. The hypothetical goal of its designers at the Department of Homeland Security is also a goal. That does not make the robot's goal not a goal; it just makes it a different goal.

We feel like we have goals and preferences because we do, in fact, have goals and preferences, and we not only have them, but we are also aware of having them. The robot is not aware of having the goal that it has. It merely has it.

                  R |
                    |
                    V
                +-------+
                |       |
           +--->|       |----+
           |    |       |    |
           ^    +-------+    v
           |                 |
....... P  | ............... | O .......
           |                 |
           ^    +-------+    v
           |    |       |    |
           +----|       |<---+
                |       |
                +-------+
                    ^
                    |
                  D |

                +-------+
            D   |       |   O
          ----->|       |----->
                |       |
                +-------+

This robot is not a consequentialist - it doesn't have a model of the world which allows it to extrapolate (models of) outcomes that follow causally from its choices. It doesn't seem to steer the universe any particular place, across changes of context, because it explicitly doesn't contain a future-steering engine.

What exactly is meant by the robot having a human-level intelligence? Does it have two non-interacting programs: shoot blue and think?

Ah, excellent. This post comes at a great time. A few weeks ago, I talked with someone who remarked that although decision theory speaks in terms of preferences and information being separate, trying to apply that into humans is fitting the data to the theory. He was of the opinion that humans don't really have preferences in the decision theoretic sense of the word. Pondering that claim, I came to the conclusion that he's right, and have started to increasingly suspect that CEV-like plans to figure out the "ultimate" preferences of people are somewhat misguided. Our preferences are probably hopelessly path-, situation- and information-dependent. Which is not to say that CEV would be entirely pointless - even if the vast majority of our "preferences" would never converge, there might be some that did. And of course, CEV would still be worth trying, just to make sure I'm not horribly mistaken on this.

The ease at which I accepted the claim "humans don't have preferences" makes me suspect that I've myself had a subconscious intuition to that effect for a long time, which was probably partially responsible for an unresolved disagreement between me and Vladimir Nesov earlier.

I'll be curious to hear what you have to say.

... the mistake began as soon as we started calling it a "blue-minimizing robot".

Agreed. But what kind of mistake was that?

Is "This robot is a blue-minimizer" a false statement? I think not. I would classify it as more like the unfortunate selection of the wrong Kuhnian paradigm for explaining the robot's behavior. A pragmatic mistake. A mistake which does not bode well for discovering the truth, but not a mistake which involves starting from objectively false beliefs.

Why does the human-level intelligence component of the robot care about blue? It seems to me that it is mistaken in doing so. If my motor cortex was replaced by this robot's program, I would not conclude that I had suddenly started to only care about blue, I would conclude that I had lost control of my motor cortex. I don't see how it makes any difference that the robot always had it actions controlled by the blue-minimizing program. If I were the robot then, upon being informed about my design, I would conclude that I did not really care about blue. My human-level intelligence is the part that is me and therefore contains my preferences, not my motor cortex.

Without overfitting, the robot has the goal of shooting at what it sees blue. It achieves its goal. What I get from the article is that the human intelligence mis interprets the goal. Here I see the definition of a goal to equal what the program is written to do, hence it seems inevitable that the robot wll achieve its goal (if there is a bug in the code that misses shooting a blue object every 10 days, then this should be considered part of the goal as well, since we are forced to define the goal in hindsight, if we have to define one)

Does that "human level intelligence module" have any ability to actually control the robot's actions, or just to passively observe and then ask "why did I do that?" What're the rules of the game, as such, here?

Robin Hanson would also argue that the robot wants to minimize things that are 'far', since blue is 'far'

One of the obvious extensions of this thought experiment is to posit a laser-powered blue goo that absorbs laser energy, and uses it to grow larger.

This thought experiment also reminds me: Omohundro's arguments regarding likely uFAI behavior are based on the AI having goals of some sort - that is, something we would recognize as goals. It's entirely possible that we wouldn't perceive it as having goals at all, merely behavior.

Is there a sense in which we can conclude that the robot is a blue-minimizing robot, in which we can't also conclude that it's an object-minimizing robot that happens to be optimized for situations where most of the objects are blue or most of the backgrounds are non-blue? (Perhaps it's one of a set of robots, or perhaps the ability to change its filter is an intentional feature.)

A couple of points here. First, as other people seem to have indicated, there does seem to be a problem with saying 'the robot has human level intelligence/self-reflective insight' and simultaneously that it carries out its programming with regards to firing lasers at percepts which appear blue unreflectively, in so far as it would seem that the former would entail that the latter would /not/ be done unreflectively. What you have here are two seperate and largely unintegrated cognitive systems one of which ascribes functional-intentional properties to things, including the robot, and has human level intelligence on the one hand and the robot on the other.

The second point is that there may be a confusion based upon what your functional ascriptions to the robot are tracking. I want to say that objects have functions only relative to a system in which they play a role, which means that for example the robot might have 'the function' of eliminating blue objects within the wider system which is the Department of Homeland Security, however there is no discoverable fact about the robot which describes it's 'function simpliciter'. You can observe what appears to be goal directed behaviour, of course, but your ascriptions of goals to the robot are only good in so far as they serve to predict future behaviour of the robot (this is a standard descriptivist/projectivist approach to mental content ascription, of the sort Dennett describes in 'The Intentional Stance'). So when you insert the investing in front of the robots camera it ceases to exercise the same goal directed behaviour or (what amounts to the same, expressed differently) your previous goal ascriptions to the object cease to be able to make reliable predictions of the robots future behaviour and need to be corrected. ((I'm going to ignore the issue of the ontological status of these ascriptions. If this is an interest you happen to have Dennett discusses his views on the subject in an essay entitled 'Real Patterns' and there is further commentary in a couple of the articles in Ross and Brook's 'Dennett's Philosophy)).

I realise you are consciously using a naive version of behaviourism as the backdrop of your discussion, so it's possible that I'm just jumping ahead to 'where you're going with this' but it does seem that with subsequent post-behaviourist approaches to mental content ascription the puzzle you seem to be describing of how to correctly describe the robot dissolves. ((N.B. - You might want to Millar's Understanding People which surveys a broad range of the various approaches to mental state ascription)).

Shouldn't the human intelligence part be considered part of the source code? With its own goals/value functions? Otherwise it will be just a human watchig the robot kind of thing.

I wonder how many people upvoted this post less for the ideas expressed and more because they like robots.

I think I upvoted it for the ideas, but can't honestly guarantee that the "oooh, shiny cool robot analogy appeals to my geeky heart" factor wouldn't have made me upvote even if I found the ideas uninteresting.

I haven't thought much about why it feels like we have goals (desires), so I look forward to that! I do think it's quite possible that eliminativism about 'beliefs' and 'desires' will turn out to be the best way to go. Certainly, the language of 'reinforcers' fits well with our understanding of the reward-learning system in the brain.

One can absolutely construct a utility function for the robot. It's a "shooting-blue maximizer". Just because the appearing utility function is wrong doesn't mean there isn't a utility function.

I consider all of the behaviors you describe as basically transform functions. In fact, I consider any decision maker a type of transform function where you have input data that is run through a transform function (such as a behavior-executor, utility-maximizer, weighted goal system, a human mind, etc.) and output data is generated (and in the case of humans sent to our muscles, organs, etc.). The reason I mention this is that trying to describe a human's transform function (i.e., what people normally call their mind) as mostly a behavior-executor or just a utility-maximizer leads to problems. A human's transform function is enormously complex and includes both behavior execution aspects and utility maximization aspects. I also find that attempts to describe a human's transform function as 'basically a __' results in a subsequent failure to look at the actual transform function when trying to figure out how people will behave.

Well-written and insightful. This post reminds me very much of the free will sequence, except that the reduction is on the level of genes and the adaptations they code for rather than physical laws. I look forward to seeing the rest of the sequence, and I'm interested to see how you will dissolve the feeling that our behavior and actions are goal-oriented.

The entire example is deeply misleading. We model the robot as a fairly stupid blue minimizer because this seems to be a good succinct description of the robots entire externally observable behavior and would cease to do so if it also had a speaker or display window with which it communicated it's internal reflections.

So to retain the intuitive appeal of describing the robot as a blue minimizer the robots human level intelligence must be walled off inside the robot unable to effectively signal to the outside world. But so long as the human level intelligence is irrelevant to predicting the robots exterior behavior the blue minimizing model is an appropriate one to keep in mind to guide our interactions with the robot. That is like any good scientific model it provides good predictive power relative to it's cost in mental (or simulating computer's) effort and memory.

It's pretty obvious why it's useful to us to describe stuff in ways that lets us feasibly predict/approximate the behavior of external entities/effects we encounter. Perhaps though you are puzzled or arguing against the idea that belief-desire style models are often a good tradeoff between accuracy and ease of use. However, this is also easily explained as a result of evolutionary hard wiring that effectively functions as a hardware accelerator for belief desire models so we didn't get eaten and the salience of objects designed by other humans to our lives. The acceleration means that even in domains were the fit is poor (like charges want to get away from each other) the ease of application still makes them a useful heuristic. Also since human made objects are usually built to achieve a particular goal that had to be represented usefully in the builders mind these objects usually offer the most effective behavior for accomplishing that goal relative to a given level of computational complexity.

In other words since the guy who builds the robot to lase the blue dyed cancer cells does so by coming up with a goal he wants the robot to achieve (discriminating between blue cells and other blue things is hard so we will just build a robot to fry all the blue things it sees) and then offering up the best implementation he can come up with given the constraints the resulting behavior can be well modeled as the object desiring some end but being stupid in various ways. In other words if you want to zap blue cells you don't add extra code to 1 time in a million zap yellow nor would you tack on AI without needing it's expertise to implement the desired behavior so the resulting behavior looks like a stupid creature trying to achieve the inventors chosen goal.

Interstingly I suspect that being well described by a belief-desire model probably simply corresponds to being on the set of non-dominated ways to achieve a goal people can reasonably conceptualize. Thus we see it all the time in evolution as we can easily understand both the species level goal of survival and individual level goals of avoiding suffering and satisfying some basic wants and natural selection ensures that the implementations we usually see are at least locally non-dominated (if you want to make a better hunter on the savannah than the lion you have to either jump to a whole new basic design or use a bigger computational/energy budget.)

EDIT: It just clicked after finishing my thought.

If its human handlers (or itself) want to interpret that as goal directed behavior, well, that's their problem.

I was thrown off by all the comments about the robot and its behavior. This is more about the comparison of behavior-executor vs. utility-maximizer, not the robot. EDIT:

Perhaps I am missing the final direction of this conversation, but I think the intelligence involved in the example has mapped the terrain and failing to update the map once it has been seen to not be correct.

Watching the robot's behavior, we would conclude that this is a robot that destroys blue objects.

Correct, the robot is designed to shoot its laser at blue things.

Here the robot is failing at its goal of being a blue-minimizer.

The robot is failing at nothing. It is doing exactly as programmed. The laser, on the other hand, is failing to eliminate the blue object, as what is expected when the laser is shot at something. Now that this experiment has been conducted and the map has been found to be wrong, correct the map. It is no longer a Blue-Minimizing Robot, it is a Blue-Targeting-and-Shooting Robot. The result of the laser shot is variable.

The right way to reduce the amount of blue in the universe is to destroy the projector; instead its beams flit harmlessly through the hologram.

The discussion has mapped it as a Blue-Minimizing Robot, it is not, as proven by this experiment. If this were to be made such, more programming would have to be implemented in-case of the laser not having the intended effect. Being there is not a way of altering the programming, there is no way of changing the terrain, so the map must be changed.

The human-level intelligence version of the robot will notice its vision has been inverted. It will know it is shooting yellow objects. It will know it is failing at its original goal of blue-minimization.

I don't see that, why would it "care" if its goal isn't complex enough to allow it to care about the subversion of its sensors? I mean, the level of intelligence seems irrelevant here. Intelligence isn't even instrumental to such simple goals because all it "wants" is to fire a laser at blue objects. Its utility function says nothing about maximizing its efficiency or anything like that.

Take three common "broad" or "generally-categorizable" demographics of minds: Autistic people, Empaths (lots of mirror neurons dedicated to modeling the behavior of others), Sociopaths, or "Professional Psychopaths" (high-functioning without mirror neurons, responsible for most systemic destruction, precisely because they can appear to be "highly functional and productive well-respected citizens"), Psychopaths (low-functioning without mirror neurons, most commonly an "obvious problem"). All of the prior humans' minds work on the principle of emergent order, with logic, reason, introspection being the alien or uncommon state of existence that is a minor veneer on the surface of the vast majority of brain function which is: "streaming perception and prediction of emergent patterns."

A robot that never evolved to "get along" with other sentiences, and is programmed in a certain way can "go wrong" or have any of billions of irrational "blue minimizing" functions. It seems that sure, it's a "behavior executor," not a "utility maximizer." I would go further and say that humans are not "ultility maximizers" either, except when they are training themselves to behave in a robotic fashion toward the purpose of "maximizing a utility" based on the very small number of patterns that they have consciously identified.

There's no reason for a super-human intelligence (one with far more neocortex, or far more complex neocortex that's equipped to do far more than model linear patterns, which perhaps automatically "sees" exponentials, cellular automata, and transcendental numbers) to be so limited.

Humans aren't much good at intelligent planning that takes other minds, and "kinds of minds" into account. That's why our societies regularly fall into a state of dominion and enslavement, and have to be "started over" from a position of utter chaos and destruction (ie: the rebuilding of Berlin and Dresden).

Far be it from me to be "mind-killed," but I think avoiding that fate should be a common object of discussion among people who are "rational." (ie: "What not to do.")

I also don't think it's fair to lump "behaviorists" (other than perhaps B.F. Skinner) into an irrational school of "oversimplification." Even Skinner noted that his goal was to get to the truth, via observation. (Eliminate biases.) By trying to make an entire school out of the implications of some minds, some of the time, we oversimplify the complex reality.

Behaviorism has caught scores of serial killers. (According to John Douglas, author of Mindhunter, originator of the FBI's Investigative Support Unit.) How? It turns out that serial killer behavior isn't that complex, and it's seeking goals that superior minds actually can model quite accurately. (This is much like a toddler chasing a ball into the street. Every adult can model that as a "bad thing," because their minds are superior enough to understand 1-what the child's goal is 2-what the child's probable failures in perception are 3-how the entire system of child, ball, street, and their inter-related feedbacks are likely to produce, as well as how the adult can, and should, swoop in and prevent the child from reaching the street.)

So, behaviorism does help us do two things: 4-eliminate errors from prior "schools" of philosophy (which were, themselves, not really "schools" but just significant insights) 5-reference "just what we can observe," in terms of revealed preferences. Revealed preferences are not "the whole picture." However, they do give us a starting point for isolating important variables.

This can be done with a robot or a human, but the human is a group of "messy emergent networks" (brain regions, combined with body feedback, with nagging long-term goals in the background, acting as an "action-shifting threshold") whose goals are the result of modeled patterns and instances of reward. The robot, on the other hand, lacks all the messy patterns, and can often deal with reality as a set of extreme reductions, in a way that no (or few) humans can.

The entire "utility function" paradigm appears to be a very backwards way of approximating thought to me. First you start with perceived patterns, then, you evolve ever-more-complex thought.

This allows you to develop goals that are really worth solving.

What we want in a super-intelligence is actually "more effective libertarians." Sure, we've found that free markets (very large free networks of humans) create wealth and prosperity. However, we've also found that there are large numbers of sociopaths who don't care about wealth and prosperity for all, just for themselves. Such a goal structure can maximize prosperity for sociopaths, while destroying all wealth and prosperity for others. In fact, this repeatedly happens throughout history, right up to the present. It's a cycle that's been interfered with temporarily, but never broken.

Would any robot, by default, care about shifting that outcome of "sociopaths dominate grossly-imperfect legal institutions"? I doubt it. Moreover, such a sociopath could create a lasting peace by creating a very stable tyranny, replete with highly-functional secret police, and a highly effective algorithm for "how to steal the most from every producer, while sensing their threshold for rebellion."

In fact, this is what the current system attempts to accomplish: There's no reason for the system to decay to Hitler's excesses, when scientists, producers, engineers, etc. have found (enough) happiness (and fear) in slavery. How much is "enough"? It's "enough (happiness) to keep producing without rebellion," and "enough (fear) to disincentivize rebellion."

In the rest of this sequence, I want to expand upon this idea. I'll start by discussing some of the foundations of behaviorism, one of the earliest theories to treat people as behavior-executors. I'll go into some of the implications for the "easy problem" of consciousness and philosophy of mind. I'll very briefly discuss the philosophical debate around eliminativism and a few eliminativist schools. Then I'll go into why we feel like we have goals and preferences and what to do about them.

This is like baling a few thousand gallons of water while the Titanic is sinking. 6-It won't make any difference to any important goal, short-term or long-term 7-It deals with a local situation that is irrelevant to anything important, worldwide 8-It deals with theories of the mind that are compatible with Francis Crick and Jeff Hawkins' work, but only useful to narrow sub-disciplines like "How do we know when law enforcement should take action?" or "When we see this at a crime scene, it's a good threshold-based variable for how many resources we should throw at the problem." 9-Every "school" that stops referring to reality and nature, to the extent it does so, is horribly flawed (this is Jeff Hawkins, who is right about almost everything, screwing up royally in dismissing science fiction as "not having anything important to say about brain building.") 10-When you're studying human "schools," you're studying a narrow focus of human insight described with words("labels" and "maps") instead of the insight they've derived from their modeling of the territory. (Kozybski,who himself,turned a few insights into a "school")

This article got me thinking about a few things. Centrally, there is no necessary condition on things which really do have goal directed behaviour to maximise. Revealed preference theory in economics effectively identifies the choices people make with their preference. Thus humans are seen as maximisers of their preferences in an almost trivial sense, if preferences are also assumed to fall under a ranking (another thing economists assume). Yet this goal orientated explanation in terms of rationality could be wrong for two reasons: firstly, the choices could not be goal orientated - they could simply be behaviour with no goal-based cause - or they could be the result of a different goal, which may not maximise. The first possibility is discussed in the article. The second is not, but is interesting, as the second condition could be met for many reasons. If I went to see a movie, a believer in revealed preference theory may conclude I preferred seeing that movie over all else that night. But perhaps I went to the wrong movie by mistake. Perhaps I was forced to go to the movie by gunpoint, and preferred to be at home and safe. Perhaps I really preferred to go to see a play, but hadn't really reflected on my preferences very hard. Or perhaps I went to the movie for a friend who wanted to see it with me, but in fact I actually would have preferred to not go.

In some of those cases, we might explain the supposed problem away by reflecting that in those particular circumstances, my preference was in fact to go to the movie; but had the circumstances been different I would have preferred otherwise. For instance, at gunpoint, in those circumstances, I wouldn't prefer to walk home, as I would get shot. However, this counterargument presumes we can't preferences over counterfactual situations, which we evidently can have. Also, it isn't very useful to the economist, who would like to conclude that preferences are stable, as so that the choice a person makes at one time will be repeated in the future.

Also, the assumption that the laser is for destruction may be false. What is the laser tags or scans blue objects? Maybe for a space probe, or a scan on enemy robots? What if the robot makes the assumption that the laser is for destruction when it's not, or vise-versa? Does it really matter, when it is programmed to do it anyway? What if the AI is destroying, or thinks it's destroying fellow robots? What if it does not want to? Maybe this is an example of religion from assumption? Because it is 'destroying' blue things, it assumes that is it's purpose. Behavior/motivation from assumption...

This was very interesting, thanks for sharing. My first reply on-site (for a while at least), hope it wasn't in too much of a different direction from what you meant.

the robot continuously analyzes the average RGB value of the pixels in the camera image; if the blue component passes a certain threshold, the robot stops, fires its laser at the part of the world corresponding to the blue area

I note that if the the robot only looks at the blue RGB component, then it will end up firing its laser not just at blue things (low R, low G, high B), but also white things (high R, high G, high B), fuchsia things (high R, low G, high B), teal things (low R, high G, high B), and variants of said colors. "Blue-minimizing" is not even a correct description!

The robot is a behavior-executor, not a utility-maximizer.

Or it only gets disutility from seeing blue objects :P

It seems like we need to taboo the word "goal" and replace it with several different things. 1. Actions. 2. Conscious (verbal) intentions (past intentions may conflict with present intentions). 3. "unconscious intentions" (this should probably be tabooed, but I haven't figured out how best to do it. Perhaps an unconscious intention is something which we do, but we don't know why or we confabulate why.).

it seems that Anna Salamon's and Hanson's interpretations could simply be viewed as changing verbal intentions and the question if how to gain goal (intention) stability.

A lot of this is discussed at length in the book The Ecological Approach to Visual Perception by Gibson. He put out a pretty radical idea that all cognition is goal driven, and perception is the action of converting sensory energy into scored models (scores are computed as anticipated experience of goal-fulfillment). Fundamentally, all thinking is planning. There's quite a bit of literature coming out on this in the fields of online learning theory and actionable information theory, in particular by Soatto at UCLA.

Curious that you worded it: "...we would be tempted to call this robot a blue-minimizer." Then you said the robot "wants". The entire discussion which followed is invalid, because everyone involved assumed human characteristics to an electronic mechanical device. Robots do not have "goals" either, nor self-motivation. IT has no concept of reducing blue objects. Your "temptation" was your sensed instinct that you were stepping over the line of limitations. Even the origin of the word "robot" meaning "forced labor" is inaccurate.

And are you a utility-optimizer, or a behavior-executor?

Yes, yes, yes!

I think all of these explanations hold part of the puzzle, but that the most fundamental explanation is that the mistake began as soon as we started calling it a "blue-minimizing robot".

Your post reminded me of a quote by Eliezer Yudkowsky:

An electron is not a billiard ball, and it’s not a crest and trough moving through a pool of water. An electron is a mathematically different sort of entity, all the time and under all circumstances, and it has to be accepted on its own terms.

The universe is not wavering between using particles and waves, unable to make up its mind. It’s only human intuitions about QM that swap back and forth. The intuitions we have for billiard balls, and the intuitions we have for crests and troughs in a pool of water, both look sort of like they’re applicable to electrons, at different times and under different circumstances. But the truth is that both intuitions simply aren’t applicable.

Humans are not selfish or altruistic, humans probably don't even have a utility-function because our goals are not stable. Our goals are rigidly coupled with the environment and circumstances we reside in. Words are too simplistic to describe what we are and what we want.

What is the difference between a smart 'shoot lasers at "blue" things' robot and a really dumb 'minimize blue' robot with a laser?

This post brings to mind a fault I see with trying to create a trust system of Friendly AI's. Humans are inherently untrustworthy and random. A robotic AI that is built to be friendly (or at least interact) with humans should by most concepts have a set of rules to work with that minimize its desire to kill us all and turn us into computer fuel. Even in an imperfect world, AI's would be trained to deal with humans in a forthright and honest fashion, to give the truth when asked, and to build assumptions based on facts and real information. Humans, however, are irrational creatures that lie, cheat, and steal when it is within our own best interest to do so, and we do it on a regular basis. For those of you who disagree with that premise, please look at the litany of laws we are asked to follow on a daily basis, starting with traffic laws. Imagine a world of AI drivers and place a human in their midst. Then take away all 'rules' that force every driver to move in x direction at y speed on a given roadway. The AI drivers would move with purpose, but be programmed to understand how important their speed and direction was based on the purpose of their travel. Those going to work at a leisurely pace would drive slower, and congregate in one or two areas of the road. The AI's that need to run a speedy errand or who are in an emergency would move faster and be programmed to take into account the slower vehicles. But a human in their midst would not care about the others so much as about their own personal issues. They would want to move faster because they like driving fast, or would want to move in the right because they get nervous in the left lanes. Or perhaps they would drive in the area that gave them the best view of the sunset, and slow down to enjoy it - forcing the AI's behind them to slow down as well.

And when we take the example of an AI who is supposed to work with humans as a receptionist, what does it do when the AI is faced with a human who lies to get past it? If the human lies convincingly and the AI let's him go, how will the AI react when it finds out the human lied? Are all humans bad? If the same human returns and is now part of the company, will the AI no longer 'trust' that human's information? If a human uses the AI to mess with another human (don't tell me people never use computers to play pranks on each other) how will the AI 'feel' about being used in such a manner? As humans, we have a set of emotions and memories that allow us to deal with people who do such things. Perhaps we would have a stern chat with the guy who tried to get past us, or play a prank back on the gal who messed with us last time. But should computers be equipped with such a mechanism? I really do not believe so. It is a slippery slope for a robot to play tricks on a human. Unless they are very advanced (such as body scanners that serve as lie detectors), there is little room for them to do anything but trust us.