Posts
Comments
Whether not there is a good definition of intelligence depends on whether there is a sufficiently unitary concept there to be defined. That is crucial because it also determines whether AI is seedable or not.
Think about a clever optimising compiler that runs a big search looking for clever ways of coding the source that it is compiling. Perhaps it is in a competitions based on compiling a variety of programs, running them and measuring their performance. Now use it to compile itself. It runs faster, so it can search more deeply, and produce cleverer, faster code. So use it to compile itself again!
One hopes that the speed ups from successive self-compilations keep adding a little. 1, 1+r, 1+r+r², 1+r+r²+r³ If it works like that, then the limiting speed up is 1/(1-r) with a singularity at r=1 when the software wakes up. So far software disappoints these hopes. The tricks work once, add a tiny improvement second time around, and makes things worse on the third go for complicated and impenetrable reasons. It appears very different from the example of a nuclear reactor in which each round of neutron multiplication is like the previous round and runaway is a real possibility.
The core issue is the precise sense in which intelligence is real. If it is real in the sense of there being a unifying, codify-able theme, then we can define it and write a seed AI. But maybe it is real only in the "I know it when I see it" sense. Each increment is unique and never comes as "more of the same".
A similar phenomenon arises in trying to bound the error of a numerical computation by running it using interval arithmetic. The result is conservative, but sometimes useful. However, once in a while one applies it to a slowly converging iterative process that produces an accurate answer. Lots of arithmetic leads to large intervals even though the error to be bounded is small.
I think I can answer the question about Ayn Rand. Turn away from her heroes and look at her villains. They seem realistic and scary. How did she manage it? Well, she left Russia in 1926, having seen the aftermath of the revolution up close and personal. So I guessed that she attained realism by drawing on her own real life experiences. I wasn't happy with this answer, because her villains were too redolent of the corporatist new-speak of the Heath-Wilson years, too Westernised for Bolsheviks. I knew little of the Lenin years, so I left this little puzzle on the back burner.
Recent financial turmoil has sparked interest in the causes of the Great Depression. The basic tale is of a monetary contraction of one third. Trade barriers made a small contribution. I was not happy with the basic tale because the quantity theory of money suggests that a monetary contraction should lead to deflation. After a dreadful couple of years, in which both prices and wages fall by a third, the economy should start working as before, running on a third less money. Why did the Great Depression last so long?
Enter the National Recovery Administration. Its job was to introduce regulations for price and wage controls to stop deflation. This leads to a situation in which Hoover doesn't understand why unemployment is 25%, but he does know that he has done something right because real wages of the still employed have risen. Whoops! The NRA stopped the economy working around the failure of the Federal reserve and introduced lots of police state style snooping to enforce price and wage controls, plus the inevitable corruption and arbitrage opportunities for those in the know with the pull to take advantage of the situation.
Ayn Rand comes to the US in 1926, aged 21. Glad to escape the soviet system. Three years later, the Crash. That is followed by Americans making extra-ordinary policy errors and the American economy going down the tubes as though John Galt were seducing the competent people away. Plus she gets to learn the NewSpeak used to justify those policy errors and defend them from criticism. She goes on to write her wishfulfillment fantasies, in which these events don't just happen, there is grand historical design behind them. Her villains are skillfully written. It is worth reading Atlas Shrugged, all 3^^^3 pages, to meet them.
Her personal history is fleeing halfway round the world, to escape the bad guys, only to have "them" catch up with her 7 years later.
This seems closely related to inside-view versus outside-view. The think-lobe of the brain comes up with a cunning plan. The plan breaks an ethical rule but calculation shows it is for the greater good. The executive-lobe of the brain then ponders the outside view. Every-one who has executed an evil cunning plan has run a calculation of the greater good and had their plan endorsed. So the calculation lack outside-view credibility.
What kind of evidence could give outside-view credibility? Consider a plan with lots of traceability to previous events. If it goes badly, past events will have to be re-interpreted, and much learning will take place. Well, people generally don't learn from the past. If the think-lobe's cunning plan retains enough debugging information to avoid going wrong and later going wrong again, that distinguishes it from what people usually do and gives it outside-view credibility.
Randomised controlled trials of medical treatments can be attacked on ethical grounds from both sides. They deny some patients medical treatments that is quite likely beneficial. They inflict unproven and potentially dangerous treatment on others. Both attacks lack outside-view credibility. We always think we know. The randomised trial itself has outside-view credibility. It will place us in the position that we can do the right thing without having to use our judgement or be clever.
"one man's modus ponens in another man's modus tollens."[1][2] is maxim that is easily weaponised by the Dark Side by taking it in a one sided way. One sees ones own implications as proving their consequents and the other sides implications as casting doubt on their antecedents.
On the question of blocking thoughts, may I offer a personal anecdote, conscious that readers of Overcoming Bias will read it heterophenomenologically?
Years ago, when my health was good, I had a Buddhist meditation practice of great vigour and depth. Sitting on my cushion, noticing my train of thought pull into the station of consciousness, refusing to board the train and watching the thoughts leave, I would become more and more aware that it was the same old crap coming round again and again.
Forcibly stopping my thoughts had always worked badly. I coined a meditation slogan encapsulating what I had learned: When thoughts spin round in your head, like the wheels on a bicycle, don't apply the breaks, just stop peddling.
There was little pleasure to be had, peddling away, only to see the same old crap coming back into view yet again. No peddling. No thought.
That was bloody scary. I was an intellectual. All these clever thoughts? They were me, my identity, my core. Without them, who was I? Did I still like cats? Did I still like music?
I needn't have worried. After I few days Mara noticed that my mind was calm and free from distraction. Did He concede defeat, admitting that another human had gained enlightenment and slipped from his grasp? No, ofcourse not. I had seen through the old familiar crap, but it was crap and there was no problem about improving the quality. I had learned to resist the temptations of low quality distracting thoughts, but all that happened was that my mind came up with more creative, more clever, more insightful, and more distracting thoughts.
Soon I was caught up in them, back to business as usual.
I see a secular moral to this tale. If you want more insightful and creative thoughts all you have to do is stop recycling the usual crud. You would guess that withdrawing your mental energy from the pumps that circulate the usual shit round your head would leave an empty silence, but the mind doesn't work like that.
Study this deranged rant. Its ardent theism is expressed by its praise of the miracles God can do, if he choses.
And yet,... There is something not quite right here. Isn't it merely cloakatively theistic? Isn't the ringing denounciation of "Crimes against silence" militant atheism at its most strident?
So here is my idea: Don't try to doubt a whole core belief. That is too hard. Probe instead for the boundary. Write a little fiction, perhaps a science fiction of first contact, in which you encounter a curious character from a different culture. Write him a borderline belief, troublingly odd to both sides in a dispute about which your own mind is made up. He sits on one of our culture's fences. What is his view like from up there?
Is he "really" on your side, or "really" on the other side. Now there is doubt you can actually be curious about. You have a thread to pull on; what unravels if you tug?
Wearing my mechanical engineer's hat I say "Don't be heavy handed.". Set your over-force trips low. When the switch is hard to flip or the mechanism is reluctant to operate, fail and signal the default over-force exception.
You can always wiggle it, or lubricate it and try again, provided you haven't forced it and broken it. For me, trying is about running the compiler with the switches set to retain debugging information and running the code in verbose mode. It is about setting up a receiver down-range. Maybe the second rocket will blow up, just like the first did, but at least I will still be recording the telemetry.
I think that Plan A will be stymied by Problem Y, but I try it anyway, before I try to solve Problem Y. My optimistic side is hoping Problem Y might not actually matter, while my pessimistic side thinks Problem X is lurking in the shadows, ready to emerge and kill Plan A whether I solve Problem Y or not.
I try in order to gain information.
It is usually important to procede with confidence. When things go wrong they throw off fragments of broken machinery and fragments of information. Suprised, we fail to catch the flying fragments of information, and must try again, forewarned.
Two meanings of the word "try" fight for mind share.
To try: to position oneself in the right spot to catch the flying fragments of information flung out from failure.
To try: The psychological mechanism that lets us fail through faint-heartedness, again and again, but never quite understand why.
Two meanings sharing a word is a common problem with natural language. The particular danger I see for Eliezer is when the second meaning hides the first.
He says he isn't ready to write code. If you don't try to code up a general artificial intelligence you don't succeed, but you don't fail either. So you can't fail earlier and harder than you ever expected and cannot suspect that the singular is far. If you won't try, you'll never know.
The interesting question is "What would it be like if we lived for 700 years instead of 70 years?". This is more interesting than contemplating immortality because it pries open the issue of scaling. What changes by a factor of 10? What changes by a factor of 100? What changes by a factor of 3.162?
Presumably acient towers get a straight ten times less impressive.
Speed limits would be set much lower. You lose ten times as much when you die in a car accident so you would be willing to spend more time on your journey to avoid that.
An academic could go into his subject ten times as deeply, or study ten different subjects. A longer life raised interesting questions about the depth versus breadth trade-off.
Proportional scaling suggests that we would study history ten times as long, but would we? We might no longer feel the need, because we would have so much more personal experience. Oppositely we might feel that we will have to live with the consequences of avoidable error for so much longer that we were willing to spend a greater proportion of our time learning the lessons of history.
When someone sets out to write an atheistic hymn - "Hail, oh unintelligent universe," blah, blah, blah - the result will, without exception, suck.
I've just submitted my Militant Atheists' Marching Song to Reddit. Does it suck? Since I wrote the words, it is not my place to judge.
Dagon made a point about the social importance of guarantees. If a promise is broken, you know you have been cheated. If you are persuaded that there is only a 10% chance of losing your investment and you are unlucky, what do you know?
I doubt that we can over come our biases by focusing on what they are bad for and how they hurt us. We also need to think about what they are good for and why we have them.
Rather that estimating a probability, it would have been more interesting to ask "What emotional need are you trying to meet with this?"
If Mr Still-a-chance yearns for "souls go to heaven and meet God" why does he care about evolution? Isn't the soul the magic, special sauce that converts an ordinary animal body into a human? How does denying evolution help him?
Meanwhile, 20000000 years in the future, a multi-generation interstellar space ship has set up a colony on a distant planet with existing biology. The colony collapses but man does not go extinct, and 100000 years later they have re-established a civilisation of sorts.
They find that man is not an animal. His biology is entirely distinct. Which goes well with their myths of a double fall, from the sky to the ground and from the golden age to barbarism, but what really do they gain when they find that they do not have genealogical ties to the animals around them. Why is our far future Mr Still-a-chance the 2nd so pleased?
I find myself unable to imagine how Mr Still-a-chance would have answered, which piques my curiosity
Perhaps this post needs to be rehosted at http://www.sufferingfrombias.com for it gives no suggestion or hypothesis about overcoming bias. Here are three.
ONE: Friendships with people from different cultures helps one to realise that stuff one was brought up to accept sounds deeply weird to those who first encounter it as adults.
TWO: There are tells: little warning glitches. The trouble is that from the inside the tell doesn't make sense, but human memory depends on embedding items in networks of meaning, so the tells will not embed and get forgotten. An example from my personal life is reading about the recent shooting spree in Finland that left eight dead. On comment on this was that it was a big shock because the standard Finish murder involves a man going on a fishing trip with his best friend, knifing him during a drunken row, and, overcome by remorse, turning himself in.
I found the standard Finish murder incomprehensible. I could never imagine doing anything like that. Recently I've come to realise that I'm the odd one out. Perhaps I have an attachment disorder. I'm terrified of rows with friends because I see relationships as fragile, easily destroyed by a petty quarrel, and the breaking of affectional bonds as unbearably painful. (Notice that my beliefs are internally inconsistant: if breaking affectional bonds is as painful as I fear, my friends are clearly not going to inflict that pain on themselves by dropping me over a petty quarrel. Whoops!)
Here is the general interest bit: rows within marriages, love affairs, and close friendships, are a well known phenomenon (is this the understatement of the month?). So I've spent 30 years noticing this and forgetting it without realising the implications, noticing and forgetting, noticing and forgetting, over and over, without ever holding it in my awareness for long enough to say "Hey! Wait a minute..."
Suggestion two is to keep a little black book of the stuff that keeps getting dropped on the floor. So it one was brought up a Christian one might notice when a daughter or fiancee unexpectly falls pregnant this never raised hopes of a second coming or a new relevation. After a while one realises that one never follows this thought anywhere, even though there is something odd about it. Into the black book it goes.
Later you can look through the book and start trying to worry out what is so slippery about the thought. Write down the stuff that doesn't make sense so that it has a chance against the much more easy to remember stuff that fits into your world view.
THREE: Fine distinctions of sarcasm. When one of one's favorite views is attacked by a sarcastic comment it sometimes irritates and it sometimes stings.
Sometimes the humour depends on an oversimplification. Perhaps the humour priviledges the ex post perspective over the ex ante perspecitive. This irritates because the attack is short and funny and the repost is long and dull, and yet ex post is not inherently better than ex ante, the argument is being decided by structural matters independent of the merits of the case.
Othertimes the humour lies in the fact that the sarcasm cuts through the bullshit and the defense mechanisms and pierces to the heart of the matter. This stings. The only defence is to remember all the worthless sarcasm that depends on distortion and irritates rather than stings and to pretend that this sting is also an irritation.
There is an interesting bias in the way that we approach these issues. We see ourselves as the prisoners of the system not the jailors. Or maybe I should say that we don't like to admit that we are lords as well as serfs.
One political response to the problem of lost purposes is to focus on government coercion which leads to libertarianism. I don't think that we should rush so quickly to a conclusion. First we should examine the origins of the problem more closely.
The key concept is contestability: is there an alternative BA course you can take that does not require mastery of 12 century knitting patterns. This not lead automatically to free-marketry. One can image contestible socialism.
Think about driving licences. Why does there have to be a single agency that issues them. The government could fund two, splitting the money in proportion to the number of licences issued. If queues are too long at one office, drivers will go to the over. It gets rather interesting if the competing offices get to set different driving tests. Do you go for the easy one or the hard one. Perhaps drivers who have passed the hard one get cheaper insurance rates. Perhaps that also tells you something important about the extra preparation being personally worth while, not just a money saving measure.
My favourite example is prison regimes. What do we expect from prison? We could set up a Prison Audit Agency that follows up prisoners 5 years after release to see how they are earning their money. There are many possibilities: professional criminal, dead, junkie, wino, deadbeat, honest employee, honest sole-trader, honest employer.
Imagine that the rehabilitation figures got the same kind of political attention as the crime figures. It would open the door to randomised assignment to different regimes. The hangers and floggers could run one prison, the hugs and therapy crowd another.
My guess is that neither really knows much about rehabilitation, a point that would become very clear a decade on as the figures are disappointing for both regimes. Both groups would be humbled and forced to think harder about what they were doing least their rivals responded more wisely and eventually built up a commanding lead in the results.
School vouchers are another example. Most state supported school systems are designed as pay-twice systems. If you pull your child out, you continue to pay taxes as before, but do not get the money back. You must find a second tranche of funds to pay your child's school fees. So schooling is only weakly constestible.
The curious point is that state power is weakly contestible, leading to poor delivery of services to the voter, who wearies of his political leaders and votes them out of office. Why do politicians favour a system that costs them office?
As we follow politics we see that reforms that make public services contestible, even when they remain in the state sector, are bitterly contested by civil servants. And not just public services. Trade barriers are about stopping the consumer from buying Toyota instead of General Motors, because the auto workers don't want their claim to be building wonderful cars to be contestible.
The 90% of us who are above average will do anything to keep the sunshine of contestibility from illuminating our particular patch line of work and causing our illusion to crumble to dust.
I don't see any hope for a direct attack on our tendency to lose our purposes. We like it that way and will fight to keep it like that. More insidiously we will always see ourselves as serfs not lords. Is there a name for this bias, the one that stops us admitting to ourselves that we are doing it to ourselves?
X3J13, the ANSI committee that standarised Common Lisp, had many problems to solve. Kent Pitman credits Larry Masinter with imposing the disciple of seperating problem descriptions from proposed solutions and gives insights into what that meant in practise in a post to comp.lang.lisp
The general interest lies in that fact that the X3J13 Issues were all written up and are available on line.
http://www.lispworks.com/documentation/HyperSpec/Front/X3J13Iss.htm
or
http://www.lisp.org/HyperSpec/FrontMatter/X3J13-Issues.html
so if you wish to study how this works there is a resource you can analyse.
I should confess that my interest has been in content not process. I have been reading these issues to learn Common Lisp. Are these pages really a useful resource for scholars wishing to study the separation of problem descriptions from proposed solutions? I don't know.
Tom, I think we are actually agreeing. I'm arguing that if you already know the situation is complicated you cannot just appeal to Occam's Razor, you need some reason specific to the situation about why the simple hypothesis should win.
You are proposing a reason, specific to economics, about why the complications might be washed away, making it reasonable to prefer the simpler hypothesis. My claim is that those extra reasons are essential. Occam's Razor, on its own, is useless in situations known to be complicated.
Do high level descriptions of the world frequently account for most of the variance in high level phenomena without containing the known complexity of the substrate?
I think you can constrast thermodynamics and sociology by noticing that there is no Princess Diana molecule. All the molecules are on the same footing. None of them get to spoil the statistics by setting a trend and getting in all the newspapers papers. So perhaps Occam's Razor grabs credit not due to it, as researchers favour simple theories when they have specific reasons to do so.
An example of the mis-use of Occam's Razor arises in discussion of the question of whether minimum wage laws cause unemployment. Many people think they do and it is reasonable to imagine a politician finding an increase in the minimum wage to be politically necessary even as he wonders how to dodge blame for the subsequent rise in unemployment that he believes will follow. He will likely look to timing, seeking to delay the increase until there is a good chance of a tightening labour market raising wages.
How can you do empirical research on the effect of minimum wage laws on employment when practical men are scheming to conceal the very effect that you are looking for? One way is to appeal to Occam's Razor. Let us prefer the simpler hypothesis that increases to the minimum wage are random. That is bogus. We already know of the politicing and scheming that goes on. If our research methods cannot accommodate it, they leave us in the dark and Occam's Razor does not light our way.
Occam's Razor has two aspects. One is model fitting. If the model with more free parameters fits better that could merely be because it has more free parameters. It would take a thorough Bayesian analysis to work out if it was really better. A model that fits just as well but with fewer parameters is obviously better.
Occam's Razor goes blunt when you already know that the situation is complicated and messy. In neurology, in sociology, in economics, you can observe the underlying mechanisms. It is obvious enough that there are not going to be simple laws. If two models fit equally well, you just don't know, even if one is simpler than the other.
The "quant" trying to make money on the financial markets can take a modelling approach and may find the Razor sharp, but the scientist, trying to get to the bottom of things, has little reason to go for an explanation simpler than the known complexity of the underlying mechanisms.
The Vapnik Chernovenkis Dimension also offers a way of filling in the detail of the the concept of "simple" appropriate to Occam's Razor. I've read about it in the context of statistical learning theory, specifically "probably approximately correct learning".
Having successfully tuned the parameters of your model to fit the data, how likely is it to fit new data, that is, how well does it generalise. The VC dimension comes with formulae that tell you. I've not been able to follow the field, but I suspect that VC dimension leads to worst case estimates whose usefulness is harmed by their pessimism.
Tiiba, I've always considered this bible quote
"I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all."
to be a paradigm of bad writing. The nasty trick it exemplifies is using a paradox to get one up on the reader without committing to a specific meaning.
If the race is not to the swift, who does win? The lucky? Contrast two aphorisms "the race is to the swift" and "the race is to the lucky". The second of these is useless. You cannot assess some-one's luck ahead of time, you have to discover who is lucky by waiting to see who wins and then say smugly "see I told you the lucky one would win."
This passage irritates me because the writer forces the reader to create the meaning of the passage from nothing and then the writer appropriates the readers efforts. There is something important to be said on this theme:
The race is to the swift and the battle to the strong, but the odds are never quite so short as the book maker offers and the greatest number are drowned when the sea takes the unshinkable ship with no lifeboats.
I've no doubt that the author of Ecclesiastes would say "that's it, you have got my meaning exactly" but that would be a lie; it is my meaning, I said it and he didn't.
[rhetorical pose] We shouldn't balance the risks and opportunities of AI. Enthusiasts for AI are biased. They under estimate the difficulties. They would not be so enthusiastic if they grasped how disappointing progress is likely to be. Detractors of AI are also biased. They under estimate the difficulties too. You will have a hard time convincing them of the difficulties, because you would be trying to pursuade them that they had been frightened of shadows.
So there are few opportunities which are likely to be altogether lost if we hang back through unnecessary fear. [/rhetorical]
Well, I happen to believe the two paragraphs above, but distinct from the question of whether I am right or not is the question of whether the phrase "We need to balance the risks and opportunities of AI." means something or whether it is merely an applause light.
I think it is trivially true that we need to balance the actual risks and actual opportunities of AI. There is room for disagreement about whether we need to balance the perceived risks and perceived opportunities. If perceptions are accurate we should, but there is scope to say, for example, that the common perception is wrong and a rogue AI will in fact be quite stupid and easily unplugged. This opens the way to a decoding of language in which
o We need to balance the risks and opportunities of AI.
is the position that we are assessing the risks and opportunities correctly and
o We shouldn't balance the risks and opportunities of AI.
is the position that we are assessing the risks and opportunities incorrectly and should follow a different path from that indicated by our inaccurate assessments. Such a position needs fleshing out with a rival account of the risks and opportunities.
One question that I dwell on is "how do intelligent and well-intention persons fall to quarrelling?". The idea of an Applause Light is illuminating, but I think it is also quite tangled. There is the ambiguity between whether a phrase is an Applause Light or a Policy Proposal. I suspect that the core problem is that it is awfully tempting to exploit this ambiguity rhetorically, deliberating coding ones policy proposals in language that also functions as an Applause Light so that they come across as obviously correct.
The fun starts when one does this subconsciously and some-one else thinks it is deliberate and takes offence. Once this happens there is little chance of discovering the actual disaggreement (which might be about the accuracy of risk assessments) for the conversation will be derailed into meta-conversations about empty phrases and rhetoric.
There is an awful lot of history. Preliminary to whether we imagine the past vividly enough for it to carry proper weight, we must select a cannon of ``important'' events to which we turn our attention.
In a recent thread on Reddit: http://reddit.com/info/2k77b/comments/c2k80o
I drew attention to Argentina because the story of Argentina's 20th century economic disappointments jars uncomfortably with the cultural tradition in which I swim. I swim in a cultural stream in which the misfortunes which may befall a country live in a hierarchy. At the top are the bad misfortunes, losing wars, and fighting wars. Somewhere near the bottom are petty misfortunes: many countries are under the thumb of absolute rulers and if the caudillo retains power by pursuing popular policies then his rule is not so bad.
I know little of Argentinian history and understand it even less. What little I know threatens my hierarchy of misfortune. It looks as though well meaning but economically unsophisticated absolute rulers are the top misfortune. They are much worse than wars, which are intense, but brief.
I want to overcome my bias by learning about Argentinian history. I find myself struggling. There is a standard way of looking at recent history with Hitler, Stalin, Mao, Great War etc. I notice that I'm very dependent on social support and just get sucked into that looking at history from that point of view because it is the common one.
So there is a second sense in which History may or may not be available. Frist it is important to feel the force of history sufficiently strongly. But this could make things worse if we cultivate our feeling for a limited selection of history, chosen to support our standard narratives. The second requirement is for breadth, and this is very difficult if the people around you aren't interested.