Posts
Comments
:-) Maths is the product of the same abstracting mechanisms that create all our visions of the world. As such, maths has no more or less validity than any other of our self-consistent constructs of reality, and it is no accident that our maths has applications in our real world models. They're the products of the same mental systems. What is depressing is when a mathematical model which represents 5% of the available data is worshipped because it has internal coherence. As in Aumann's model. Tara.
Caledonian, why do you still care ? Cover the whole thing with an SOEP field. As for me, when I read these guy's maths, I'm awestruck. I don't have their maths. But when I read their non-math papers, I think..... where's the judgement ? Where's the common sense ? Think of Aumann. He got his (pseudo-)Nobel prize for creating a mathematical proof that honest Bayesian arguers could never disagree. Unfortunately, Aumann's proof had ABSOLUTELY NOTHING to do with real people or real situations. That didn't stop him getting his gong from people who admire maths for maths' sake. The bias is that maths has somehow something to do with truth or reality. But somehow, I don't hink I'll see Eliezer or Robin addressing that bias here. Move on. Leave them to it.
Kip : The Newcombe problem only needs about 30 seconds thought: as soon as you've postulated reversed causality, any reasoning based on the premise 'there's 1m€ in box B at the moment of decision' breaks down on the meaninglessness of the notion 'at the moment of decision' under reversed causality. Are all 'philosophical paradoxes' so trite ? At least I suppose while people are 'exercising their thinking' over such trivialities they're not doing us serious harm by working on self-improving AI.
Come to think of it, there was a system which held the rank and file to be the employers and the politicians to be employees. It was called Marxism.... Must check up how it worked out.
Between the post and the comments we have a slippage from : a) the human tendency to sort ourselves into 'us' vs 'them', presumably for reasons which had selective advantage (group solidarity and heightened stimulation levels) b) our capacity to keep the positive aspects of a diluted form of this tendency, without having to pay the price of all out warfare, by choosing (deservedly highly paid) sports teams to be our 'champions' (in the sense of the word where a 'champion' was designated to represent a warring group in single combat) in facing 'them' c) the transfer of this 'champion' role from sports teams to elected politicians, typified by the Blues & the Greens d) the confusion between the 'champion' role and the 'delegate' role, to which I could add the 'mandated' role, in our actual political systems. e) then all the usual mutterings about politicians.
OK so we're tribal, and IMO we're confused in what we want from our politicians. So what ? Where do we go from here ?
Eliezer suggests a further development in the theory of democratic government, that of considering our elected representatives as 'employees'. I disagree. The role of employer supposes an autonomously chosen set of strategies which it is imposed on the employee to execute. How do you get to set the strategic agenda without first being a politician (or, better, a politician's 'éminence grise'. Or spouse) ?
Between the post and the comments we have a slippage from : a) the human tendency to sort ourselves into 'us' vs 'them', presumably for reasons which had selective advantage (group solidarity and heightened stimulation levels) b) our capacity to keep the positive aspects of a diluted form of this tendency, without having to pay the price of all out warfare, by choosing (deservedly highly paid) sports teams to be our 'champions' (in the sense of the word where a 'champion' was designated to represent a warring group in single combat) in facing 'them' c) the transfer of this 'champion' role from sports teams to elected politicians, typified by the Blues & the Greens d) the confusion between the 'champion' role and the 'delegate' role, to which I could add the 'mandated' role, in our actual political systems. e) then all the usual mutterings about politicians.
OK so we're tribal, and IMO we're confused in what we want from our politicians. So what ? Where do we go from here ?
Eliezer suggests a further development in the theory of democratic government, that of considering our elected representatives as 'employees'. I disagree. The role of employer supposes an autonomously chosen set of strategies which it is imposed on the employee to execute. How do you get to set the strategic agenda without first being a politician (or, better, a politician's 'éminence grise'. Or spouse) ?
Politicians are the Hated Enemy today ?
Q : Why is everyone linking together cryonicism, life-extensionism, trans-humanism, and the singularity ? In addition to Caledonian's irritability, I would add : A : Because the two main posters here seem to subscribe to the extreme desirability of all three, (counting trans-humanism and the singularity for one item translating as Self-Improving AI), in a nexus centred on the Singularity Institute.
personal take : a) Cryonics : couldn't care less b) Radical life extension : playing with fire c) Self-improving AI : burning the house down.
Just read 'The Reversal Test'. A good, honest, decent paper, but does little to address the issue. It only considers modifications in one parameter. I'd like to see a reversal test for modifying one parameter out of 100, when the 100 parameters are in some sort of equilibrium, potentially unstable, and the equilibrium is one which you don't understand too well. Even given that the status quo equilibrium is by all accounts pretty lousy.
Thanks Eliezer. My previous post on the 'CounterCultishness' thread would have been more relevant here. This is a good opportunity to give you, Robin, and any other occasional posters a vote of thanks for your (always) thought provoking and (mostly :-) ) incisive posts, whose interest keeps me, at least, coming back here, whatever my feelings about the SI.
Not sure why EY redefined the debate in terms of cultishness. Was anyone under the illusion they were being asked to pack their things for Guyana ?
Doubts about the objectives of the SI arise more from the seeming contradiction between the professed rationality of its members (Bayesian rationality, weighing the risks, putting all the 'Friendly' safeguards in place etc.) and the passion with which in their writings they seem to hail the Singularity and radical life extension like the Second Coming. Which leads one to fear a certain bias. Fear only, mind you. My slovenly and inadequate heuristics don't push me into a superhuman effort to get involved.
BTW, the very abuse of the term Bayesian, except humouristically, is in itself worrying. It's only a statistical method for Chrissake. Very useful in well defined scientific investigation, of no use at all in areas where the priors are (a) innumerable (b) inestimable, like, in all areas in the 'humanities'.
BBTW : The word 'Singularitarianism'. Any word ending in '-arianism' denotes a belief system, no ? So using that word does indicate that its users have gone beyond the domain of ideas and are in the domain of beliefs.
OC, in the immortal words of Paul Simon : "A Bayesian estimates the priors he wants to estimate "And disregards the rest... "Lih lih lih........."
By that time everyone knew it was time to leave, they had seen the lights repeatedly dimmed, but they were comfortable in the hall, and as long as no individual could be blamed for the antisocial act of staying, they would do so. Nevertheless their discomfort level was rising. Your action precipitated the decision, like seeding a supersaturated solution precipitates crystallisation. It's another example of an unstable group equilibrium just waiting to be disturbed, like the lonely dissenter in a group where the majority have private doubts. If the lights hadn't previously been repeatedly dimmed, the group might well not have followed you.
I've always wondered, since I was very small, why 'The Emperor's New Cloths' as commonly told doesn't include the scene where the Emperor has the Imperial Guard clear the street with a sabre charge.
'This may come as some surprise' to Asch & Aumann, but rationality is not the design point of the human brain (otherwise this blog would have no reason to exist), getting by in the real world is. And getting by in the real world involved, for our ancestors through tens of millenia, group belonging, hence group conformity. See J. Harris, 'No Two Alike', Chaps. 8 & 9 for a discussion which references the Asch work. This does not mean of course that group conformity was the only adaptation factor. Being right and being 'in' both had (and have...) fitness value, and it's pefectly natural that both tendencies exist, in tension.
There's a lot of confusion here. 1) Don't confuse respect for religion (unreasonable) with respect for people who have deep religious beliefs, however daft. In some abuse of religion I sense a lot of contempt for religious people. I try to fight my contemptuous side, knowing how strong it is. 2) Don't confuse 'the harm done by religion' with harm done by people, who would have done it anyway , who find in religion a convenient cloak. 3) This is not the place for a post on the human need for religion or the rag-bag of needs it subsumes (social, political, historical, personal identity definition, ethical, the love of the marvellous, transcendental etc.). However, I strongly suspect that some of those same needs might not be a million miles away from the motivations that attach people so strongly to the aims of a certain Institute..... Saul/Paul was not the first nor the last human to have radically changed his beliefs while maintaining the underlying personality structure which drove him to give himself so totally to the first set, then to the second. And to found his own personal religion, but that's another story.
It's illuminating to see this post next to the one on procrastination. I doubt Musashi would insist on delaying your sword stroke until you were absolutely sure you would cut at the same time as parry. His perfectionism concerns the initial state of mind, not the outcome. Raising the prior, in other words.
To the following phrase : "You can't hate someone while laughing at his foibles" I should of course have added that you may, however, get a sense of reclaiming the human high ground in what might otherwise be situational inferiority.
Scott Adams' jokes about pointy headed bosses are 'release of tension' jokes : the tension that arises from having to live with the species. You could call it, being constrained to live in absurdity. In that sense, some say they serve rather to avoid the phb becoming a hated enemy. You can't hate someone while laughing at his foibles. I guess that is the distinction, we're laughing at the phb's absurdity, not at his discomfiture. There is no such tension with a co-worker, hence no joke.
Robin, you're right, most people do think economics is a cult, even though there may be a small proportion of usefulness in the teachings... characteristics are, cult members cut off from contact with non-cult members (in this case by the ignorance of the non-cult members, of course), devotion to the cult leader (Keynes ! Friedman ! the Gourd! the Sandal!), proclamations of infallibility (the market is infallible), progressive alienation (this is a science, I can believe six impossible things before breakfast), and ending in total learned helplessness (for instance, when a team of six beauticians, or whoever it happens to be this week, outperform the nation's best fund managers yet again...). Only teasing, but I was just reading some very old threads and came across one where you professed surprised at relative levels of acceptance of announcements in economics and physics, and am still suffering from vertigo. Happy Christmas !
Robin, transmission of expertise in non-rational domains has to rely on authority rather than argument, so is more susceptible to slide into abuse of authority than transmission in rational domains. The original post here is strange in that it supposes such a type of transmission in the field of rational teaching. The definition of cult in the field of master / disciple relationships has to start with an examination of whether authority is being abused by, for example, being exercised in areas unrelated to the teaching. Don't take sweets from philosophers.
That lastr one got through, so let's try : Random malfunction ?
Trying to work out the biases of the new antispam filter. Frequency of comments from same individual in same thread ?
Just had a response to Goplat rejected as spam. Wonder what the biases built in to the new antispam filter are ?
Goplat, can't answer for Caledonian, but as I'm pretty sad & pathetic myself, I'll take a stab. The unborn represent variety and potentiality. More of the same represents sterility. Sure I'd like to live 500 productive & happy years, but am in my better moments conscious that with present biotechnology this is unlikely. With SIAI improved biotechnology who knows ? However, my totally uninformed intuition is that however superproductive & longlived the ultra-new curly-wurly chromosomes that my friendly neighbourhood SIAI will give me are, they would do better (in accordance with their interest) endowing them on the young of the species. Your argument that we now are happy living 80 years where our ancestors were lucky to make 40 is pertinent, but adding years after 40 still doesn't increase the productive lifespan of a mathematician. Jesus died at 30 (or was it 33 ?). Mother Theresa was doing productive caring work into advanced old age. So perhaps youth = creativity, age = caring. A 'Self Improving' AI would surely privilege the 1st option. For better or for worse. Personally I'm for balance, and am all for the increase of life expectancy at a rate which is compatible with human capacity to adapt. I wrote a piece on the Impossibility of a 'Friendly' SIAI which I may inflict on the world someday.
Couldn't resist adding a complaint about the abuse of the term 'guru' as a term of ...abuse. It represents in fact an exponent of a perfectly respectable form of expertise transmission in non-rational domains. Drift into abuse of authority by such an exponent is perhaps more likely because the method relies on authority rather than argument, but that doesn't mean that the concept is invalid, or indeed that there is any other method possible in those domains.
What about the Guru who wrote 'Why work towards the Singularity' ? It is a text with a distinctly Messianic feel. Or, to be more generous, a Promethean feel. While it is true that Hom Sap has a nasty itch to create anything that can be created, regardless, thre's no need for such pseudo valuations as the following : "If there's a Singularity effort that has a strong vision of this future and supports projects that explicitly focus on transhuman technologies such as brain-computer interfaces and self-improving Artificial Intelligence, then humanity may succeed in making the transition to this future a few years earlier, saving millions of people who would have otherwise died. Around the world, the planetary death rate is around fifty-five million people per year (UN statistics) - 150,000 lives per day, 6,000 lives per hour. These deaths are not just premature but perhaps actually unnecessary. At the very least, the amount of lost lifespan is far more than modern statistics would suggest." Who says that continuing the lives of us dull old farts, to the inevitable detriment of the unborn, has any positive value ? I'd say that's monstruous. The transhuman AI may be an unavoidable consequence of our Luciferian inclination to meddle. That doesn't mean it's a cause. Any chance of it becoming a cult ?
Robin : "For example, "ostracizing anyone who dared contradict her" would seemingly apply to a great many, perhaps the majority, of ordinary human organizations." : Yes, but there is a difference between ostracizing = damning to the nethermost pits of hell with no hope of salvation and ostracizing = delaying your next pay increase by a couple of months. i.e., the cult-dom-ness is contingent on the existential nature of the ostracization.
EY, thanks for the link in this post to your Global Risks paper. It addresses in passing something that had puzzled me : how would an AI acquire hands ? On Ayn Rand : the concept of purity seems to me central in cult formation, more so than that of absoluteness. See, for instance, the deviations of the Self Realisation Foundation in handling Yogananda's legacy. Or, for that matter, General Jack D Ripper in Dr Strangelove. So, let your knowledge and wisdom increase, but let them not be pure. Amen.
"an "environmentalist" is not someone who believes in the existence of the environment." Non sequitur. An environmentalist is someone who believes in the value of the environment. sloppy, sloppy, sloppy.......
It's amusing to see 'criterion of goodness' as a simile for 'criterion of correctness'. The Inquisition believed they were both 'correct' and 'good'. In torturing you, they were saving your soul, which was, for them, the ultimate in Utility. So, in calculating utility, beware of your assumptions.
Having only just caught up with the Paris Hilton thread I've only just realised what Eliezar is trying to do and am suitably humbled. However, I choose the lottery thread to point up the unimaginable orders of magnitude of difference between the significance of trying to devise an optimal morality for the engineered intelligence which will supercede us (and yes, I do know that the etymology of supercede does include our death), and the significance me and my better half buying a lotto ticket. 'Wasted hope' implies that we are to some extent free agents. Before even going there, Eliezar, you need to define your position on free will vs determinism & Chalmers vs Dennett. No doubt you have, in which case please excuse me and point me there. To answer the lotto question, just look to how your post singularity AI will handle frustration, disappointment, and low self esteem. I don't have the math but I do have the questions. Our ability to handle our own dysfunctions is not even in its infancy. Our psychological models are a shambles (just look at the Tree of Knowledge as a smile- or tear- inducing example of how not to get there). Our therapeutic methodologies are at the shamanism 1.0.1 stage . And yet we hope to legislate for the intelligence that will replace us ? Call that a bias, a triumph of hope over experience ! Next step, the Paris Hilton discussion on values was suitably learned, but however high you get in meta- meta- meta- values theory, there is an irreducible 'my values are what seems right to me'. Your post-singularity IA will have its own, unless it is very severely constrained (but then I guess it wouldn't be post-singularity, in which case we should all go to the beach and shut up, because nothing we could do or say will make any difference). That's why I like Ian McDonald's book, it focusses on that polarity. BTW, I agree with the poster who postulated creepiness as a value. Cryogenics is definitely creepy. Also, please get in touch when you've produced an AI program to match the smile on my wife's face when she comes in with a Lotto ticket and says 'this is for you', and the effect it has on me even though I know all the probability statistics. Tara.
I strongly encourage any AI worker who hasn't already done so to read Ian McDonald's 'River of Gods'. He's pretty positive (in timescale terms...) on AI, his answer to the question "How long will it be until we have human-level AI?" is 2047 AD, and it's a totally gob-smacking, brilliant, read.
Nick, sure, heroically not doing something will never grab the attention in the way that doing something does. Today, approximately 1,000,000 cars in Paris were not burned. So what makes the headlines ?
Statistics is actually fun, as the notion of probability is so non-intuitive. There's a 1 in 6 chance of throwing a deuce. What does that mean in the real world ? Well, if I throw the die 6 times, it should come up once ? euh no... Well if I throw 100 sequences of 6 throws I can predict the number of times the deuce will show up ? euh, no.... Well, if I throw 1000 runs of 100 sequences of 6 throws...... sorry, you still don't know one damn thing about what the result will be. So what does probability mean ? It's great ! One of life's rich prizes is to watch someone making a prediction on a particular instance based on statistical reasoning.
BTW, significant data was withheld in the examples given : a) how many dips do you get at the jellybeans ? Do the red ones taste better ? What is their market value with the current weak dollar ? b) 10,000 people overall or 10,000 infected people ? Degree of infectiousness of the disease ? But that's what the affect heuristic is for : taking decisions in situations of incomplete data. 150 people is a single bounded set, 98% of x people sounds as though it just might be a replicable set. Go for it.
Ha, Spock vs McCoy. I think Kirk's position was that it's the affect heuristic that makes us warm, cuddly, and human, data processors, even if it can be faulted in some artificial situations.. This ties in with the other thread about how far we look down possible chains of results in deciding on an action. We're wired to look to proximal results with high affect, and I'm all for it.
Hi Richard, any relation to the punch card guy ? IBM paid my salary for 35 years. Someone in one of these threads got squashed flatter than a pancake for supposedly confusing maps and territories, so let's be careful with models of reality. When I say 'dependant on usefulness', I just meant that the selectivity and level of detail of the map would depend on what you want to use it for. Not much point in going to the doctor and telling him the 'truth' about my finger, which would involve energy fields and dark matter, if what I want from him is a sticking plaster. Lovely article here on what the Romans thought was important in a map, and why it doesn't look like one that we'd find useful, or 'truthful', today. http://news.bbc.co.uk/2/hi/europe/7113810.stm
I'm left in 'awe and wonder' at the literalism of the debates going on here. The OT is a bunch of mythology and folklore, so, what else is new ? The NT is a heterogenous collection of Roman imperial propaganda, Jewish apocalyptic propaganda, and perhaps, some vague recollections of what a good man once said. So ? What does any of that have to do with logical categories ? Eliezar is guilty, as Anna pointed out, of mixing up the crudest OT literalism with any and every other level of religious experience and expression. I understand that, he was traumatised at age 5. Perhaps that also explains the violence of his reaction to Anna. The only interesting debate on the 'singulsrity' of religion is exactly the same debate as that on the 'singularity' of consciousness. Either there is a 'watcher', in the void, behind all thought and image, which constitutes the irreducible core of my consciousness , as for instance Daniel Dennett would not agree, or there is not. If there is, then there is a basis for religion. If there is not, then there is a basis for saying that we will never know final causes nor final intents, and what the hell.
'Scuse me, but isn't this trivial ? Both pragmatic and epistemic instances depend on available information. If you drive to Carrefour, you need some information to tell you they're out of chocolate. And to see the 'Out of chocolate' sign, you need to have driven to Carrefour. So, dear friends, both instances depend on (a) purpose (b) information relative to the achievability of the purpose. Unless of course your purpose is 'enculage des mouches', in which case, don't go to Carrefour. Go to Tesco. PS Truth does not reduce to usefulness. Truth is a relative concept dependant on usefulness. I asked Schrodinger's cat to contribute but she was busy with her Whiskas.
Forgive me for not picking up on the irony of including corporations and nanodevices in the same sentence. Eliezer is obviously correct in that corporations don't evolve because they don't replicate. A childish wish to gloat has to be held in check so as not to name and shame all those 'child' corporations whose DNA is specifically contrary to their parents'. The anti-wish list for nanodevices, on the other hand, is relevant and necessary. However, it is also entirely superfluous, as we all know, thanks to Dr Denning, that we are in a deterministic universe and that 'Que sera, sera'. Sit back and enjoy the ride.
'Genocide' refers to intentions rather than consequences, but it seems to me just fine to have a National Native American Genocide Day to remind us that sometimes consequences should have been taken into consideration. Even if they weren't, which of course is another question. A bit like Iraq.. (oops !! No Damn !!!! I didn't mean to say that !!!). So let's have a nice polite debate on the Instrumental Values and the Terminal Values in the Iraq war.. I've looked hard but not found any leaky generalisations in the area.....
Eliezar, something of a 'rant' ? 'the people who invented the Old Testament stories could make up pretty much anything they liked'.... overlooking that we're talking about oral traditions committed to writing centuries later. Of course the domain covered by the books of the old testament covers law, social customs, and a whole bunch of stuff which is now the domain of other institutions. Of course ideas have moved on in most of those domains. I'd be more interested in reading your ideas about why the fears, insecurities, and identitiy issues so many of us face in an age of increasing change and complexity are leading to a 'back to the 17thC', 'back to the womb' type increase in clinging to both 'believing' and 'believing in' this particular dragon in the garage. This is not just a US phenomenon, we're seeing it in the UK also. Derision won't help, nor, most certainly, will logical argument.
mtc, you could make a few volumes of Dilbert your required reading, to inform your faith in the 'sane corporation'. In fact, in such high powered company, I'm surprised to see non-qualified discussion of the intentionality of any organisation. Organisations don't have intentionality, individuals do (perhaps... if we don't let the Evolutionary Psychology crowd take it away from us..). As all Eliezer's anecdotes illustrate, these get lost in the wash of large numbers and multiple levels, networks, and gridlocks, so we are left with the dross of the latest organisational mantra which everyone gives lip service to and no-one believes. It is trivial to observe that the dominant intentionalities will be those of the most powerful individuals, but these also are destined to be diluted and lost. However, my personal believe is that inadequate individuation (accepting cultural norms of need and intentionality) is a far bigger problem than such intentionality as we have getting lost. Let's first work out who we are and what we have to say before worrying about our voice getting lost.