Posts
Comments
I doubt Eliezer - champion of truth and science - would permit himself artistic license with this sort of thing. I think it is more likely that this is a genuine mistake on his part.
In chapter 104 we have this: "Harry had refreshed the Transfigurations he was maintaining, both the tiny jewel in the ring on his hand and the other one, in case he was knocked unconscious". The other one was Hermione's body. This suggests that the glasses are not a transfigured item.
Great idea! When everyone has inhaled the gas Harry can truthfully say in parseltongue that if he dies, everyone present will die (because that would cancel the transfiguration).
Edit: This work well with all the early foreshadowing about how transfiguration is extremely dangerous. In Ghostbusters we establish early on that you're not supposed to cross the streams because that is extremely dangerous. And then, at the end of the move, when all is lost, what you do is to deliberately cross the streams.
It's of course possible that this Bock guy knows what he's doing on the hiring front. But in these interviews he has no incentive to give Google's competitors coherent helpful information on how to hire people - and every incentive to send out obfuscated messages which might flatter the preconceptions of NYT readers.
Bock said ... that learning ability was much more important indicator of whether someone would be a good fit for Google than I.Q.
I have limited trust in a source which says things like that.
Edited to add: More on Bock's learning ability:
For every job, though, the No. 1 thing we look for is general cognitive ability, and it’s not I.Q. It’s learning ability. It’s the ability to process on the fly. It’s the ability to pull together disparate bits of information.
Yeah, nope.
Truth has her throne on the shadowy back of doubt.
-- Sri Aurobindo (1872-1950), Savitri - A Legend and a Symbol
Ye say that those ancient prophecies are true. Behold, I say that ye do not know that they are true.
Ye say that this people is a guilty and a fallen people, because of the transgression of a parent. Behold, I say that a child is not guilty because of its parents.
And ye also say that Christ shall come. But behold, I say that ye do not know that there shall be a Christ. And ye say also that he shall be slain for the sins of the world –
And thus ye lead away this people after the foolish traditions of your fathers, and according to your own desires; and ye keep them down, even as it were in bondage, that ye may glut yourselves with the labors of their hands, that they durst not look up with boldness, and that they durst not enjoy their rights and privileges.
Yea, they durst not make use of that which is their own lest they should offend their priests, who do yoke them according to their desires, and have brought them to believe, by their traditions and their dreams and their whims and their visions and their pretended mysteries, that they should, if they did not do according to their words, offend some unknown being, who they say is God -- a being who never has been seen or known, who never was nor ever will be.
-- The Book of Mormon (Alma 30.24-28)
Edit: I'm mildly surprised by the reactions to this quote. The thing I find interesting about it is that Joseph Smith was apparently sufficiently familiar with Voltairesque anti-Christian ideas that he could relay them coherently and with some gusto. This goes some way towards passing the ideological Turing test.
No, if I were inclined to go ahead and believe in ghosts, I would not then proceed to dismiss their threat so easily.
I agree, that seems to be the weakest step. What I guess he means is that if there are ghosts they seem to be quite wispy and unobtrusive. If they went around and did a lot of stuff we would presumably have good evidence for their existence.
You don't believe in ghosts, right? Well, neither do I. But how would you like to spend a night alone in a graveyard? I am subject to night fears, and I can tell you that I shouldn't like it at all. And yet I am perfectly well aware that fear of ghosts is contrary to science, reason, and religion. If I were sentenced to spend a night alone in a graveyard, I should know beforehand that no piece of evidence was going to transpire during the night that would do anything to raise the infinitesimal prior probability of the hypothesis that there are ghosts. I should already know that twigs were going to snap and the wind moan and that there would be half-seen movements in the darkness. And I should know that the inevitable occurrences of these things would be of no evidential value whatever. And yet, after I had been frog-marched into the graveyard, I should feel a thrill of fear every time one of these things happened. I could reason with myself: "I believe that the dead are in Heaven or Hell, or else that they sleep until the General Resurrection. And if my religion is an illusion, then some form of materialism is the correct metaphysic, and materialism is incompatible with the existence of ghosts. And if the Church and the materialists are both wrong and there are ghosts, what could be the harm in a ghost? What could such a poor wispy thing do to one?" And what would the value of this very cogent piece of reasoning be? None at all, at least in respect of allaying my fear of ghosts.
I'm confident by now that conditional on my paraphrase being true to the original, Dr. Yagami's statistics don't make any sense. But, like you say, it's possible that I missed something. If someone were willing to take a look at the original text with me, we could probably settle that.
This was my attempt to make up a story where the math would match something real:
Statistically comparing two samples of equids would make some sense if Dr. Yagami had sampled 2987 horses and 8 zebras while Dr. Eru had sampled 2995 horses and 0 zebras. Then Fisher's exact test could tell us that they did, with high probability, not sample the same population with the same methods.
But in the actual case what we have is just a "virtual sample". I'm wondering if there are any conceivable circumstances where a virtual sample would make sense.
Yes, I'd prefer not to give Dr. Yagami's exact words so as not to make it too easy to find him - or for him to stumble on this post. I, too, worry that I may have left something essential out - but I can't for the life of me see what.
If I can swear you to secrecy, I'd be happy to send you a scan of the actual couple of pages from the actual book.
The main reason I posted this is that I am sometimes wrong about things. Maybe the zebra example turns out to make sense in some way I hadn't thought of. Maybe Yagami is using some sort of standard method. Maybe there's some failure mode I haven't thought of. It would be really good to know this before I make an ass of myself with the review. And talking about asses - there are some wild asses in Mongolia which got left out of my parable - but they're kind of cute so here is a link.
I do all my driving north of the 64th parallel. It's been all ice, snow and darkness for the past few months. That's probably coloring my perception here.
But do we come with pre-programmed methods for moving around - or do we just pick it up as we go along? I noticed that my two children used very different methods for moving around as babies. My daughter sat on her butt and pushed herself around. My son somehow jumped around on his knees. Both methods were surprisingly effective. There's supposedly a "crawling stage" in development but neither of my kids did any crawling to speak of. I guess this isn't as straightforwardly innate as one might think. Maybe Esther Thelen had it right.
These people white-knuckle it, constantly engaging their full slow and unsuitable System 2 in the loop, and consequently they find the normal driving activity exhausting, rather than relaxing
There's some of that in me. I probably am an overcautious driver.
Thus your hope for a safe AGI .. seems misplaced
Fair enough. Your regularly scheduled doom and gloom will resume shortly.
(e.g. moon-landing, relativity theory, computers, science in general, etc).
Or nuclear weapon design. Chicago Pile-1 did work. Trinity did work. Little Boy did work. Burster-Able failed - but not catastrophically. Who knows if whatever the North Koreans cobbled together worked as intended - but it doesn't seem to have destroyed anything it wasn't supposed to. No-one has yet accidentally blown up a city. That's something. Anyway, I'll edit the post.
I'm occasionally still amazed that traffic works as well as it does. I must say I'm hesitant at using this example to claim that people are more capable than you might think.
I actually agree. I'm not sure what lesson to draw from the fact that humans can drive. But it's interesting that so many of you seem to share my intuition that this is surprising or counterintuitive.
By the way, this is a good example showing that social life and human behaviour in general is much more "law-like" and indeed predictable than many "anti-positivists" in the social sciences would have it.
A good point - compare with this comic.
While I do like that visualization a lot, I think it is misleading in some ways. It is trivial to add to the sum of human knowledge. Go and count the coins in your wallet. I don't know how many are in mine so I'll go and check. Okay, there are 18 coins in my wallet. Now we know something we didn't know before.
"Oh, but that's not knowledge, that's just data - and just one datum at that. By 'knowledge' we mean stuff you can get published in research papers - something containing analysis and requiring insight." Bah, I tell you. You totally could get a coins-in-wallet study peer-reviewed and published. You just have to look in more wallets. Imagine you gathered a sample from 100 people in your city. Take 100 people from another city. Add cities in other countries. Do it all again in a year. Keep doing it for 10 more years. Now you have lots of data. Go and write your paper. It won't require any insight or genius - it will practically write itself. Just present your data in some legible way with nice graphs and you will find someone interested enough to publish it. Maybe not a top journal but definitely a legitimate journal in a relevant field.
The boundary of knowledge isn't hard and doesn't require years until you come up with some "breakthrough". You can just start right away, collecting information no-one currently has.
Or if you'd like to define 'human knowledge' so restrictively that the coins-in-wallet study wouldn't count, then the average PhD won't count either. Look at this zombie. Isn’t it racist and sexist? Yes, it is.
I like that formulation, thank you!
So... you value following duty as a character trait?
I guess you could spin it that way - but let me take an example.
For the last couple of weeks, my wife and I have been involved in some drama in our extended family. When we discuss in private and try to decide how we should act, I've noticed my wife keeps starting off with "If we were to do X, what would happen?". She likes to try to predict different outcomes and she wants to pick the action that leads to the best one. So maybe she is a consequentialist through and through.
I tend to see the whole sorry business as too complicated for us to predict, especially since I don't want to neglect consequences 10 or 20 years down the line. So I fall back to trying to apply rules that would be generally applicable. "What is our duty to family member X? What is our duty to family member Y?"
It's not that I would ever say "We should do X, even though it leads to worse outcomes." But I do want to consider the long run and I'd prefer not to destroy useful Schelling points for short term gain.
Are you a virtue ethicist at heart?
No, but I'm a deontologist at heart. Only in death does duty end.
Lots of good points here. In addition to the Matrix analogy (which, as you point out, is hardly a neutral way to frame the divide), keep in mind that in the US, blue and red are also the conventional colors of the left and the right.
We continue to have our little 'reactionary paradox' in that the census results show overwhelming support for feminism, but the discussion on the ground seems oddly 'red'. As you have already suggested this effect might be partially explained by LessWrong's fondness for contrarians.
I wasn't aware of these sub factions. Are they real?
It's an idealization, to be sure. And I don't think there are cliques meeting in smoke-filled IRC-channels to plot downvoting sprees. But still, I think my comment above describes something real.
Previous discussion here.
It's not that people hate your ex and want to downvote all sympathy for her. Rather, this is just one of many manifestations of our ongoing culture war. Roughly speaking, we have two teams:
Team Blue is on board with romantic love and feminism and emphasizes personal autonomy. On this view, a successful love relationship is about finding a person you click with, which could mean any number of quirky things. The problem with your marriage is that your wife was never that into you - which sucked for her. Now that she's found a person she clicks with, the seeds have been sown for more future happiness for all. No-one is really at fault, especially since you have both done your best to minimize disruption for the children.
Team Red is sympathetic to "red pill" advice (Athol Kay etc.), emphasizes biology and is less individualistic. A successful relationship is not primarily about a unique connection between unique individuals - rather it is about acting in accordance with an already existing model of what human males and females desire in each other. A particular necessity is that the male should, where appropriate, display confident and assertive behavior. On this view, our modern society has fallen into a trap (we have some harmful memes floating around, or whatever) where many males become doormats - which makes them unattractive to their wives or potential wives. What ruined your marriage is not that you were somehow inherently the wrong man for your wife - but that society failed to teach you to be assertive when appropriate. (She thought she could bring her lover into your home and you would just conveniently scurry away. This is how little she thought of you.)
You totally could express sympathy to your ex within a Team Red perspective but it would go something like this: "I feel for your wife. She couldn't control her biology and feel attracted to a man who was not displaying sufficient alpha traits." In practice, Team Red views do sometimes come with a certain regrettable bitterness towards women - especially wives who leave their husbands. So a Red-ist may tend to read sympathy to such a woman, if not explicitly framed as Red-ist, as Blue-ist propaganda.
Anyway, the point is that Team Red and Team Blue are locked in a low-grade war on LessWrong. Comments on the relevant issues will often have downvotes as well as upvotes. The comment by shminux is currently at +9, 76% positive, while the comment by Viliam Búr is at +9, 68% positive.
If you are a partisan of this, it is hard not to downvote the opposing team because you feel that they are directly harming people with their counterproductive advice and toxic memes. I don't know if we can work out a downvote ceasefire.
I like that you didn't move out and I like that you took up fencing.
My instinct is to agree with this. I spent decades learning the intricacies of North-European politeness and I think I've finally more or less got it. Now that I've learned it, I might be motivated to think that there is some actual point to all this dancing around!
I like Stefan's idea of connecting guess/ask with wait/interrupt. We might also want to bring the guilt/shame axis into this.
It sounds like ask/interrupt/shame should make for a more honest and efficient society. The guess/wait/guilt stuff sounds pretty frakked up when it is described. But in practice it seems to be correlated with the best places to live in. Maybe this is one of those Chesterton's fence things:
If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.
Or for a more recent version, xkcd on drama.
This is a very hard field to work in, psychologically, because there's no reliable process for producing valuable work (this might be true generally, but I get the sense that in the sciences it's easier to get moving in a worthwhile direction).
I think you're right that philosophy is particularly difficult in this respect. In many fields you can always go out, gather some data and use relatively standard methodologies to analyze your data and produce publishable work from it. This is certainly true in linguistics (go out and record some conversations or whatever) and philology (there are always more texts to edit, more stemmas to draw etc.). I get the impression that this is also more or less possible in sociology, psychology, biology and many other fields. But for pure philosophy, you can't do much in the way of gathering novel data.
worlds where outright complex hallucination is a normal feature of human experience
What sort of hallucinations are we talking about? I sometimes have hallucinations (auditory and visual) with sleep paralysis attacks. One close friend has vivid hallucinatory experiences (sometimes involving the Hindu gods) even outside of bed. It is low status to talk about your hallucinations so I imagine lots of people might have hallucinations without me knowing about it.
I sometimes find it difficult to tell hallucinations from normal experiences, even though my reasoning faculty is intact during sleep paralysis and even though I know perfectly well that these things happen to me. Here are two stories to illustrate.
Recently, my son was ill and sleeping fitfully, frequently waking up me and my wife. After one restless episode late in the night he had finally fallen asleep, snuggling up to my wife. I was trying to fall asleep again, when I heard footsteps outside the room. "My daughter (4 years old) must have gotten out of bed", I thought, "she'll be coming over". But this didn't happen. The footsteps continued and there was a light out in the hall. "Odd, my daughter must have turned on the light for some reason." Then through the door came an infant, floating in the air. V orpnzr greevsvrq ohg sbhaq gung V jnf cnenylmrq naq pbhyq abg zbir be fcrnx. V gevrq gb gbhpu zl jvsr naq pel bhg naq svanyyl znantrq gb rzvg n fhoqhrq fuevrx. Gura gur rkcrevrapr raqrq naq V fnj gung gur yvtugf va gur unyy jrer abg ghearq ba naq urneq ab sbbgfgrcf. "Fghcvq fyrrc cnenylfvf", V gubhtug, naq ebyyrq bire ba zl fvqr.
Here's another somewhat older incident: I was lying in bed beside my wife when I heard movement in our daughter's room. I lay still wondering whether to go fetch her - but then it appeared as if the sounds were coming closer. This was surprising since at that time my daughter didn't have the habit of coming over on her own. But something was unmistakeably coming into the room and as it entered I saw that it was a large humanoid figure with my daughter's face. V erpbvyrq va ubeebe naq yrg bhg n fuevrx. Nf zl yrsg unaq frnepurq sbe zl jvsr V sbhaq gung fur jnfa'g npghnyyl ylvat orfvqr zr - fur jnf fgnaqvat va sebag bs zr ubyqvat bhe qnhtugre. Fur'q whfg tbggra bhg bs orq gb srgpu bhe qnhtugre jvgubhg zr abgvpvat.
The two episodes play our very similarly but only one of them involved hallucinations.
I've sort of forgotten where I was going with this, but if Will would like to tell us a bit more about his experiences I would be interested.
Do you feel overworked and desparate as a PhD student or is it basically fun? Have you published any articles yet or are you planning to? What are your career plans?
I skimmed it and nothing seemed obviously wrong. If you're interested, you could try for yourself. If you download Marlowe's corpus, Shakespeare's corpus and stylo you can get a feel for how this works in a couple of hours.
You're right, it's a horrible term. For one thing, the methods involved are pretty well-established by now. I just use it by habit. As for that old Marlowe/Shakespeare hubbub, here's a recent study which finds their style similar but definitely not identical.
Yes, you could turn the quote upside down and it would still work. That was kind of the point. For effective communication it's not a good idea to talk as if your opponent is operating on your assumptions rather than her own assumptions.
I no longer try to steelman BETA-MEALR [Ban Everything That Anyone Might Experience And Later Regret] arguments as utilitarian. When I do, I just end up yelling at my interlocutor, asking how she could possibly get her calculations so wrong, only for her to reasonably protest that she wasn’t make any calculations and what am I even talking about?
I've always wanted to know more about how authorship attribution is done; is this, found with a quick search, a reasonable survey of current state of the art, or perhaps you'd recommend something else to read?
The Stamatatos survey you linked to will do fine. The basic story is "back in the day this stuff was really hard but some people tried anyway, then in 1964 Mosteller and Wallace published a landmark paper showing that you really could do impressive stuff, then along came computers and now we have a boatload of different algorithms, most of which work just great". The funny thing about stylometry is that it is hard to get wrong. Count up anything you like (frequent words, infrequent words, character n-grams, whatever) and use any distance measurement you like and odds are you'll get usable results. If you want to play around with this for yourself you can install stylo and turn it loose on a corpus of your choice. Gwern's little experiment is also a good read.
My involvement with stylometry has not been to tweak the algorithms (they work just fine) but to apply them in some particular cases and to try to convince my fellow scholars that technological wizardry really can tell them things worth knowing.
Are your fields, and humanities in general, trying to move towards open publishing of academic papers, the way STEM fields have been trying to?
Yes. Essentially every scholar I know is in favor of this. As far as I can see, It will happen and is happening.
Do you plan to stay in academia or leave, and it the latter, for what kind of job?
I worked as an engineer for a few years but found I wasn't that into it and really missed school. So I went back and I'd like to stay.
Thank you. You brought up considerations I hadn't considered.
Okay, I'll bite. Do you think any part of what MIRI does is at all useful?
Thank you. I didn't phrase my question very well but what I was trying to get at was whether making a friendly AGI might be, by some measurement, orders of magnitude more difficult than making a non-friendly one.
I claim the bet is fair if both players expect to make the same profit on average.
I like this idea. As you say, it's not the only way to define it but it does seem like a very reasonable way. The two players have come upon a situation which seems profitable to both of them and they simply agree to "split the profit".
Can you talk about your specific field in linguistics/philology?
I've mucked about here and there including in language classification (did those two extinct tribes speak related languages?), stemmatics (what is the relationship between all those manuscripts containing the same text?), non-traditional authorship attribution (who wrote this crap anyway?) and phonology (how and why do the sounds of a word "change" when it is inflected?). To preserve some anonymity (though I am not famous) I'd rather not get too specific.
what are the main challenges?
There are lots of little problems I'm interested in for their own sake but perhaps the meta-problems are of more interest here. Those would include getting people to accept that we can actually solve problems and that we should try our best to do so, Many scholars seem to have this fatalistic view of the humanities as doomed to walk in circles and never really settle anything. And for good reason - if someone manages to establish "p" then all the nice speculation based on assuming "not p" is worthless. But many would prefer to be as free as possible to speculate about as much as possible.
Do you have a stake/an opinion in the debates about the Chomskian strain in syntax/linguistics in general?
Yes. I think the Chomskyan approach is based on a fundamentally mistaken view of cognition, akin to "good old fashioned artificial intelligence". I hope to write a top-level post on this at some point. But I'll say this for Chomsky: He's not a walk-around-in-circles obscurantist. He's a resolutely-march-ahead kind of guy. A lot of the marching was in the wrong direction, but still, I respect that.
Back when you joined Wikipedia, in 2004, many articles on relatively basic subjects were quite deficient and easily improved by people with modest skills and knowledge. This enabled the cohort that joined then to learn a lot and gradually grow into better editors. This seems much more difficult today. Is this a problem and is there any way to fix it? Has something similar happened with LessWrong, where the whole thing was exciting and easy for beginners some years ago but is "boring and opaque" to beginners now?
What probability would you assign to this statement: "UFAI will be relatively easy to create within the next 100 years. FAI is so difficult that it will be nearly impossible to create within the next 200 years."
You can ask me things if you like. At Reddit, some of the most successful AMAs are when people are asked about their occupation. I have a PhD in linguistics/philology and currently work in academia. We could talk about academic culture in the humanities if someone is interested in that.
Good point on majority voting. It matters a lot whether a comment has 18 upvotes and 14 downvotes or 14 upvotes and 18 downvotes. So a relatively narrow majority on polarized subjects can give you important control over the conversation.
Correct, but then you shouldn't handwave into existence an assertion which is really at the core of the dispute.
The argument I am trying to approach is about proposals which make sense under the assumption of little or no relevant technological development but may fail to make sense once disruptive new technology enters the picture. I'm assuming the tree plan made sense in the first way - the cost of planting and tending trees is such and such, the cost of quality wood is such and such and the problems with importing it (our enemies might seek to control the supply) are such and such. Other projects we could spend the same resources on have such and such cost benefit-evaluations of their own. And so on and so forth. In this thought experiment you could assume a very sophisticated analysis which comes up smelling like roses. The only thing it doesn't take into account is disruptive new technology. That's the specific issue I'm trying to address here so that's why I'm willing to assume all the other stuff works for the sake of argument.
In actual history, maybe the tree plan never even made any sense to begin with - maybe wood was cheap and plentiful and planting the oak trees was difficult and expensive. For all I know the whole thing was a ridiculous boondoggle which didn't make sense under any assumption. But that's just an uninteresting case which need not detain us.
For the sake of argument I'm assuming the plan made prima facie sense and was only defeated by technological developments. Sufficiently familiarizing myself with the state of affairs in 1830s Sweden to materially address the question would, I think, be excessively time-consuming.
As usual, gwern has made a great comment. But I'm going to bite the bullet and come out in favor of the tree plan. Let's go back to the 1830s.
My fellow Swedes! I have a plan to plant 34,000 oak trees. In 120 years we will be able to use them to build mighty warships. My analysis here shows that the cost is modest while the benefits will be quite substantial. But, I hear you say, what if some other material is used to build warships in 120 years? Well, we will always have the option of using the wood to build warships and if we won't take that option it will be because some even better option will have presented itself. That seems like a happy outcome to me. And wood has been useful for thousands of years - it will surely not be completely obsolete in a century. We could always build other things from it, or use it for firewood or designate the forest as a recreational area for esteemed noblemen such as ourselves. Or maybe the future will have some use for forests we cannot yet anticipate [carbon sequestration]. I don't see how we can really go wrong with trees.
Back to the present. I'm concerned with avoiding disasters. "The benefits of this long-term plan were not realized because something even better happened" is only a disaster if the cost of the plan was disastrous. Of course, some people argue that the costs of addressing some of Dr. Jubjub's problems are disastrous and that's something we can discuss on the merits.
I hadn't thought of this either! It does sound like fun to hunt with the group.
The distinction you are making between robustness and resilience was not previously familiar to me but seems useful. Thank you.
Obviously, "no significant technological advances" is a basically impossible scenario. I just mean it as a baseline. If you're able to handle techno-stagnation in all domains you're able to handle any permutation of stagnating domains.