Posts
Comments
Why is it OK to use deduction theorem, though? In standard modal logics like K and S5 the deduction theorem doesn't hold (otherwise you could assume P, use necessitation to get []P, and then use deduction theorem to get P -> []P as a theorem).
Would be good to see some more references and discussion of illusionism as a view in its own right. For my money the recent work of Wolfgang Schwarz on imaginary foundations and sensor variables gives a powerful explanation of why we might have this illusion.
I'd be interested to hear how this compares with Wolfgang Schwarz's ideas in 'Imaginary Foundations' and 'From Sensor Variables to Phenomenal Facts'. Sounds like there's some overlap, and Schwarz has a kind of explanation for why the hard problem might arise that you might be able to draw on.
Link to the second of the papers mentioned: https://www.umsu.de/papers/sensorfacts.pdf
Very interesting. I'm stuck on the argument about truthfulness being hard because the concept of truth is somehow fraught or too complicated. I'm envisaging an objection based on the T-schema ('<p> is true iff p').
Nate writes:
Now, in real life, building a truthful AGI is much harder than building a diamond optimizer, because 'truth' is a concept that's much more fraught than 'diamond'. (To see this, observe that the definition of "truth" routes through tricky concepts like "ways the AI communicated with the operators" and "the mental state of the operators", and involves grappling with tricky questions like "what ways of translating the AI's foreign concepts into human concepts count as manipulative?" and "what can be honestly elided?", and so on, whereas diamond is just carbon atoms bound covalently in tetrahedral lattices.)
(end of quote)
But this reference to "the definition of 'truth'" seems to presuppose some kind of view, where I'm not sure what that view is, but know it's definitely going to be philosophically controversial.
Some think that 'true' can be defined by taking all the instances of the T-schema, or a (perhaps restricted) universal generalisation of it.
And this seems not totally crazy or irrelevant from an AI design perspective, at least at first blush. I feel I can sort of imagine an AI obeying a rule which says to assert <p> only if p.
Trying to envisage problems and responses, I hit the idea that the AI would have degrees of belief or credences, and not simply a list of things it thinks are true simpliciter. But perhaps it can have both. And perhaps obeying the T-schema based truthfulness rule would just lead it to confine most of its statements to statements about its own credences or something like that.
I think I see a separate problem about ensuring the AI does not (modify itself in order to) violate the T-schema based truthfulness rule. But that seems different at least from the supposed problem in the OP about the definition of 'true' being fraught or complicated or something.
If it wasn't already clear I'm a philosophy person, not an alignment expert, but I follow alignment with some interest.
This is an instance of arc that clever people have been going through for ages, so I'd like to see more teasing apart of the broader phenomenon from the particular historical episode of the Sequences etc.
A lot of the mixed feelings and lack of identification as rationalists on the part of lots of people who found the Sequences interesting reading is to be explained in terms of their perceiving the vibe you describe and being aware of its pitfalls.
Interesting to read, here are a couple of comments on parts of what you say:
>the claim that all possibilities exist (ie. that counterfactuals are ontologically real)
'counterfactuals are ontologically real' seems like a bad way of re-expressing 'all possibilities exist'. Counterfactuals themselves are sentences or propositions, and even people who think there's e.g. no fact of the matter with many counterfactuals should agree that they themselves are real.
Secondly, most philosophers who would be comfortable with talking seriously about possibilities or possible worlds as real things would not go along with Lewis in holding them to be concrete. The view that possibilities really truly exist is quite mainstream and doesn't commit you to modal realism.
>what worlds should we conceive of as being possible? Again, we can make this concrete by asking what would >happen if we were to choose a crazy set of possible worlds - say a world just like this one and then a world with >unicorns and fountains of gold - and no other worlds
I think it's crucial to note that it's not the presence of the unicorns world that makes trouble here, it's the absence of all the other ones here. So what you're gesturing at here is I think the need for a kind of plenitude in the possibilities one believes in.
Ramsey could be on the list too but I guess his tragically short life makes it hard to do some of the cells.
Maybe a bit off-colour to call the fact that three of Wittgenstein's brothers committed suicide 'delicious'...
Wittgenstein had so many ideas and is such a difficult thinker that I think one ought to read him before secondary sources. Also he's a wonderful writer.
I think there's a potentially confusing fact which you're neglecting in this post, namely the reality of literature as territory not map. If you're interested in literature, then when you read it you get lots of knowledge of what e.g. certain books contain, what certain authors wrote, and that can be very instructive not just within literature. I'd like to see you and others with this kind of viewpoint wrestle more with this kind of consideration.
Mill?! When are you from, John David Galt?!
I just want to say that the title of this post is fantastic, and in a deep sort of mathy way, beautiful. It's probably usually not possible, but I love it when an appropriate title - especially a nice not-too-long one - manages to contain, by itself, so much intellectual interest. Even just seeing that title listed somewhere could plant an important seed in someone's mind.
I just want to say that the title of this post is fantastic, and in a deep sort of mathy way, beautiful. It's probably usually not possible, but I love it when an appropriate title - especially a nice not-too-long one - manages to contain, by itself, so much intellectual interest. Even just seeing that title listed somewhere could plant an important seed in someone's mind.
I don't see any reason to think he's trying to convey that scientists in general, or good ones, or anything like that, believe in fake reductionism. Some people do, and it's more charitable to Keats to presume he was just alluding to them.
I agree with Robin that that indeed seems the weak point. It is far from clear to me, and I suspect it is not the case, that Keats here is doing something along the lines of actually trying to convey that, oh, there's nothing special about rainbows, science has explained them, or whatever. Rather, he's invoking and playing with that sort of trope, for a sophisticated poetic purpose.
I think the main point or points of Eliezer's post here are sound, but even suggesting that that sort of thing could be pinned on Keats is a needless distraction. Obviously serious poetry isn't Eliezer's strong point, as I'm sure he'd be the first to agree. The introductory quote could still be used to good effect though.
I think you're probably right about this (not based on first-hand experience of having a child, mind - I haven't), but I can't quite see what it's doing here. Is this meant to be some sort of objection to the comment you're replying to? It isn't obviously in tension with it.
Yep, what The Ancient Geek said. Sorry I didn't reply in a timely way - I'm not a regular user. I'm glad you basically agree, and pardon me for using such a recherche word (did I just do it again?) needlessly. Philosophical training can do that to you; you get a bit blind to how certain words are, while they could be part of the general intellectual culture, actually only used in very specific circles. (I think 'precisification' is another example of this. I used it with an intelligent nerd friend recently and, while of course he understood it - it's self explanatory - he thought it was terrible, and probably thought I just made it up.)
Hope you look at Wittgenstein!
Filled in. This is a good idea. I would be interested in getting some feedback on the feedback, or seeing a writeup of some of the lessons or issues that come out of this.
How specifically could being "definite" be a a problem for language? Take any specific thing, apply an arbitrary label, and you are done.
This remark seems to flow from an oversimplified view of how language works. In the context of, for example, a person or a chair, this paradigm seems pretty solid... at least, it gets you a lot. You can ostend the thing ('take' it, as it were) and then appy the label. But in the case of lots of "objects" there is nothing analogous to such 'taking' as a prior, discrete step from talking. For example, "objects" like happiness, or vagueness or definiteness themselves.
I think you may benefit from reading Wittgenstein, but maybe you'd just hate it. I think you need it though!
For my part, I've found the economic notions of opportunity cost and marginal utility to be like this.
'my writing is more enthusiastic than the evidence would call for, but alas I must excite my readers and get the pageviews'
For my money, that's just contemptible. And there's no 'must' about it: you can, and probably should, stop doing that, even if it means you get less pageviews.
More than six years late, but better late than never...
'Concepts are utilized in reasoning to reduce and structure search space' - anyone have any references or ideas for further developments of this line of thought? Seems very interesting and related to the philosophical idea of abduction or inference to the best explanation. (Perhaps the relation is one of justification.)
Also, since I find the OP compelling despite this point, I would be interested to see how far they can be reconciled.
My guess, loosely expressed, is that the stuff in Eliezer's OP above about the importance of good bookkeeping to prevent update messages bouncing back is sound, and should be implemented in designing intelligent systems, but some additional, more abductionesque process could be carefully laid on top. And when interpreting human reasoning, we should perhaps try to learn to distinguish whether, in a given case of a non-predictive empirical belief, the credence comes from bad bookkeeping, in which case it's illegitimate, or an abductive process which may be legitimate, and indeed may be legitimated along the lines of Vladimir's tantalizing hint in the parent comment.
'It's like, if you're going to stab me in the back, is it better if it's with a white knife?'
It's not like that at all! 'Deceive' isn't a dirty word - i.e. it doesn't automatically mean something that is bad to do. 'Stabbing in the back', on the other hand, seems to. 'He kindly deceived me' may sound odd, but not at all self-contradictory like 'He kindly stabbed me in the back' (metaphorical meaning intended, of course). It seems perfectly reasonable to me to think that deception is sometimes a very decent, kind, considerate practice to engage in. The idea that it's automatically bad seems childish to me.
This is interesting, particularly in connection with your grativation towards materialism - thanks for sharing.
An extended answer to your question is given in the original post - the post is all about answering that question, and it seems very clearly written to me. So I think you're being silly.
'if you can't communicate with them honestly, they shouldn't be your friends/partners in the first place'
I think that, insofar as this sounds plausible, it doesn't conflict with what Chris is saying in the OP. It seems perfectly possible for it to be the case that you can (and by and large do) communicate with someone honestly, simultaneously with it being the case that it's sometimes best to lie to them.
And FWIW, I think that realizing that lying is sometimes the way to go is part and parcel of a mature and able approach to interpersonal relationships. The other view seems to me both simplistic and morally smug. I find the complete lack of argument in your comment quite telling.
Yeah, this looks more like the Law of Non-Contradiction than the Law of Excluded Middle to me (which makes Manfred's jokey response seem doubly foolish).
'They contain the same content because A->B says that A and not-B is impossible, and saying that A and not-B is impossible says that A->B. For example, "it raining but not being wet outside is impossible."'
If you're talking about standard propositional logic here, without bringing in probabilistic stuff, then this is just wrong or at best very misleadingly put. All 'A->B' says is that it is not the case that A and not-B - nothing modal.
This seems well below the standard often reached here. The writing seems very sloppy and telegraphic... for instance, of course we can say and think 'Body, I command you not to bleed'! It just won't do anything.
And: 'At the same time, there's a view that we have full control and choice over our actions in a given situation.' - This seems a bit like a strawman. What's the view exactly? Who has ever held it? And why should not being able to stop bleeding at will constitute a counterexample? No one normally classifies bleeding as an 'action'.
And regarding the two 'takeaways': you acknowledge that the first is probably not very new or important, and the second, as you've summed it up (second last paragraph), seems incredibly trite. We can't choose to not bleed. OK. Things that seem like choices aren't always so. OK, that seems true too, but also trite, and the bleeding thing doesn't seem like an example: who ever thought you could stop bleeding at will, who ever thought there was any choice?
I think Lesswrong can do better than this.
Pardon a second comment (I hope that's not bad etiquette), but here are a couple of further qualms/criticisms attending to which could improve the post:
Regarding your use of the phrase 'foundations of probability' to refer to arguments for why a certain kind of robot should use probabilities: this seems like a rather odd use for a phrase that already has at least two well established uses. (Roughly (i) basic probability theory, i.e. that which gives a grounding or foundation in learning the subject, and (ii) the philosophical or metaphysical underpinnings of probability discourse: what's it about, what kinds are there, what makes true probability claims true etc.?) Is it really helpful to be different on this point, when there is already considerable ambiguity?
Furthermore, and perhaps more substantively, your bit on Dutch Books doesn't seem to give any foundations in your sense: Dutch Book arguments aren't arguments for using probability (i.e. at all, i.e. instead of not using it), but rather for conforming, when already using probability, to the standard probability calculus. So there seems to be a confusion in your post here.
'But to assign some probability to the wrong answer is logically equivalent to assigning probability to 0=1.'
Huh? This doesn't make sense to me. First of all, it seems like a basic category-mistake: acts of assigning probabilities don't seem to be the sorts of things that can bear logical relations like equivalence to each other.
Perhaps that's just pedantry and there's a simple rephrasing that says what you really want to say, but I have a feeling I would take issue with the rephrased version too. Does it trade on the idea that all false mathematical propositions are logically equivalent to each other? (If so, I'd say that's a problem, because that idea is very controversial, and hardly intuitive.)
I really liked the introduction - really well done. (shminux seems to agree!)
Some constructuve criticisms:
'There are playing fields where you should cooperate with DefectBot, even though that looks completely insane from a naïve viewpoint. Optimality is a feature of the playing field, not a feature of the strategy.' - I like your main point made with TrollBot, but this last sentence doesn't seem like a good way of summing up the lesson. What the lesson seems to be in my eyes is: strategies' being optimal or not is playing-field relative. So you could say that optimality is a relation holding between strategies and playing fields.
Later on you say 'It helps to remember that "optimality" is as much a feature of the playing field as of the strategy.' - but, my criticism above aside, this seems inconsistent with the last sentence of the previous quote (here you say optimality is equally a feature of two things, whereas before you said it was not a feature of the strategy)! Here you seem to be leaning more toward my proposed relational gloss.
Another suggestion. The Omega argument comes right after you say you're going to show that we occupy a strange playing field right now. This tends to make the reader prepare to object 'But that's not very realistic!'. Maybe you like that sort of tension and release thing, but my vote would be to first make it clear what you're doing there -- i.e., not right away arguing about the real world, but taking a certain step toward that.
One final suggestion. You write 'Knowing this, I have a compartment in which my willpower doesn't deplete', and something relevantly similar just earlier. Now this is obviously not literally what you mean - rather, it's something like, you have a compartment housing the belief that your willpower doesn't deplete. Obviously, you get a certain literary effect by putting it the way you do. Now, I realize reasonable people may think I'm just being overly pedantic here, but I suspect that's wrong, and that in this sort of discussion, we should habitually help ourselves to such easily-had extra precision. Since things get confusing so quickly in this area, and we're liable to slip up all over the place, apparently minor infelicities could make a real difference by sapping resources which are about to be taxed to the full.
'At its core, model theory is the study of what you said, as opposed to what you meant.'
One way to improve the clarity of this gloss, and make it more ecumenical (to be frank, I imagine as it stands, many philosophers would balk and sort of go 'WTF?' and treat this as a weird, confused thing to say), might be as follows: distinguishing the meaning of an expression in some language from the speaker's intended meaning in producing that expression. These can of course diverge, but both are semantic notions. (Your use of the two different terms above may obscure this, by suggesting that you can't use 'mean' and its cognates for speaker-meaning.)
Kripke has a paper called 'Speaker's Reference and Semantic Reference' which may be helpful. (There's an online copy here at present, but in any case it's not hard to find.) What it seems you want to do, insofar as you think there's more to meaning than reference, is something like: generalize Kripke's basic idea here to meaning in general (and so factoring in internal components of meaning such as 'role in the system', as well as just reference) and then use that distinction to say that model theory is about linguistic as opposed to speaker-meaning.
But I'm still not sure why you'd want to say (or emphasize) that. My reaction is: yeah, applications of model theory are often geared that way, but why couldn't you also give model-theoretic accounts of speaker-meaning? But perhaps I've misunderstood.
Stimulating as always! I have a criticism to make of the use made of the term 'rigid designation'.
Multiple philosophers have suggested that this stance seems similar to "rigid designation", i.e., when I say 'fair' it intrinsically, rigidly refers to something-to-do-with-equal-division. I confess I don't see it that way myself [...]
What philosophers of language ordinarily mean by calling a term a rigid designator is not that, considered purely syntactically, it intrinsically refers to anything. The property of being a rigid designator is something which can be possessed by an expression in use in a particular language-system. The distinction is between expressions-in-use whose reference we let vary across counterfactual scenarios (or 'possible worlds'), e.g. 'The first person to climb Everest', and those whose reference remains stable, e.g. 'George Washington', 'The sum of two and two'.
There is some controversy over how to apply the rigid/non-rigid distinction to general terms like 'fair' (or predicates like 'is fair') - cf. Scott Soames' book Beyond Rigidity - but I think the natural thing to say is that 'is fair' is rigid, since it is used to attribute the same property across counterfactual scenarios, in contrast with a predicate like 'possesses my favourite property'.
The beginning of this comment, up to the comma, sounds so very like the beginning of one of those Chuck Norris format jokes. I was honestly surprised when it turned out not to be.
Come to think of it, "every book is terrible" may also be correct for Steven King.
I wonder if this principle works in the case of a murder which rapidly changes the murderer. (Later that day, they may bear no responsibility.)
If only lukeprog had thought to tell Alice that at the time!
I find that edit sort of chilling!
Wow, that is extreme. And potentially dangerous. Do you really think you would have followed through in the event of failure? I don't think I could have.
Regarding your problem: if you haven't taken steps to rule out narcolepsy, I recommend you do.
I don't think this is rude at all. One of the things I like about Less Wrong, and which seems characteristic of it, is that the writing in posts - style and form as well as more basic stuff - is often constructively discussed with a view to improving the author's writing.
Is this on today (18th), or at all? I notice a comment from Oklord proposing to change the date, and Observer saying they're cool with that, so does that mean it was on the 17th? A reply ASAP would be good, cheers.