The Useful Idea of Truth

Thank you for being positive.

I've been recently thinking about this, and noticed that despite things like "why our kind can't cooperate", we still focus on criticisms of minor points, even when there are major wins to be celebrated.

This is a great post. I think the presentation of the ideas is clearer and more engaging than the sequences, and the cartoons are really nice. Wild applause for the artist.

I have a few things to say about the status of these ideas in mainstream philosophy, since I'm somewhat familiar with the mainstream literature (although admittedly it's not the area of my expertise). I'll split up my individual points into separate comments.

Alfred Tarski is a famous mathematician whose theory of truth is widely known.

Summary of my point: Tarski's biconditionals are not supposed to be a definition of truth. They are supposed to be a test of the adequacy of a proposed definition of truth. Proponents of many different theories claim that their theory passes this test of adequacy, so to identify Tarski's criterion with the correspondence theory is incorrect, or at the very least, a highly controversial claim that requires defense. What follows is a detailed account of why the biconditionals can't be an adequate definition of truth, and of what Tarski's actual theory of truth is.

Describing Tarski's biconditionals as a definition of truth or a theory of truth is misleading. The relevant paper is The Semantic Conception of Truth. Let's call sentences of the form 'p' is true iff p T-sentences. Tarski's claim in the paper is that the T-sentences constitute a criterion of adequacy for any proposed theory of truth. Specifically, a theory of truth is only adequate if all the T-sentences follow from it. This basically amounts to the claim that any adequate theory of truth must get the extension of the truth-predicate right -- it must assign the truth-predicate to all and only those sentences that are in fact true.

I admit that the conjunction of all the T-sentences does in fact satisfy this criterion of adequacy. All the individual T-sentences do follow from this conjunction (assuming we've solved the subtle problem of dealing with infinitely long sentences). So if we are measuring by this criterion alone, I guess this conjunction would qualify as an adequate theory of truth. But there are other plausible criteria according to which it is inadequate. First, it's a frickin' infinite conjunction. We usually prefer our definitions to be shorter. More significantly, we usually demand more than mere extensional adequacy from our definitions. We also demand intensional adequacy.

If you ask someone for a definition of "Emperor of Rome" and she responds "X is an Emperor of Rome iff X is one of these..." and then proceeds to list every actual Emperor of Rome, I suspect you would find this definition inadequate. There are possible worlds in which Julius Caesar was an Emperor of Rome, even though he wasn't in the actual world. If your friend is right, then those worlds are ruled out by definition. Surely that's not satisfactory. The definition is extensionally adequate but not intensionally adequate. The T-sentence criterion only tests for extensional adequacy of a definition. It is satisfied by any theory that assigns the correct truth predicates in our world, whether or not that theory limns the account of truth in a way that is adequate for other possible worlds. Remember, the biconditionals here are material, not subjunctive. The T-sentences don't tell us that an adequate theory would assign "Snow is green" as true if snow were green. But surely we want an adequate theory to do just that. If you regard the T-sentences themselves as the definition of truth, all that the definition gives us is a scheme for determining which truth ascriptions are true and false in our world. It tells us nothing about how to make these determinations in other possible worlds.

To make the problem more explicit, suppose I speak a language in which the sentence "Snow is white" means that grass is green. It will still be true that, for my language, "Snow is white" is true iff snow is white. Yet we don't want to say this biconditional captures what it means for "Snow is white" to be true in my language. After all, in a possible world where snow remained white but grass was red, the sentence would be false.

Tarski was a smart guy, and I'm pretty sure he realized all this (or at least some of it). He constantly refers to the T-sentences as material criteria of adequacy for a definition of truth. He says (speaking about the T-sentences), "... we shall call a definition of truth 'adequate' if all these equivalences follow from it." (although this seems to ignore the fact that there are other important criteria of adequacy) When discussing a particular objection to his view late in the paper, he says, "The author of this objection mistakenly regards scheme (T)... as a definition of truth." Unfortunately, he also says stuff that might lead one to think he does think of the conjunction of all T-sentences as a definition: "We can only say that every equivalence of the form (T)... may be considered a partial definition of truth, which explains wherein the truth of this one individual sentence consists. The general definition has to be, in a certain sense, a logical conjunction of all these partial definitions."

I read the "in a certain sense" there as a subtle concession that we will need more than just a conjunction of the T-sentences for an adequate definition of truth. As support for my reading, I appeal to the fact that Tarski explicitly offers a definition of truth in his paper (in section 11), one that is more than just a conjunction of T-sentences. He defines truth in terms of satisfaction, which in turn is defined recursively using rules like: The objects a and b satisfy the sentential function "P(x, y) or Q(x, y)" iff they satisfy at least one of the functions "P(x, y)" or "Q(x, y)". His definition of truth is basically that a sentence is true iff it is satisfied by all objects and false otherwise. This works because a sentence, unlike a general sentential function, has no free variables to which objects can be bound.

This definition is clearly distinct from the logical conjunction of all T-sentences. Tarski claims it entails all the T-sentences, and therefore satisfies his criterion of adequacy. Now, I think Tarski's actual definition of truth isn't all that helpful. He defines truth in terms of satisfaction, but satisfaction is hardly a more perspicuous concept. True, he provides a recursive procedure for determining satisfaction, but this only tells us when compound sentential functions are satisfied once we know when simple ones are satisfied. His account doesn't explain what it means for a simple sentential function to be satisfied by an object. This is just left as a primitive in the theory. So, yeah, Tarski's actual theory of truth kind of sucks.

His criterion of adequacy, though, has been very influential. But it is not a theory of truth, and that is not the way it is treated by philosophers. It is used as a test of adequacy, and proponents of most theories of truth (not just the correspondence theory) claim that their theory satisfies this test. So to identify Tarski's definition/criterion/whatever with the correspondence theory misrepresents the state of play. There are, incidentally, a group of philosophers who do take the T-sentences to be a full definition of truth, or at least to be all that we can say about truth. But these are not correspondence theorists. They are deflationists.

I haven't particularly run across any philosophy explicitly making the connection from the correspondence theory of truth to "There are causal processes producing map-territory correspondences" to "You have to look at things in order to draw accurate maps of them, and this is a general rule with no exception for special interest groups who want more forgiving treatment for their assertions". I would not be surprised to find out it existed, especially on the second clause.

Depends on what you mean by "explicitly". Many correspondence theorists believe that an adequate understanding of "correspondence" requires an understanding of reference -- how parts of our language are associated with parts of the world. I think this sort of idea stems from trying to fill out Tarski's (actual) definition of truth, which I discussed in another comment. The hope is that a good theory of reference will fill out Tarski's obscure notion of satisfaction, and thereby give some substance to his definition of truth in terms of satisfaction.

Anyway, there was a period when a lot of philosophers believed, following Saul Kripke and Hilary Putnam, that we can understand reference in terms of causal relations between objects in the world and our brains (it appears to me that this view is falling out of vogue now, though). What makes it the case that our use of the term "electron" refers to electrons? That there are the appropriate sorts of causal relations, both social -- the causal chain from physicists who originated the use of the word to contemporary uses of it -- and evidential -- the causal connections with the world that govern the ways in which contemporary physicists come to assert new claims involving the word "electron". The causal theory of reference is used as the basis for a (purportedly) non-mysterious account of satisfaction, which in turn is used as the basis for a theory of truth.

So the idea is that the meanings of the elements in our map are determined by causal processes, and these meanings link the satisfaction conditions of sentential functions to states of affairs in the world. I'm not sure this is exactly the sort of thing you're saying, but it seems close. For an explicit statement of this kind of view, see Hartry Field's Tarski's Theory of Truth. Most of the paper is a (fairly devastating, in my opinion) critique of Tarski's account of truth, but towards the end of section IV he brings up the causal theory.

ETA: More broadly, reliabilism in epistemology has a lot in common with your view. Reliabilism is a refinement of early causal theories of knowledge. The idea is that our beliefs are warranted in so far as they are produced by reliable mechanisms. Most reliabilists I'm aware of are naturalists, and read "reliable mechanism" as "mechanism which establishes appropriate causal connections between belief states and world states". Our senses are presumed to be reliable (and therefore sources of warrant) just because the sorts of causal chains you describe in your post are regularly instantiated. Reliabilism is, however, compatible with anti-naturalism. Alvin Plantinga, for instance, believes that the sensus divinitatis should be regarded as a reliable cognitive faculty, one that atheists lack (or ignore).

One example of a naturalist reliabilism (paired with a naturalist theory of mental representation) is Fred Dretske's Knowledge and the Flow of Information. A summary of the book's arguments is available here (DOC file). Dretske tries to understand perception, knowledge, the truth and falsity of belief, mental content, etc. using the framework of Shannon's communication theory. The basis of his analysis is that information transfer from a sender system to a receiver system must be understood in terms of relations of law-like dependence of the receiver system's state on the sender system's state. He then analyzes various epistemological problems in terms of information transfer from systems in the external world to our perceptual faculties, and information transfer from our perceptual faculties to our cognitive centers. He's written a whole book about this, so there's a lot of detail, and some of the specific details are suspect. In broad strokes, though, Dretske's book expresses pretty much the same point of view you describe in this post.

Speaking as the author of Eliezer's Sequences and Mainstream Academia...

Off the top of my head, I also can't think of a philosopher who has made an explicit connection from the correspondence theory of truth to "there are causal processes producing map-territory correspondences" to "you have to look at things to draw accurate maps of them..."

But if this connection has been made explicitly, I would expect it to be made by someone who accepts both the correspondence theory and "naturalized epistemology", often summed up in a quote from Quine:

The stimulation of his sensory receptors is all the evidence anybody has had to go on, ultimately, in arriving at his picture of the world. Why not just see how this construction really proceeds? Why not settle for psychology? ...Epistemology, or something like it, simply falls into place as a chapter of psychology and hence of natural science.

(Originally, Quine's naturalized epistemology accounted only for this descriptive part of epistemology, and neglected the normative part, e.g. truth conditions. In the 80s Quine started saying that the normative part entered into naturalized epistemology through "the technology of truth-seeking," but he was pretty vague about this.)

Edit: Another relevant discussion of embodiment and theories of truth can be found in chapter 7 of Philosophy in the Flesh.

You might also be interested in checking out what Mohandas Gandhi had to say about "the meaning of truth", just in case you were wondering what things are like in the rest of the world outside the halls of philosophy departments.

Here's a quote from Perry Anderson's recent (highly critical) essay on Gandhi:

There can be no doubt that he was, so far as he himself went, sincere enough in his commitment to non-violence. But as a political leader, his conception of himself as a vessel of divine intention allowed him to escape the trammels of human logic or coherence. Truth was not an objective value – correspondence to reality, or even (in a weaker version) common agreement – but simply what he subjectively felt at any given time. ‘It has been my experience,’ he wrote, ‘that I am always true from my point of view.’ His autobiography was subtitled The Story of My Experiments with Truth, as if truth were material for alteration in a laboratory, or the plaything of a séance. In his ‘readiness to obey the call of Truth, my God, from moment to moment’, he was freed from any requirement of consistency. ‘My aim is not to be consistent with my previous statements,’ he declared, but ‘with truth as it may present itself to me at a given moment’: ‘since I am called “Great Soul” I might as well endorse Emerson’s saying that “foolish consistency is the hobgoblin of little minds.”’ The result was a licence to say whatever he wanted, regardless of what he had said before, whenever he saw fit.

I haven't particularly run across any philosophy explicitly making the connection from the correspondence theory of truth to "There are causal processes producing map-territory correspondences" to "You have to look at things in order to draw accurate maps of them, and this is a general rule..."

Trying to include mainstream academia other than philosophy, and going off your blog post "The Second Law of Thermodynamics, and Engines of Cognition", it seems the idea of the general rule that you have to look at and interact with things to form accurate beliefs about them was largely due to Leo Szilard in his 1939 paper "On the Decrease in Entropy in a Thermodynamic System by the Intervention of Intelligent Beings" which analyzed Maxwell's demon thought experiment and introduced the Szilard engine and the entropy cost of gaining information. You gave a more Bayesian analysis than Szilard in that post, possibly going off Jaynes' work in statistical mechanics, like his 1957 papers "Information Theory and Statistical Mechanics" parts one and two, which are the earliest mention of Liouville's theorem I can find in that context. Does Pearl have anything to throw in the mix, like a fancy rule about concluding a past causal interaction when you see corresponding maps and cities?

I haven't particularly run across any philosophy explicitly making the connection from the correspondence theory of truth to "There are causal processes producing map-territory correspondences" to "You have to look at things in order to draw accurate maps of them, and this is a general rule with no exception for special interest groups who want more forgiving treatment for their assertions". I would not be surprised to find out it existed, especially on the second clause.

DevilWorm and pragmatist point to the "reliabilism" school of philosophy (http://en.wikipedia.org/wiki/Reliabilism & http://plato.stanford.edu/entries/reliabilism). Clicking on either link reveals arguments concerned mainly with that old dispute over whether the word "knowledge" should be used to refer to "justified true belief". Going on the wording I'm not even sure whether they're considering how photons from the Sun are involved in correlating your visual cortex to your shoelaces. But it does increase the probability of a precedent - does anyone have something more specific? (A lot of the terminology I've seen so far is tremendously vague, and open to many interpretations...)

Incidentally, there might be an even higher probability of finding some explicit precedent in a good modern AI book somewhere?

I haven't particularly run across any philosophy explicitly making the connection from the correspondence theory of truth to "There are causal processes producing map-territory correspondences" to "You have to look at things in order to draw accurate maps of them, and this is a general rule with no exception for special interest groups who want more forgiving treatment for their assertions". I would not be surprised to find out it existed, especially on the second clause.

Process reliabilism maybe? Defines the "justified" part in "justified true belief" as the belief being formed by a reliable truth-producing process.

From the Stanford Encyclopedia of Philosophy article:

Alvin Goldman offered his first formulation of a reliable process theory of knowing — as a refinement of the causal theory of knowing — in a short paper on innate knowledge (Goldman, 1975).

Whatever a "causal theory of knowing" is. But it sounds like the kind of thing you're talking about.

I've edited the OP to try and compartmentalize off the example a bit more.

Is this actually a standard term? I was trying to make up a new one, without having to actually delve into the pits of darkness and find a real postmodern literary term that doesn't mean anything.

If someday the vast majority of people decided that what is known as "blue" should be renamed "snarffle" then eventually it would cease to be blue. Instead it would be snarffle because that is the belief. But that doesn't change the reality that it is the wavelength 475 nm. Human beliefs determine how we interpret information, not reality.

I think a more general notion of truth could be defined as correspondence between a map and any structure. If you define a structure using axioms and are referencing that structure, then you can talk about the correspondence properties of that reference. This at least cover both mathematical structures and physical reality.

If I say "Alice has a false belief that she should two-box in Newcomb's problem" it doesn't seem like I'm saying that her map doesn't correspond to the territory.

The problem with Alice's belief is that it is incomplete. It's like saying "I believe that 3 is greater than" (end of sentence).

Even incomplete sentences can work in some contexts where people know how to interpret them. For example if we had a convention that all sentences ending with "greater than" have to be interpreted as "greater than zero", then in given context the sentence "3 is greater than" makes sense, and is true. It just does not make sense outside of this context. Without context, it's not a logical proposition, but rather a proposition template.

Similarly, the sentence "you should X" is meaningful in contexts which provide additional explanation of what "should" means. For a consequentialist, the meaning of "you should" is "maximizes your utility". For a theist, it could mean "makes Deity happy". For both of them, the meaning of "should" is obvious, and within their contexts, they are right. The sentence becomes confusing only when we take it out of context; when we pretend that the context is not necessary for completing it.

So perhaps the problem is not "some truths are not about map-territory correspondence", but rather "some sentences require context to be transformed into true/false expressions (about map-territory correspondence)".

Seems to me that this is somehow related to making ideas pay rent, in sense that when you describe how do you expect the idea to pay rent, in the process you explain the context.

Michael Lynch has a functionalist theory of truth (described in this book) that responds to concerns like yours. His claim is that there is a "truth role" that is constant across all domains of discourse where we talk about truth and falsity of propositions. The truth role is characterized by three properties:

1. Objectivity: The belief that p is true if and only if with respect to the belief that p, things are as they are believed to be.

2. Norm of belief: It is prima facie correct to believe that p if and only if the proposition that p is true.

3. End of inquiry: Other things being equal, true beliefs are a worthy goal of inquiry.

Lynch claims that, in different domains of discourse, there are different properties that play this truth role. For instance, when we're doing science it's plausible that the appropriate realizer of the truth role is some kind of correspondence notion. On the other hand, when we're doing mathematics, one might think that the truth role is played by some sort of theoretical coherence property. Mathematical truths, according to Lynch, satisfy the truth role, but not by virtue of correspondence to some state of affairs in our external environment. He has a similar analysis of moral truths.

I'm not sure whether Lynch's particular description of the truth role is right, but the functionalist approach (truth is a functional property, and the function can be performed by many different realizers) is very attractive to me.

Maybe the first two just argues for Platonism and modal realism (although I note that Eliezer explicitly disclaimed being a modal realist).

I think Yudkowsky is a Platonist, and I'm not sure he has a consistent position on modal realism, since when arguing on morality he seemed to espouse it: see his comment here.

For example, "You should two-box in Newcomb's problem." If I say "Alice has a false belief that she should two-box in Newcomb's problem" it doesn't seem like I'm saying that her map doesn't correspond to the territory.

I don't think that "You should two-box in Newcomb's problem." is actually a normative statement, even if it contains a "should": you can rephrase it epistemically as "If you two-box in Newcomb's problem then you will maximize your expected utility".

Therefore, if you say "Alice has a false belief that if she two-boxes in Newcomb's problem then she will maximize her expected utility" you are saying that her belief doesn't correspond to the mathematical constructs underlying Newcomb's problem. If you take the Platonist position that mathematical constructs exist as external entities ("the territory"), then yes, you are saying that her map doesn't correspond to the territory.

If I say "Alice has a false belief that she should two-box in Newcomb's problem" it doesn't seem like I'm saying that her map doesn't correspond to the territory.

It seems that way to me. Specifically, in that case I think you're saying that Alice (wrongly) expects that her decision is causally independent from the money Omega put in the boxes, and as such thinks that her expected utility is higher from grabbing both boxes.

I don't think 2 is answered even if you say that the mathematical objects are themselves real. Consider a geometry that labels "true" everything that follows from its axioms. If this geometry is consistent, then we want to say that it is true, which implies that everything it labels as "true", is. And the axioms themselves follow from the axioms, so the mathematical system says that they're true. But you can also have another valid mathematical system, where one of those axioms is negated. This is a problem because it implies that something can be both true and not true.

Because of this, the sense in which mathematical propositions can be true can't be the same sense in which "snow is white" can be true, even if the objects themselves are real. We have to be equivocating somewhere on "truth".

He says that counterfactuals do have a truth value, though IMO he's a bit vague about what that is (or maybe it's me who can't fully understand what he says).

Actually, "relativist" isn't a lot better, because it's still pretty clear who's meant, and it's a very charged term in some political discussions.

I think it's a bad rhetorical strategy to mock the cognitive style of a particular academic discipline, or of a particular school within a discipline, even if you know all about that discipline. That's not because you'll convert people who are steeped in the way of thinking you're trying to counter, but because you can end up pushing the "undecided" to their side.

Let's say we have a bright young student who is, to oversimplify, on the cusp of going down either the path of Good ("parsimony counts", "there's an objective way to determine what hypothesis is simpler", "it looks like there's an exterior, shared reality", "we can improve our maps"...) or the path of Evil ("all concepts start out equal", "we can make arbitrary maps", "truth is determined by politics" ...). Well, that bright young student isn't a perfectly rational being. If the advocates for Good look like they're being jerks and mocking the advocates for Evil, that may be enough to push that person down the path of Evil.

Wulky Wilkinson is the mind killer. Or so it seems to me.

Particularly since many LWers believe things like:

The progress of science is measured as much by deaths among the Old Guard as by discoveries from the Young Idealists.

or

Psychological diagnosis (like those listed in the DSM) function to separate the socially acceptable from the unacceptable and do not even try to cut the world at its joints.

The other day Yvain was reading aloud from Feser and I said I wished Feser would read The Simple Truth. I don't think this would help quite as much.

The Simple Truth sought to convey the intuition that truth is not just a property of propositions in brains, but of any system successfully entangled with another system. Once the shepherd's leveled up a bit in his craftsmanship, the sheep can pull aside the curtain, drop a pebble into the bucket, and the level in the bucket will remain true without human intervention.

I also really enjoyed this post, and specifically thought that the illustrations were much nicer than what's been done before.

However, I did notice that out of all the illustrations that were made for this post, there were about 8 male characters drawn, and 0 females. (The first picture of the Sally-Anne test did portray females, but it was taken from another source, not drawn for this post like the others.) In the future, it might be a good idea to portray both men AND women in your illustrations. I know that you personally use the "flip a coin" method for gender assignment when you can, but it doesn't seem like the illustrator does (There IS a 0.3% chance that the coin flips just all came up "male" for the drawings)

Thanks!

Ditto.

I dislike the "post utopian" example, and here's why:

Language is pretty much a set of labels. When we call something "white", we are saying it has some property of "whiteness." NOW we can discuss wavelengths and how light works, or whatnot, but 200 years ago, they had no clue. They could still know that snow is white, though. At the same time, even with our knowledge of how colors work, we can still have difficulties knowing exactly where the label "white" ends, and grey or yellow begins.

Say I'm carving up music-space. I can pretty easily classify the differences between Classical and Rap, in ways that are easy to follow. I could say that classical features a lot of instrumentation, and rap features rhythmic language, or something. But if I had lots of people spending all their lives studying music, they're going to end up breaking music space into much smaller pieces. For example, dub step and house.

Now, I can RECOGNIZE dubstep when I hear it, but if you asked me to teach you what it was, I would have difficulties. I couldn't necessarily say "It's the one that goes, like, WOPWOPWOPWOP iiinnnnnggg" if I'm a learned professor, so I'll use jargon like "synthetic rhythm," or something.

But not having a complete explainable System 2 algorithm for "How to Tell if it's Dubstep" doesn't mean that my System 1 can't readily identify it. In fact, it's probably easier to just listen to a bunch of music until your System 1 can identify the various genres, even if your System 2 can't codify it. The example is treating the fact that your professor can't really codify "post utopianism" to mean that it's not "true". (this example has been used in other sequence posts, and I disagreed with it then too)

Have someone write a bunch of short stories. Give them to English Literature professors. If they tend to agree which ones are post utopian, and which ones aren't, then they ARE in fact carving up literature-space in a meaningful way. The fact that they can't quite articulate the distinction doesn't make it any less true than knowing that snow was white before you knew about wavelengths. They're both labels, we just understand one better.

Anyways, I know it's just an example, but without a better example, i can't really understand the question well enough to think of a relevant answer.

A set of beliefs is not like a bag of sand, individual beliefs unconnected with each other, about individual things. They are connected to each other by logical reasoning, like a lump of sandstone. Not all beliefs need to have a direct connection with experience, but as long as pulling on the belief pulls, perhaps indirectly, on anticipated experience, the belief is meaningful.

When a pebble of beliefs is completely disconnected from experience, or when the connection is so loose that it can be pulled around arbitrarily without feeling the tug of experience, then we can pronounce it meaningless. The pebble may make an attractive paperweight, with an intricate structure made of elements that also occur in meaningful beliefs, but that's all it can be. Music of the mind, conveying a subjective impression of deep meaning, without having any.

For the hypothetical photon disappearing in the far-far-away, no observation can be made on that photon, but we have other observations leading to beliefs about photons in general, according to which they cannot decay. That makes it meaningful to say that the far away photon acts in the same way. If we discovered processes of photon decay, it would still be meaningful, but then we would believe it could be false.

If a person with access to the computer simulating whichever universe (or set of universes) a belief is about could in principle write a program that takes as input the current state of the universe (as represented in the computer) and outputs whether the belief is true, then the belief is meaningful.

(if the universe in question does not run on a computer, begin by digitizing your universe, then proceed as above)

Before reading other answers, I would guess that a statement is meaningful if it is either implied or refuted by a useful model of the universe - the more useful the model, the more meaningful the statement.

For a belief to be meaningful you have to be able to describe evidence that would move your posterior probability of it being true after a Bayesian update.

This is a generalization of falsifiability that allows, for example, indirect evidence pertaining to universal laws.

Consider "Elaine is a post-utopian and the Earth is round" This statement is meaningless, at least in the case where the Earth is round, where it is equivalent to "Elaine is a post-utopian." Yet it does constrain my experience, because observing that the Earth is flat falsifies it. If something like this came to seem like a natural proposition to consider, I think it would be hard to notice it was (partly) meaningless, since I could still notice it being updated.

This seems to defeat many suggestions people have made so far. I guess we could say it's not a real counterexample, because the statement is still "partly meaningful". But in that case it would be still be nice if we could say what "partly meaningful" means. I think that the situation often arises that a concept or belief people throw around has a lot of useless conceptual baggage that doesn't track anything in the real world, yet doesn't completely fail to constrain reality (I'd put phlogiston and possibly some literary criticism concepts in this category).

My first attempt is to say that a belief A of X is meaningful to the extent that it (is contained in / has an analog in / is resolved by) the most parsimonious model of the universe which makes all predictions about direct observations that X would make.

For every meaningful proposition P, an author should (in theory) be able to write coherently about a fictional universe U where P is true and a fictional universe U' where P is false.

What rule could restrict our beliefs to just propositions that can be meaningful, without excluding a priori anything that could in principle be true?

A variation on this question "what rule could restrict our beliefs to just propositions that can be decided, without excluding a priori anything true?" is known to hopeless in a strong sense.

Incidentally I think the phrase "in principle" isn't doing any work in your koan.

Meaningful seems like a odd word to choose, as it contains the answer itself. What rule restricts our beliefs to just propositions that can be meaningful? Why, we could ask ourselves if the proposition has meaning.

The "atoms" rule seems fine, if one takes out the word "atoms" and replaces it with "state of the universe," with the understanding that "state" includes both statics and dynamics. Thus, we could imagine a world where QM was not true, and other physics held sway- and the state of that world, including its dynamics, would be noticeably different than ours.

And, like daenerys, I think the statement that "Elaine is a post-utopian" can be meaningful, and the implied expanded version of it can be concordant with reality.

[edit] I also wrote my koan answers as I was going through the post, so here's 1:

Supposing that knowledge only exists in minds, then truth judgments- that is, knowledge that a belief corresponds to reality- will only exist in heads, because it is knowledge.

The postmodernists are wrong if they seek to have material implications from this definitional argument. What makes truth judgments special compared to other judgments is that we have access to the same reality. If Sally believes that the marble is in the basket and Anne believes the marble is in the box, the straw postmodernist might claim that both have their own truth- but two beliefs do not generate two marbles. Sally and Anne will both see the marble in the same container when they go looking for it.

Again, the bare facts agree with the postmodernists- Sally and Anne would need to look to see where the marble is, which they can hardly do without their heads! But the lack of an unthinking truth oracle does not make "the concordance of beliefs with reality"- what I would submit as a short definition of truth- a useful and testable concept.

And 2:

Quite probably, as it would want to have beliefs about the potential pasts and futures, or counterfactuals, or beliefs in the minds of others.

Beliefs must pay rent.

I suspect that, if we are born, we already have a first model of physics, a few built-in axioms. As we grow older, we acquire beliefs that are only recursive applications and elaborations of these axioms.

I would say that, if a belief can be reduced to this lowest level of abstraction, it is a meaningful belief.

Proposition p is meaningful relative to the collection of possible worlds W if and only if there exist w, w' in W such that p is true in the possible world w and false in the possible world w'.

Then the question become: to be able to reason in all generality what collection of possible worlds should one use?

That's a very hard question.

"God's-eye-view" verificationism

A proposition P is meaningful if and only if P and not-P would imply different perceptions for a hypothetical entity which perceives all existing things.

(This is not any kind of argument for the actual existence of a god. Downvote if you wish, but please not due to that potential misunderstanding.)

Propositions must be able in principle to be connected to a state of how the world could-be, and this connection must be durable over alternate states of basic world identity. That is to say, it should be possible to simulate both states in which the proposition is true, and states in which it is not.

I don't think there can be any such rule.

Internal consistency. Propositions must be non self-contradictory. If a proposition is a conjunction of multiple propositions, then those propositions must not contradict each other.

When we try to build a model of the underlying universe, what we're really talking about it is trying to derive properties of a program which we are observing (and a component of), and which produces our sense experiences. Probably quite a short program in its initial state, in fact (though possibly not one limited by the finite precision of traditional Turing Machines).

So, that gives us a few rules that seem likely to be general: the underlying model must be internally consistent and mathematically describable, and must have a total K-complexity less than the amount of information in the observable universe (or else we couldn't reason about it).

So the question to ask is really "can I imagine a program state that would make this proposition true, given my current beliefs about my organization of the program?"

This is resilient to the atoms / QM thing, at least, as you can always change the underlying program description to better fit the evidence.

Although, in practice, most of what intelligent entities do can more precisely be described as 'grammar fitting' than 'program induction.' We reason probabalistically, essentially by throwing heuristics at a wall to see what offers marginal returns on predicting future sense impressions, since trying to guess the next word in a sentence by reverse-deriving the original state of the universe-program and iterating it forwards is not practical for most people. That massive mess of semi-rational, anticipatorially-justified rules of thumb is what allows us to reason in the day to day.

So a more pragmatic question is 'how does this change my anticipation of future events?' or 'What sense experiences do I expect to have differently as a result of this belief?'

It is only when we seek to understand more deeply and generally, or when dealing with problems of things not directly observable, that it is practical to try to reason about the actual program underlying the universe.

I'm pleased to find this post and community; the writing is thoughtful and challenging. I'm not a philosopher, so some of the post waltzes off the edge of my cognitive dance floor, yet without stumbling or missing a beat. Proposing a rule to restrict belief seems problematic; who will enforce the restriction and how will bear on the outcome being "just." So, the only just enforcer can be the individual believer. Perhaps the rule might pertain to the intersection of belief and action: beliefs may not cause actions that limit others' freedom or well-being. Person A believes the sky is blue. Person B complains that person A's belief limits their ability to believe that the sky is green. But person B's complaint is out of bounds, as it's based on B's desire for unanimity, a desire that limits others' freedom. Hmm.

For some reason, I did not find this option here (perhaps it is implied somewhere in the chains): a statement makes sense if, in principle, it is possible to imagine its sensory results in detail. Depends on whether Russell's teapot makes sense, and also suggests that 2+2=3 doesn't make sense.

Restrict propositions to observable references? (Or have a rule about falsifiablility?)

The problem with the observable reference rule is that sense can be divorced from reference and things can be true (in principle) even if un-sensed or un-sensable. However, when we learn language we start by associating sense with concrete reference. Abstractions are trickier.

It is the case that my sensorimotor apparatus will determine my beliefs and my ability to cross-reference my beliefs with other similar agents with similar sensorimotor apparatus will forge consensus on propositions that are meaningful and true.

Falsifiability is better. I can ask another human is Orwell post-Utopian? They can say 'hell no he is dystopian'... But if some say yes and some say no, it seems I have an issue with vagueness which I would have to clarify with some definition of criteria for post-Utopian and dystopian.

Then once we had clarity of definition we could seek evidence in his texts. A lot of humanities texts however just leave observable reference at the door and run amok with new combinations of sense. Thus you get unicorns and other forms of fantasy...

All the propositions must be logical consequences of a theory that predicts observation, once you've removed everything you can from the theory without changing its predictions, and without adding anything.

It seems to me that we at least have to admit two different classes of proposition:

1) Propositions that reflect or imply an expectation of some experiences over others. Examples include the belief that the sky is blue, and the belief that we experience the blueness of the sky mediated by photons, eyes, nerves, and the brain itself.

2) Propositions that do not imply a prediction, but that we must believe in order to keep our model of the world simple incomprehensible. An example of this would be the belief that the photon continues to exist after taxes outside of our light cone.

Solomonoff induction! Just kidding.

If I, given a universal interface to a class of sentient beings, but without access to that being's language or internal mind-state, could create an environment for each possible truth value of the statement, where any experiment conducted by a being of that class upon the environment would reflect the environment's programmed truth value of the statement, and that being could form a confidence of belief regarding the statement which would be roughly uniform among beings of that class and generally leaning in the direction of the programmed truth value, then the statement has meaning.

In other words, I put on my robe and wizard's cap, and you put on your haptic feedback vest and virtual reality helmet, and you tell me whether Elaine is a Post-Utopian.

This should cover propositions whose truth-value might not be knowable by us within our present universe if we can craft the environment such that it is knowable via the interface to the observer. e.g. hyperluminal messaging / teleportation / "pause" mode / "ghost" mode, debug HUDs, etc.

Explicitly assuming realism and reductionism. I think.

A meaningful statement is one that claims the "actual reality" lies within a particular well-defined subset of possible worlds, where each possible world is a complete and concrete specification of everything in that universe, to the highest available precision, at the lowest possible level of description, in that universes own ontology.

Of course, superhuge uncomputable subsets of possible worlds are not practically useful, so we compress by talking about concepts (like "white", "snow", "5", "post-utopian"), among other things. Unfortunately, once we get into turing-complete compression, we can construct programs (concepts) that do all sorts of stupid stuff (like not halt). Concepts need to be portable between ontologies. This might sink this whole idea.

For example, "snow is white" says the One True Reality is within the (unimaginably huge) subset of possible worlds where the substructures that the "snow" concept matches are also matched by the "white" concept.

For example "2 + 2 = 5" refers to the subset of possible worlds where the concept generated by the application of the higher-order concept "+" to "2" and "2" will match everything matched by "5". (I unpacked "=" to "concepts match same things", but you don't have to) There's something really neat about these abstract concepts, but they don't seem fundamentally different from other ones.

TL;DR: So the rule is "your beliefs should be specified by a probability distribution over exact possible worlds", and I don't know of a compression language for possible world subsets that can't express meaningless concepts (and it probably isn't worth it to look for one).

"A statement can be meaningful if a test can be constructed that will return only one result, in all circumstances, if the statement is true."

Consider the satement: If I throw an object off this cliff, then the object will fall. The test is obvious; I can take a wide variety of objects (a bowling ball, a rock, a toy car, and a set of music CDs by ) and throw them off the cliff. I can then note that all of them fall, and therefore improve the probability that the statement is true. I can then take one final object, a helium balloon, and throw it off the cliff; as the balloon rises, however, I have therefore shown that the statement is false. (A more correct version would be "if I throw a heavier-than-air object off this cliff, then the object will fall." It's still not completely true yet - a live pigeon is heavier than air - but it's closer).

By this test, however, the statement "Carol is a post-utopian author" is meaningful, as long as there exist some features which are the features of post-modern authors (the features do not need to be described, or even known, as long as their existence can be proven - repeatable, correct classification by a series of artificial neural networks would prove that such features exist).

Here's my first swing at it: A proposition is meaningful if it constrains the predicted observations of any theoretically possible observer.

This way, the proposition "the unmanned starship will not blink out of existence when it leaves my light cone" is meaningful because it's possible that there might potentially be an observer nearby who observes the starship not disappear.

On the other hand, the statement "The position of this particle is exactly X and its momentum exactly P" is not meaningful under this rule, and that's a feature.

My $0.02:

A proposition P is meaningful to an observer O to the extent that O can alter its expectations about the world based on P.

This doesn't a priori exclude anything that could be true, although for any given observer it might do so. As it should. Not every true proposition is meaningful to me, for example, and some true propositions that are meaningful to me aren't meaningful to my mom.

Of course, it doesn't necessarily exclude things that are false, either. (Nor should it. Propositions can be meaningful and false.)

For clarity, it's also perhaps worth distinguishing between propositions and utterances, although the above is also true of meaningful utterances.

Maps are models of the territory. And the usefulness of them is often that they make predictions about parts of the territory I haven't actually seen yet, and may have trouble getting to at all. The Sun will come up in the morning. There isn't a leprachaun colony living a mile beneath my house. There aren't any parts of the moon that are made of cheese.

I have no problem saying that these things are true, but they are in fact extrapolations of my current map into areas which I haven't seen and may never see. These statements don't meaningfully stand alone, they arise out of extrapolating a map that checks out in all sorts of other locations which I can check. One can then have meaningful certainty about the zones that haven't yet been seen.

How does one extrapolate a map? In principle I'd say that you should find the most compressible form - the form that describes the territory without adding extra 'information' that I've assumed from someplace else. The compressed form then leads to predictions over and above the bald facts that go into it.

The map should match the territory in the places you can check. When I then make statements that something is "true", I'm making assertions about what the world is like, based on my map. As far as English is concerned, I don't need absolute certainty to say something is true, merely reasonable likelihood.

Hence the photon. The most compressible form of our description of the universe is that the parts of space that are just beyond visibility aren't inherently different from the parts we can see. So the photon doesn't blink out over there, because we don't see any such blinking out over here.

If by "meaningful" you mean "either true or false" and by "meaningless" you mean "neither true nor false", then a Platonist and a formalist would disagree about the meaningfulness of the continuum hypothesis. Since I don't know any knockdown argument for either Platonism or formalism, I defy everyone who claims to have a crisp answer to your question, including possibly you.

OK. Here's my best shot at it.

Firstly, I don't really like the wording of the Koan. I feel like a more accurate statement of the fundamental problem here is "What rule could restrict our beliefs to propositions that we can usefully discuss whether or not they are true without excluding any statements for which we would like be base our behavior on whether or not they are true." Unfortunately, on some level I do not believe that there is a satisfactory answer here. Though it is quite possible that the problem is with my wanting to base my behavior on the truth of statements whose truth cannot be meaningfully discussed.

To start with, let's talk about the restriction about restricting to statements for which we can meaningfully discuss whether or not they are true. Given the context of the post this is relatively straightforward. If truth is an agreement between our beliefs and reality, and if reality is the thing that determines our experiences, then it is only meaningful to talk about beliefs being true if there are some sequences of possible experiences that could cause the belief to be either true or false. This is perhaps too restrictive a use of "reality", but certainly such beliefs can be meaningfully discussed.

Unfortunately, I would like to base my actions upon beliefs that do not fall into this category. Things like "the universe will continue to exist after I die" does not have any direct implications on my lifetime experiences, and thus would be considered meaningless. Fortunately, I have found a general transformation that turns such beliefs into beliefs that often have meaning. The basic idea is to instead of asking directly about my experiences to instead use Solomonoff induction to ask the question indirectly. For example, the question above becomes (roughly) "will the simplest model of my lifetime experiences have things corresponding to objects existing at times later than anything correspond to me?" This new statement could be true (as it is with my current set of experiences), or false (if for example, I expected to die in a big crunch). Now on every statement I can think of, the above rule transforms the statement A to a statement T(A) so that my naive beliefs about A are the same as my beliefs about T(A) (if they exist). Furthermore, it seems that T(A) is still meaningless in the above sense only in cases where I naively believe A to actually be meaningless and thus not useful for determining my behavior. So in some sense, this transformation seems to work really well.

Unfortunately, things are still not quite adding up to normality for me. The thing that I actually care about is whether or not people will exist after my death, not whether certain models contain people after my death. Thus even though this hack seems to be consistently giving me the right answers to questions about whether statements are true or meaningful, it does not seem to be doing so for the right reasons.

Only propositions that constrain our sensory experience are meaningful.

If it turns out that the cosmologists are wrong and the universe begins to contract, we will have the opportunity to make contact with the civilization that the colonization starship spawns. The proposition "The starship exists" entails that the probability of the universe contracting and us making contact with the descendants of the passengers of the starship is substantial compared to the probability of the universe contracting.

First, our territory is a map. This is by nature of evolving at a physical scale in which we exist on a type of planet (rather than at the quantum level or the cosmological level) and of a century/day/hour scale conception of time (rather than geological or the opposite) and of a species in which experience is shared, preserved, and consequently accumulated. Differentiating matter is of that perspective, labeling snow is of that perspective, labeling is of that perspective, causation, and so on.

By nature of being, we create a territory. For a map to be true (I don't like 'meaningful'), it must correspond with the relevant territory. So, we need more than a laplacian demon to restrict beliefs to propositions that can be true, we need a demon capable of having a perfect and imperfect understanding of nature. It'd have to carve out all possible territories (of which can conflict) from our block universe and see them from all possible perspectives, and then you would have to specify which territory you want to see corresponds with whatever map.

Meaningful means it exists. By virtue of (variants of) the macroscopic decoherence interpretations of quantum mechanics and the best understanding I and three other long time rationalists have of cosmology, everything physically possible exists, either in a quantum mechanical branch or in another hubble volume.

To narrow it down a bit (but not conclusively) start out by eliminating all propositions that presuppose violation of conservation of energy, that should give you a head start.

Anything physically possible exists within our timeless universe-structure's causal closures: When we talk "meaninful" or "not meaningful" we are really talking physics or not physics. Perpetual motion, for instance, isn't physics. Neither is (as far as I know) faster than light travel or communication, reversing entropy, ontologically basic mental entities and a lot of other things. They do not exist in any world in our universe, thus not meaningful, not a thing you can experience.

This of course presupposes knowledge of physics... I'll have to mull on that. Funny disagreeing with yourself while typing.

Can you amplify? I'd thought I'd looked this up.

If an author actually being X has no consequences apart from the professor believing that the author is "X", all consequences accrue to quoted beliefs and we have no reason to believe the unquoted form is meaningful or important. As for p-zombieness, it's not clear at this point in the sequence that this belief is meaningless rather than being false; and the negation of the statement, "people are not p-zombies", has phrasings that make no mention of zombiehood (i.e., "there is a physical explanation of consciousness") and can hence have behavioral consequences by virtue of being meaningful even if its intuitive "counterargument" has a meaningless term in it.

I agree - atoms and so forth are what our universe happens to consist of. But I can't see why that's relevant to the question of what truth is at all - I'd say that the definition of truth and how to determine it are not a function of the physics of the universe one happens to inhabit. Adding physics into the mix tends therefore to distract from the main thrust of the argument - making me think about two complex things instead of just one.

In addition to what pleeppleep said, I think there is a bit of illusion of transparency here.

As I've said elsewhere, what Eliezer clearly intends with the label "political" is not partisan electioneering to decide whether the community organizer or the business executive is the next President of the United States. Instead, he means something closer to what Paul Graham means when he talks about keeping one's identity small.

Among humans at least, "Personal identity is the mindkiller."

He didn't say "politics" was special. He seemed to be pointing out that updating is called for in circumstances other than the example. "Politics" is used to represent all other issues, and it was relevant because a common criticism of truth is that it is an illusion used to gain a political advantage.

Agreed. The previous illustrations were pretty awesome, and this post has lost a lot for it.

Agreed. The stick figures do not mesh well with the colourful cartoony backgrounds that make the images visually appealing. They feel out of place, and I found it harder to tell when I was supposed to consider one stick figure distinct from another one without actively looking for it (I also have this problem with xkcd).

Strong vote for return to the original style diagrams, with the gender imbalance fixed.

[looks back at the top-level post] Yes, they are. Especially the professor in the last picture -- it reminds me of Jack Skellington from A Nightmare Before Christmas. Using thinner lines à la xkcd would be better, IMO.

I didn't see the old stick figures, but I think the ones that are there now are fine.

Good quote, but what about the reality that I believe something? ;) The fact that beliefs themselves are real things complicates this slightly.

I could figure some cases where would find it natural to say that one proposition is more accurate then another, but not to say that it is more true. For example, saying that my home has 1000 ft.², as opposed to saying that has 978.25 ft.² Or saying that it is the morning, as opposed to saying that it is 8:30 AM.

Perhaps this: "accuracy" is a quantitative measue when "truth" is only qualitative/categorical.

"Truth" and "accuracy" are just words, and there is no inherent difference between them.

That said, if you wanted to assign useful meaning to the two, you could use truth as a noun to describe the condition of belief matching reality, and accuracy as an adjective to refer to the place of a condition on a scale of proximity between belief and reality.

Or, you could use them the other way around.

Or, you could use both words as nouns in one context, and adjectives in another. This is usually the case, with accuracy more likely to be used as an adjective as it implies lack of confidence to some degree.

I hope to write an article in the future on the uses of tools like deconstruction in the rationality and bias reduction enterprises of this site.

Please do. (But . . . with paragraphs?)

I proffer the following quotes rather than an entire article (I think the major problem with post-modernism isn't irrationality, but verbosity. JUST LOOK AT YOURSELF):

"For the sake of sanity, use ET CETERA: When you say 'Mary is a good girl!' be aware that Mary is much more than 'good'. Mary is 'good', nice, kind, et cetera, meaning she also has other characteristics." - A.E. Van Vogt, World of Null-A

"For the sake of sanity, use QUOTATIONS: For instance 'conscious' and 'unconscious' mind are useful descriptive terms, but it has yet to be proved that the terms themselves accurately reflect the 'process' level of events. They are maps of a territory about which we can possibly never have exact information. Since Null-A training is for the individuals, the important thing is to be conscious of the 'multiordinal' -that is the many valued- meaning of the words one hears or speaks." - A.E. Van Vogt, World of Null-A

A guess: prerequisites. Also, we have lots of new people, so to be safe: prerequisites to prerequisites.

Prereqs.

We know there's such a thing as reality due to the reasons you mention, not truth - that's just a relation between reality and our beliefs.

"Arrangements of atoms" play a role in the idea that not all "syntactically correct" beliefs actually are meaningful and the last koan asks us to provide some rule to achieve this meaningfulness for all constructible beliefs (in an AI).

At least that's my understanding...

The Bibliography of Eliezer’s book, Rationality: From AI to Zombies, may be of interest to you.

What does it tell about me that I immediately thought ‘what about placebo and stuff’

Your beliefs about the functionality of a "medicine," and the parts of your physiology that make the placebo effect work, are both part of reality. Your beliefs can, in a few (really annoying!) cases, affect their own truth or falsity, but whenever this happens there's a causal chain leading from the neural structure in your head to the part of reality in question that's every bit as valid as the causal chain in the shoelace example.

what about sentences whose meaning depends on the context

Ehn, the truth value depends on context too. "That girl over there heard what this guy just said" is true if that girl over there heard what this guy just said, false if she didn't, and meaningless if there's no girl or no guy or he didn't say anything.

what kind of issue EY is neglecting does this correspond to

Common knowledge, in general?

what about placebo and stuff

Beliefs are a strict subset of reality.

Connotation. The statement has no well-defined denotation, but people say it to imply other, meaningful things. Islam is a religion of peace!

Is it possible to tell them apart?

Why would you want to?

How are you encoding the non-meaningful statements? If they're encoded as characters in a string, then yes we can tell them apart (e.g. "fiurgrel" !== "dkaldjas").

Why do you want to tell them apart?

his masterpiece

Really, more like his student work. It was "Blog every day so I will have actually written something" not "Blog because that is the ultimate expression of my ideas".

Generalized across possible maps and possible cities, if your map of a city says "p" if and only iff p

If you can generalize over the correspondence between p and the quoted version of p, you have generalized over a correspondence schema between territory and map, ergo, invoked the idea of truth, that is, something mathematically isomorphic to in-general Tarskian truth, whether or not you named it.

Well, yeah, we can taboo 'truth'. You are still using the titular "useful idea" though by quantifying over propositions and making this correspondence. The idea that there are these things that are propositions and that they can appear both in quotation marks and also appear unquoted, directly in our map, is a useful piece of understanding to have.

I think politics is used to refer to social manipulation, status, and signaling here. The example is used to designate an agent that has no concern for asserting social privilege over others.

In addition to what pleeppleep said, I think there is a bit of illusion of transparency here.

As I've said elsewhere, what Eliezer clearly intends with the label "political" is not partisan electioneering to decide whether the community organizer or the business executive is the next President of the United States. Instead, he means something closer to what Paul Graham means when he talks about keeping one's identity small.

Among humans at least, "Personal identity is the mindkiller."

Response to the Second Meditation

For an AI that had no need to communicate with other agents, then the idea of truth serves as a succinct term for the map-territory/belief-reality correspondence.

It allows the AI to be more economical/efficient in how it stores information about its maps.

That's some value.

Saying that a proposition is true, is saying that it's an accurate description of the territory.

Tarski's Litany: "The sentence 'X' is true iff X."

The territory may be physical reality ("'the sky is blue' is true"), a formal system ("'2 + 2 = 4' is true"), other maps, etc.

I sort of get the point. I remember once reading here that the reason it is a decent choice to use certain axioms also used in rationality and science is that those axioms have a pretty decent track record of helping to find out truth. A track record better than say...philosophy?

If this wasn't clear: responses would be much more helpful than up/down votes.

The downvotes and no reply are a pretty good example of what's wrong with less wrong. Someone who is genuinely confused should not be shooed away then insulted when they ask again.

First of all remember to do and be what's best. If this doubt is engendering good attitudes in you, why not keep it? The rest of this is premised on it not helping or being unhelpful.

External reality is much more likely than being part of a simulation which adjusts itself to your beliefs because a simulation which adjusts itself to your beliefs is way, way more complicated. It requires more assumptions than a single level reality. If there's a programmer of your reality, that programmer has a reality too, which needs to be explained in the same way a single level one should as does their ability to program such a lifelike entity and all sorts of other things.

More fundamentally though, this is just the reality you live in, whatever its position in a potential reality chain.

If we are being simulated, trying to metagame potential matrix lords' dispositions/ ask for favours/look for loopholes/care less about its contents is only a bug of human cognition. If this is a simulation, it is inhabited by at least me, and almost certainly many other people, and there's real consequences for all of us. If you don't earn your simulation rent you'll get kicked out of your simulation place. Qualify everything with "potentially simulated-" and it changes nothing. "Real" just isn't a useful (and so, important) distinction to make in first person reguarding simulations.

and/or you could short circuit any debilitating doubt using fighting games or sports (or engaging in other similiar activities) which illustrate the potential importance of leaning all in towards the evidence without worrying about the nature of things, and are a good way to train that habit.

Also, in this potentially simulated world, social pressure is a real thing. The more infallible and sensitive you make your thinking (or allow it to be) the more prone it is to interference from people who want to disrupt you, unless you're willing to cut yourself off from people to some extent. When someone gives you an idiotic objection (and there are a lot of those here), the more nuanced your own view actually is the harder it will be to explain and the less likely people will listen fairly. You could just say whatever you think is going to influence them best but that adds a layer of complexity and is another tradeoff. If you're not going to try to be a "philosopher of perfect emptiness" taking external reality as an assumption is the most reliable to work with your human mind, and not confuse it: how are you supposed to act if there are matrix lords? There's nothing to go on so any leaning such beliefs (beliefs which shouldn't change your approaches or attitudes) prompts is bound to be a bias.

Give me an example of a specific thing relevant to constructing an AI which I should have referenced, plus the role it plays in a (self-modifying) AI. Keep in mind that I only care about constructing self-modifying AIs and not about "what is the bearer of truth".

I've read works-not-referenced on "meaning", they just don't seem relevant to anything I care about. Though obviously there's quite a lot of standard work on mathematical proof that I care about (some small amount of which I've referenced).

I think "postmodernism," "colonial alienation," and "post-utopianism" are all meant to be blanks, which we're supposed to fill in with whatever meaningless term seems appropriate.

But I share your uneasiness about using these terms. First, I don't know enough about postmodernism to judge whether it's a field filled with empty phrases. (Yudkowsky seems to take the Sokal affair as a case-closed demonstration of the vacuousness of postmodernism. However, it is less impressive than it may seem at first. The way the scandal is presented by some"science-types"--as an "emperor's new clothes" story, with pretentious, obfuscationist academics in the role of the court sycophants--does not hold up well after reading the Wikipedia article. The editors of Social Text failed to adhere to appropriate standards of rigor, but it's not like they took one look at Sokal's manuscript and were floored by its pseudo-brilliance.)

Second, I suspect there aren't any clear-cut examples of meaningless claims out there that actually have any currency.(I only suspect this; I'm not certain. Some things seem meaningless to me; however, that could be just because I'm an outsider.)

Counterexamples?

If people agree on what the term means, and they can read a book and detect (or not) colonial alienation, and thus have a test for post-utopianism, and different people will reach the same conclusions about any given book

By hypothesis, none of those things are true. If those things happen to be true for "post-utopianism" in the real world, substitute a different word that people use inconsistently and doesn't refer to anything useful.

Truth is more about how you get to know reality than it is about reality. For instance, it is easy to conceive of a possibility where everything a person knows about something points to it being true, even if it later turns out to be false. Even if you do everything right, there's no cosmic guarantee that you have found truth, and therefore cut straight through to reality.

But it is still a very important concept. Consider: someone you love is in the room with you, and all the evidence available to you points to a bear trying to get into the room. You would be ill-advised to second-guess your belief when there's impending danger.

Wouldn't "snow is white" be a true statement if people weren't around?

Not exactly. White isn't a fundamental concept like mass is. Brain perception of color is an extremely relative and sticky issue. When I go outside at night and look at snow, I'd swear up and down that the stuff is blue.

If people weren't around, then "snow is white" would still be a true sentence, but it wouldn't be physically embodied anywhere (in quoted form). If we want to depict the quoted sentence, the easiest way to do that is to depict its physical embodiment.

I think there was only one marble in the universe.

You ought to admit that the statement 'there is "the thingy that determines my experimental results"' is a belief.

Why? Didn;t anyone ever see results that conflict with their beliefs?

The actual belief is "This thingy which determines my experimental results is internally consistent and the rules governing it are time-invariant."

Where are your experimental results? Where are your beliefs? If they aren't the same thing, how can you compare them?

And finally: What would you expect to see if the thingy which determined the results of your experiments didn't have the qualities you ascribe to it? Try to avoid putting the question a meta-level up; my conclusion is that there is no evidence which doesn't support the premise that what I call reality is capricious and transient- but that if it is, there is no change in expected outcome from any decision I can make.

https://corticalchauvinism.files.wordpress.com/2014/04/dse232_1_004i.jpg

I think...

You're basically attacking (one of?) the strongest tenet of LessWrong culture with practically no basis other than "presumably", "Chalmers and others" as an authority (Chalmers' words are not taken as Authority here, and argument trumps authority anyway), and some vague phrasings about "physical terms", "by it's very nature [sic]", "can't be reduced" and "properties of quantum fields".

My own best interpretation is that you're making a question-begging ontological argument that information, learning, knowledge, consciousness or whatever other things are implied by your vague wording are somehow located in separate magisteria.

Also, please note that, as discussed in more details in the other articles following this one, Eliezer clearly states that these epistemic techniques don't rule out a priori any concepts just because they don't fit with some materialistic physical laws one assumes to be true.

Yes.

Can we verify or falsify it? Yes, iff it somehow constricts possible realities in a manner that is exclusively different from other hypotheses and in-principle observable from our reference frame(s) assuming we eventually obtain the means to make relevant observations.

Retracted

I think this includes too much. It would includes meaningless beliefs. "Zork is Pork." True or false? Consistency seems to me to be, at best, a necessary condition, but not a sufficient one.

And this sort of belief can have behavioral consequences!

If I understand it correctly, (and I am not sure, feel free to correct me.) it occurs to me that belief may have a very unusual consequence indeed, which seems to be believing that "The photon continues to exist, heading off to nowhere." is true implies that you should also believe the Probability of world P1 is greater than Probability of world P2 below.

P1: "You are being simulated on a supercomputer which does not delete anything past your cosmological horizon."

P2: "You are being simulated on a supercomputer which deletes anything past your cosmological horizon."

Which sounds like a very odd consequence of believing "The photon continues to exist, heading off to nowhere." is true, but as far as I can tell, it appears to be the case.

That's an attempt to dismiss epistemic rationality by arguing that only instrumental rationality matters.

I suppose that's true by certain definitions of "matters", but it ignores those of us who do assign some utility to understanding the universe itself, and therefore at least partially incorporate the epistemic in the instrumental....

Also, if I die tomorrow of a heart attack, I think it's still meaningful to say that the rest of the planet will still exist afterwards, even though there won't exist any experimental prediction I can make and personally verify to that effect. I find solipsism rather uninteresting.

You're applying TheOtherDave's definition in a manner that he himself has disavowed. He believes, for instance, that on his account the existence claims of simpler theories should be believed over the existence claims of more complex theories, even if those theories make the same experimental predictions. This would legitimize Eliezer's claim that the photon continues to exist, rather than blinking out, since it is a simpler model. See the discussion following this comment of mine.

Why is it accepted that experiments with reality prove or disprove beliefs?

Well, in one sense it isn't accepted... not if you want "prove" to mean something monolithic and indisputable. If a proposition starts out with a probability between 0 and 1, no experiment can reduce that probability to 0 or raise it to 1... there's always a nonzero probability that the experiment itself was flawed or illusory in some way.

But we do accept that experiments with reality give us evidence on the basis of which we can legitimately increase or decrease our confidence in beliefs. In most real-world contexts, that's what we mean by "prove": provide a large amount of evidence that support confidence in a belief.

So, OK, why do we accept that experiments do that?

Because when we predict future experiences based on the results of those experiments, we find that our later experiences conform to our earlier predictions.

Or, more precisely: the set of techniques that we classify as "reliable experiments"are just those techniques that have that predictive properties (sometimes through intermediate stages, such as model building and solving mathematical equations). Other, superficially similar, techniques which lack those properties we don't classify that way. And if we found some superficially different technique that it turned out had that property as well, we would classify that technique similarly. (We might not call it an "experiment," but we would use it the same way we use experiments).

Of course, once we've come to trust our experimental techniques (and associated models and equations), because we've seen them work over and over again on verifiable predictions, we also develop a certain level of confidence in the _un_verifiable predictions made by the same techniques. That is, once I have enough experience of the sun rising in the morning that I am confident it will do tomorrow, (including related experiences, like those supporting theories about the earth orbiting the sun etc., which also serve to predict that event), I can be confident that it will rise on October 22 2143 even though I haven't yet observed that event (and possibly never will).

So, yes. If I jump off a cliff I might start out with theories that seem to predict future behavior, and then later have unpredicted experiences as my speed improves that cause me to change those theories. Throughout this process, what I'm doing is using my observations as evidence for various propositions. "Reality" is my label for the framework that allows for those observations to occur, so what we call this process is "observing reality."

What's confusing?

It seems to me that they merely confirm or alter beliefs.

And one of the beliefs they've confirmed is "reality is really real, it isn't just a belief." :-)