Truth seeking is motivated cognition

Either I completely don't get it, or this is some kind of sophistry that seems deep. Humans make a choice what they mean when they pronounce the sound "truth". Therefore, truth is arbitrary.

The same argument can be made about anything else. Humans make a choice what they mean when they pronounce the sound "circle". Therefore, circles are arbitrary.

By learning to look up, even just a little, we can break free of our self-construct dreams and start to engage with the world as it really is.

Okay, so instead of using the word "truth" we should be saying "world as it really is"? Other than getting an extra point for poetry, I think this is what many people already mean by truth.

Similarly, the problem with the criterion -- I have no idea whether I agree or disagree with you here -- is that we gradually learn about the world, and the criteria we used in the past may turn out to be less good than we assumed. For example, one might start with "it is true if I can see it" and then realize that sometimes things make sense even if we cannot directly observe them by our senses (because they happened in the past, happen far away, require x-ray vision, etc.), so the criterion would change to... something else, which again might require an update in the future. That does not make it arbitrary, it just makes it... learning.

I think your argument here equivocates between two different claims.

1. "When we use the word 'truth' or 'true' we may mean different things by it, so the meaning of a sentence with 'truth' or 'true' in it is dependent on somewhat-arbitrary choices made by humans."
2. "That specific thing you (or I) mean by 'truth' is dependent on somewhat arbitrary choices made by humans."

The first is hard to disagree with. (And the same applies for literally any other term as well as "truth"/"true".) The second, not so much.

An analogy: Suppose something is vibrating and I say "The fundamental frequency of that vibration is approximately 256 Hz". Just as we can all propose subtly (or not so subtly) different ideas of what it means to say that something is "true", so we can all propose different definitions for "hertz".[1] Or for that matter for "fundamental" or "frequency". So two people making that statement might mean different things. But I don't think it's helpful to say that this means that the fundamental frequency of an oscillation is subjective. Once you decide what you would like "fundamental frequency" to mean and what units you'd like it to be in, any two competently done measurements will give the same value.

[1] If you think this is silly, you might want to suppose that instead I had said "... is approximately that of middle C". You could measure frequency in "octaves relative to middle C" exactly as well as in hertz, but different groups of people at different times really have called different frequencies "middle C".

Similarly, at least prima facie it's possible that (a) everything you say about the existence of different criteria-for-truth is correct but none the less (b) there is a fact of the matter, not dependent on anyone's kinda-arbitrary decisions, about e.g. what things remain when you stop believing in them, or what beliefs will reliably lead a given class of agent to more accurate predictions about the future, or what sets of beliefs and inference rules constitute consistent formal systems.

Perhaps it turns out that for some or many or all plausible notions of truth (b) is not, er, true, so that what I claimed claim 2 above is, er, true. That would be an interesting, er, truth -- to me, much more interesting than the less controversial claim 1. But if you've given any reason here for believing it, I haven't seen it.

I have the sense that it's ferociously difficult to get at the kind of thing you're pointing at here in purely conceptual terms. I wonder if it might help to give some examples of where people have made the kind of mistake you're point at here, and perhaps the solution that they've been missing. I have the sense that the proof-based agents line of research ran right into this issue via the limits to the coherence operationalization of truth in your list. I also have the sense that we are at the moment running into the limits of the predictive operationalization of truth when we try to locate human values within physical human brains. The most interesting one to me, though, is: what is the truth status of practice as a path to the end of suffering?!

I have the sense that it's ferociously difficult to get at the kind of thing you're pointing at here in purely conceptual terms. I wonder if it might help to give some examples of where people have made the kind of mistake you're point at here, and perhaps the solution that they've been missing. I have the sense that the proof-based agents line of research ran right into this issue via the limits to the coherence operationalization of truth in your list. I also have the sense that we are at the moment running into the limits of the predictive operationalization of truth when we try to locate human values within physical human brains. The most interesting one to me, though, is: what is the truth status of practice as a path to the end of suffering?!

On a meta level, you've written a bunch about the problem of the criterion, and I don't really feel like I see much productive from it. The problem with the problem of the criterion is that in general the appropriate criterion depends on what criterion you have, on the case you're dealing with. So it's extremely limited what universal things you can say about the problem of the criterion.

Less cryptically, let's take an example. What is "truth"? Well, concretely, my girlfriend recently lost her phone, what defines the truth of "where her phone was"? Well, it's the place where her phone was; the place she could go to and reach for to get the phone. That's fairly simple, and it's a resolution that is straightforward and intrinsic to the sentence of "where her phone was", rather than general.

Now you could say that "where her phone was" is defined based on what she cares about. But the sentence makes sense to you too, even though you don't know her and aren't really affected by the sentence. So clearly language is much more broad than what you just care about.

You claim:

[...9 It's easy to think about truth as a concept and argue which notion of truth is best. It's much harder to think both about how to evaluate if something is true and remember at the same time that this method of evaluation is contingent on which method you think is best.

[...]

Why argue all this? Because I think people suffer and cause unnecessary harm because they get wrapped up in their own ideas about truth (and many other things!). By learning to look up [LW · GW], even just a little, we can break free of our self-construct dreams and start to engage with the world as it really is. And if we can do that, maybe we can make a little progress on some of the hard problems we face instead of spinning our wheels trying to solve the problems we make up in our heads.

But none of this philosophizing actually helped find the phone. Breaking free of our self-construct dreams of "where her phone was" wouldn't help at all; what did help was calling her phone and following the sound, which made it turn out that it was hidden behind a bag on the table. This only really worked because we had a good map-territory correspondence that allowed us to understand what happened in various cases, which requires a vary solid grasp on truth, and it's probably also more available because we've learned this heuristic from other cases where a lost phone was a problem.

Would you like to discuss a stronger claim, that motivated cognition may be a good epistemology?

Usually people use "logical reasoning + facts". Maybe we can use "motivated reasoning + facts". I.e. seek a balance between desirability and plausibility of a hypothesis. 

Important post. The degree to which my search for truth is motivated, and to what ends, is something I grapple with frequently. I generally prefer the definition of truth as "that which pays the most rent in anticipated experience"; essentially a demand for observability and falsifiability, a combination of your correspondence and predictive criteria. This, of course, leaves what is true subject to updating if new ideas lead to better results, but I think it is the best way we have of approximating truth. So I'm constantly looking really hard at the evidence I examine and asking myself, am I convinced of this for the right reasons? What would have to happen to unconvince me? How can I take a detached stance toward this belief, if ever there comes a time when I may no longer want it? So in what way my truth-seeking could be called motivated, I aim to constrain it to at least being solely motivated by adherence to the scientific method, which is something I am unashamed to simply acknowledge.

And that means whatever we want to claim to be true is ultimately motivated by whatever it is we care about that led us to choose the definition of truth we use.

People who speak different languages don't use the symbols "truth". To what extent are people using different definitions of "truth" just choosing to define a word in different ways and talk about different things. 

In an idealized agent, like AIXI, the world modeling procedure, the part that produces hypothesis and assigns probabilities, doesn't depend on it's utility function. And it can't be motivated. Because motivation only works once you have some link from actions to consequences, and that needs a world model. 

If the world model is seriously broken, the agent is just non functional. The workings of the world model isn't a choice for the agent. It's a choice for whatever made the agent. 

It pays to taboo the term, as I've been advocating for years here, with little success. 

Say what you really mean instead of this nebulous misleading concept! Sometimes it is a truth value of a provable or disprovable mathematical statement (e.g. the Pythagoras theorem), sometimes it is someone's best guess at the truth value of a mathematical statement (P is is very likely provably not equal NP), sometimes it is a statement about accuracy of some model of the physical world (e,g, Quantum Mechanics is "true" in its domain of applicability), sometimes it is a statement of faith ("my truth" vs "your truth") etc.

Tabooing "truth" avoids pointless arguments over statements of the form "unprovable/untestable but true", like MWI is obviously true", or "Genghis Khan liked horse milk" or "BB(10)'th digit of Pi has 10% probability of being 0". Alternatives to the term "true" are "testably accurate", "holds in all but measure zero possible worlds, given a certain set of assumptions", "something I fervently believe in" and other items from your list. 

"Fact" is another term that is worth tabooing, I call it yet another four-letter f-word.

My notion of truth doesn't fit with any of the theories you listed. Truth is a relationship between propositions and the world, e.g. the proposition "this comment contains 1 or more y's" is true because this comment contains 1 or more y's.

This doesn't technically invalidate your point that truth is human-chosen. But specifically, the human-chosen element is the language we use. If we spoke a different language where the meanings of the words "more" and "fewer" were swapped, the statement would become false.

Though my counterargument here unfairly skews things to my advantage. AFAIK, there is a lot of shared structure between different human languages. Usually human languages can be translated near-losslessly into each other, but a combinatorial argument shows that this is not the case for the overwhelming majority of mathematically conceivable languages.

However, I think similar combinatorial arguments show that it is not possible to obtain information about the truth in the overwhelming majority of mathematically conceivable languages. Conceptually, if the truth of statements can depends mainly on things that you do not observe (which concepts will if they depend on random stuff, since there's a lot of unseen stuff they could potentially depend on), then you cannot learn anything about the truth.

You're saying that our languages are based on our motivations and preferences. But almost any motivations and preference would favor a language that can express concepts that are observable, as well as concepts linked to observables. I bet there's an instrumental convergence argument that could be made here; do you disagree?

I would say you if hold the correct understanding of what truth is, then truth seeking is cognition motivated by seeking truth.

So yes, it is motivated cognition. But the motivation is correct.