Open thread, October 2011

That's actually exactly what I usually try to do. Unfortunately, most advice-givers in my experience tend to mistake #4 for #3. I point out that they've made an incorrect assumption when formulating their advice, and I immediately get yelled at for making excuses. I do actually have a tendency to seek excuses for non-action, but I've been aware of that tendency in myself for a long time and counter it as vigorously as I am able to.

I suppose it couldn't hurt to explain my actual situation, though. Gooey details incoming.

I live in the southwestern suburbs of Fairfield, California, on a fixed income that's just enough to pay the bills and buy food, with a little left over. (Look the town over in Google Maps to get a sense of what kind of place it is.)

Most critically, i suffer from Non-24, which, in the past, was responsible for deteriorating health and suicidal depression during high school, for forcing me to drop even the just-for-fun classes I was taking at the community college, as well as causing me to completely lose touch with my high school acquaintances before I figured out what I had and that there was a pattern to it and not just random bouts of hypersomnia and insomnia. It rules out doing anything that involves regularly scheduled activities; I even had to quit my World of Warcraft guild because of it.

Before I lost touch with my high school acquaintances, I did get to experience some normal social gatherings, though to me there was never anything particularly fun about being pelted with straw-wrappers at Denny's or dancing to Nirvana under a strobe-light or watching them play BeerPong. None of those people were ever my friends or even much of a support structure, and I don't actually miss any of them. I've been on several dates through OkCupid and my brief time in college, but they were all failures of emotional connection and in each case I was relieved when the girl told me she didn't want to go out with me anymore. I mention this to show that I'm not just assuming certain generic solutions won't work for me; I've confirmed it by experiment.

So, I'm living without much disposable income, with a sleep disorder that precludes regularly scheduled activities of any kind, in a highway-tumor town, with no friends or contacts of any kind. Oh, and I have a mild photosensitivity condition which means I'm slaved to my sunglasses during the day and even with them can't do anything that involves exposure to direct sunlight for more than a few minutes at a time, just for the sake of thoroughness.

That's the summary of the situation.

My career goals aren't actually precluded by any of this, though becoming a successful graphic artist, or writer, or independent filmmaker or webcomic author or whatever I end up succeeding at, is made more difficult. I only included the professional category because my social goals mostly pertain to my career goals: I'd like to have a useful social network. It'd be nice to have friends just for the sake of having friends, but that's of low value to me. My only high value purely-social goal is meeting and befriending a woman with whom I can have a meaningful and lasting intimate relationship, which dissolves away the romantic category as well.

This is useful, actually. I think I've been kind of doing that indirectly, but not with a direct conscious effort. It doesn't do me much good right now, since I'm still completely isolated and don't know of anyone who got out of a situation like mine, but I think it could still be helpful.

I rather doubt there is a fully-generalizable theory of the sort you seem to be looking for. Some territories are better left than explored in more detail, if it be within your power to do so; others can be meaningfully understood and manipulated.

If you are in a dead-end job in a small town where you are socially isolated and clash culturally with the locals, do not have professional credentials sufficient to make a lateral move plausible (ie, working retail as opposed to something that requires a degree), the advice will necessarily be different than if you are in a major city and have a career of some kind.

What works in New York may not work so well in Lake Wobegon.

I think I understand the feeling you're having now. Still, It seems highly unlikely to me that you can fix this "cognitive flaw" in isolation, before you've found a few concrete avenues of advancement...I find that my habits, including habits of thought, are trainable rather than fixable in the abstract.

Are you in school? If so, would you like to study something different? If not, is there something you do want to study? Are you working or is there somewhere you want to work? These are conventional paths to life-advancement.

If the flaw lies in your choices, choose differently. If the flaw lies in your habits, practice better habits. If the flaw lies in your cognitive habits, you must do something higher up on this list in order to be able to develop different cognitive habits.

Your existing habits and choices (and arguably genetics and environment) may not be what created the situation which is becoming intolerable, but they are the easiest thing to work on.

You do not have to worry about making the right change or practices--start with whatever seems easiest. And try not to go against your 'better' judgement.

Yeah, that's not really what I was talking about. My problem is with being unable to see that there's anything I should just go out and do, not with actually going out and doing it. I don't have any trouble following a path to my goal once that path has been identified; it's identifying possible path(s) to my goal(s) in the first place that I seem to have a deficiency in. What was unclear about my question that prompted you to answer a different question than the one I asked?

There are already 'express' lanes with maximum item limits, which achieve faster service by making sure the average time to process each customer is reduced. In that case, assigning faster cashiers make sense, but it seems like the 'toll lane' idea would achieve faster service primarily by being much less crowded than other lanes (that is, if the toll lane has a line the same length as other lanes, there would be no point going to it). So having your best cashier there just ensures they spend more time idle, thereby increasing the average time for all lanes.

I share your sentiment, but you want to be a little bit careful doing this sort of thing by trial and error as there can be hysteresis effects, as with the Israeli day care experiment made famous by Freakonomics. You may find, for example, that offering sales that start after 5pm works better than charging a fee before 5pm, but only if you do it first.

this seems more likely to exist as a discount for waiting than as a surcharge for being faster

What does this mean? That it would be better for a store to have a sole 5% discount lane and no other special lanes than a 5% surcharge lane, and that it would be better for a store to have no special lanes than a 5% discount lane?

I am suggesting that a grocery store could a) unilaterally b) bring the situation closer to a Pareto optimum and c) capture much of the benefit.

To the extent consumers are rational, the store pushes off the decision to commit to a faster or cheaper store until the consumer has more information. To the extent they are irrational, it offers them an impulse purchase that is easy to rationalize.

One would compare black defendants with guilty black defendants and white defendants with guilty white defendants. It's far from obvious that (guilty black defendants/black defendants) > (guilty white defendants/white defendants). Differing arrest rates, plea bargaining etc. would be factors.

60% of convicts who have been exonerated through DNA testing are black. Whereas blacks make up 40% of inmates convicted of violent crimes. Obviously this is affected by the fact that "crimes where DNA evidence is available" does not equal "violent crimes". But the proportion of inmates incarcerated for rape/sexual assault who are black is even smaller: ~33%. There are other confounding factors like which convicts received DNA testing for their crime. But it looks like a reasonable case can be made that the criminal justice system's false positive rate is higher for blacks than whites. Of course, the false negative rate could be higher too. If cross-racial eyewitness identification is to blame for wrongful convictions then uncertain cross-racial eyewitnesses might cause wrongful acquittals.

Yes. It's important to remember that guilty defendants aren't the same thing as convicted defendants. A rational decision-maker using Bayes' theorem wouldn't necessarily put all that much weight on the decisions of past juries, knowing as we do that they're not using Bayes' theorem at all. And, of course, a Bayesian would need exactly the same amount of evidence to convict a black defendant as they did a white defendant. That question is whether skin colour counts as evidence.

The conviction rate for black defendents is sometimes much higher than the conviction rate for whites, so the solution you've suggested here would intensify the racial disparity.

No, you'd just need to keep track of how often demographic considerations influenced the outcome, so that any measure of "base rates for violence among blacks" you use for such decisions is independent of the policy.

(That's not to say that such a policy would be a good idea of course)

It's not clear that "the base rates for violence among blacks is higher" is meant to be measured by convictions. I interpreted it to be based on sociological data, for example, and in that case there would be no feedback loop. Pinker didn't cite a source, unfortunately. A very quick stroll past Google Scholar 1 2 shows that a common source used is arrest data in the FBI's Uniform Crime Report. Plainly there are also important ways in which arrest data may be biased against blacks, but I'm hesitant to simply dismiss a finding based on that difficulty as I'm willing to bet that researchers in the field would have attempted to account for the difficulty.

The percentage arrestees who are black is higher than the percentage of offenders who are black as reported by victims in surveys.

(Suggesting the base rate is screened out by the time the matter gets to the court room)

Yes indeed. And the same goes, by the way, for other kinds of rational evidence that aren't acceptable as legal evidence (hearsay, flawed forensics, coerced confessions, etc).

Do you care to elaborate? The interpretation of your response that comes to my mind is that you dissent from the moral viewpoint that Pinker expresses.

Doesn't it?

When it comes to US demographics, "black" covers a "natural" cluster of the population / identifiable blob in thingspace. Sure, there are border cases like mixed-race people and recent African immigrants, just like there are edge-cases between bleggs and rubes. "Is person X black or not?" is probably one of the top yes/no questions that would tell you the most about an American (Along with "Did he vote for Obama?", "Is he richer or poorer than the median?", or "Does he live north or south of the Mason-Dixon line?")

Sure, when it comes to world demographics, or Brazilian demographics, it doesn't cut reality at it joints as well.

For questions of, say, population genetics, I think that is quite right. But for questions of sociology or social policy I don't see why one wouldn't include 'black' as part of the ontology.

That's not too important. If I go to my closet and pull out twenty items of clothing at random, and designate those group A, and designate the rest group B, if I know what is in each group I can still make predictions about traits of random members of either group.

There are other demographic factors that could count too: sex, age, social class ... and if a policy of "OK, conviction standards for blacks are lower" would be political suicide

Isn't that the status quo?

I do believe that such demographic data can be useful to help direct resources for crime prevention, though.

Leaning more towards "increased police presence in the ghetto" or "only frisk the Muslims"?

There shouldn't be any such distinction. The audience (I assume you mean the courtroom audience) should reason the exact same way the jury does.

The prosecution is required to make an explicit presentation of the evidence for guilt, so that the mere fact that charges were brought is screened off. As a consequence, failure to present a convincing explicit case is strong evidence of innocence; prosecutors have no incentive to hide evidence of guilt! Hence any juror or audience member who reasons "the prosecution's case as presented is weak, but the defendant has a high likelihood of guilt just because they suspect him" is almost certainly committing a Bayesian error. (Indeed, this is how information cascades are formed.)

See Argument Screens Off Authority: once in the courtroom, prosecutors have to present their arguments, which renders their "authority" worthless.

Expressing disagreement as a false lack of understanding

I've found that saying, "I don't think I understand what you mean by that" or "I don't see why you're saying so" is a useful tactic when somebody says something apparently nonsensical.

Yes, when you actually don't understand, saying that you don't understand is rarely a bad idea. It's when you understand but disagree that proclaiming an inability to comprehend the other's viewpoint is ill-advised.

Referring [to] an "individual's CEV".

I think this is a good idea, because humans aren't expected utility maximizers.

I could be wrong, but this may be a terminology issue.

Coherence: Strong agreement between many extrapolated individual volitions ...

want

AIs that want to kill humans (ie. most of them) are unfriendly.

Just as "want" does not unambiguously exclude instrumental values in English, "unfriendly" does not unambiguously include instrumental values in English. As for the composite technical term "Unfriendly Artificial Intelligence"...

If you write "Unfriendly Artificial Intelligence" alone, regardless of other context, you are technically correct. If you want to be correct again, type it again, in wingdings if the mood strikes you, you will still be technically correct, though with even less of a chance at communicating. In the context of entire papers, there is other supporting context, so it's not a problem. In the context of secondary discussions, consider those liable to be confused or you can consider them confused.

We might disagree about the extent of confusion around here, we might disagree as to how important that is, we might disagree as to how much of that is caused by unclear forum discussions, and we might disagree about the cost of various solutions.

Regarding the first point, those confident enough to post their thoughts on the issue make mistakes. Regarding the fourth point, assume I'm not advocating an inane extreme solution such as requiring you to define words every comment you make, but rather thoughtfulness.

Examples of cases where this is a problem include when people go around saying "a friendly AI may torture ". Because that is by definition not friendly. Any other example of "what if a friendly AI thing did " is also a misuse of the idea.

No torture? You're guessing as to what you want, what people want, what you value, what there is to know...etc. Guessing reasonably, but it's still just conjecture and not a necessary ingredient in the definition (as I gather it's usually used).

Or, you're using "friendly" in the colloquial rather than strictly technical sense, which is the opposite of how you criticized how I said not to speak about unfriendly AI! My main point is that care to should be taken to explain what is meant when navigating among differing conceptions within and between colloquial and technical senses.

Referring to AIs that want to kill humans as simply "unfriendly".

That seems entirely legitimate. ... AIs that want to kill humans (ie. most of them) are unfriendly.

I interpreted lessdazed as objecting to the lack of emotional impact - calling it "unfriendly artificial intelligence" has a certain blasé quality, particularly when compared to "all-powerful abomination bent on our destruction", say.

Edit: Apparently I was incorrect.

It's fairly similar to "it's in latin, therefore it must be profound"

Or to sound more profound, quidquid Latine dictum sit altum videtur. With that in mind, the fallacy of dressing up in mathematical clothing might be dubbed the Quidquid Mathematice fallacy, or Quidquid Per Numeros ("whatever (is said) with numbers").

Yes, that's what I was looking for! Thank you very much for the link.

I agree that it should all be possible with enhancement, but I'm not sure he was saying that. To your second point, I don't think dogs walk around with the knowledge that they're too stupid to comprehend the universe.

Humans with our enhancements haven't yet found any aspect of the universe which we have good reason to believe will always continue to escape explanation.

What would you say would actually constitute evidence for such a thing existing?

I haven't read it yet.

What would you think of a loose convention to not say one hasn't learned about a specific thing yet?

Saying that I haven't read something yet makes me more likely to think others think I am more likely to read it than if I hadn't said "yet". But that prematurely gives me some of the prestige that makes me want to read it in the first place, making it less likely I will.

Great, thanks. Very helpful. Where do I, er, post on the LW wiki? Should I click on the "discussion" page? (Again, apologies for my callowness!)

Hm. Yeah, that makes sense I suppose, though that's a rather alien way of thinking for me. I like the organisation and permanence of the Sequences, the fact that I can read them at my own pace and in what order they interest me. Especially since we don't seem to suffer from the "don't necro old threads" disease as much as most other forums, and arguments from 2007 still get developed today. Which is good IMO, I never liked how most of the Internet has a 1-2 day attention span, and if something isn't freshly posted then it's not worth reading or replying to. But I digress.

To each their own, I suppose, it's not like this stops me from reading the original Sequences the way I like to.

Thanks for the support, Tetronian!

I think you said it well. I am "aiming for a basic introduction targeted at non-LWers." This will be my first post on the main site, so I want write something important without it being overwhelming for myself and the reader. So I want to to be short, non-technical, and without LW-vernacular.

I've read a lot of Luke's work, including the post you linked. His work is fantastic and he's a very big inspiration to me. I think his work is what (in major part) helped led me to this idea.

Not exactly a great citation, but the Wikipedia article on the history of heat suggests that their understanding was not very good.

For what it is worth, marginally related ideas seemed to be going around at least by the end of the sixth century since I seem to recall an argument in the Talmud about whether or not transfer of heat by itself from a non-kosher thing to a kosher thing could make the kosher thing non-kosher. But it is possible that I was reading that with a too modern perspective. I'll try to track down the section and see what it says.

What if they built a building or found a cave where wind ran over a bucket or pool of water, cooling the air?

http://en.wikipedia.org/wiki/Evaporative_cooler#Physical_principles

http://en.wikipedia.org/wiki/Windcatcher

A number of substances have high enthalpy heats of solution, and appear to "create" cold when added to water. Some, like calcium chloride, would likely have been known in Classical Greece.

Edit: My mistake. Dissolution of calcium chloride is actually exothermic. I'm not sure if any salts which have high endothermic dissolution occur in a naturally pure state.

How do I know those processes are actually producing creating heat and not just destroying cold?

ETA: And to be clear, where by "I" I mean someone with roughly ancient Greek knowledge levels.

A description of an experiment they could have done would be a fine answer to my question, even though they weren't inclined to do them.

Perhaps the experiment is obvious to you? I don't know what it would be.

How would this belief pay rent

This is a good question and I'm not confident it could or couldn't. It's more a thought experiment to rethink how much medium-hanging fruit there is today. If they could have figured it out even though it couldn't have paid rent, so much the better.

The original theory of thermodynamics grew out of things like this, except the possibility that they considered had to do with a "caloric fluid". All objects had this particular fluid stored inside of them, and by rubbing the sticks together or something like that, you could simply release the fluid while gradually breaking down the object. Which is a reasonable conclusion, assuming you don't have all the historical background that we do in atomic physics.

Some people think that any value, if it is the only value, naturally tries to consume all available resources. Even if you explicitly make a satisficing, non-maximizing value (e.g. "make 1000 paperclips", not just "make paperclips"), a rational agent pursuing that value may consume infinite resources making more paperclips just in case it's somehow wrong about already having made 1000 of them, or in case some of the ones it has made are destroyed.

On this view, all values need to be able to trade off one another (which implies a common quantitative utility measurement). Even if it seems obvious that the chance you're wrong about having made 1000 paperclips is very small, and you shouldn't invest more resources in that instead of working on your next value, this needs to be explicit and quantified.

In this case, since all values inherently conflict with one another, all decisions (between actions that would serve different values) are moral decisions in your terms. I think this is a good intuition pump for why some people think all actions and all decisions are necessarily moral.

Were the Babyeaters immoral before meeting humans?

If not, what would you like to call the thing we actually care about?

This is correct; for every x, there is a largest number of complexity x.

I was surprised to find out about stotting being cooperation between cheetahs and gazelles. I was amused by the "rock-paper-scissors" common side blotched lizard.

In nature, is there a Rubinstein bargaining model or a game without a value, for example?

Well, the superiority of Tit-for-Tat to most other Iterated PD strategies was discovered by evolutionary sims, and evidence has been found of its being used in nature. For instance, the behavior of WWI soldiers who stopped killing each other in the trenches by mutually choosing to only fire their artillery when fired upon first, and several instances in animals. I'm too lazy to look up the latter, but I'm pretty confident they're in The Selfish Gene. I think lessdazed is asking if there are any other important game theory findings that don't have that kind of real world support.

Assuming a speed limit of communication such as light speed, why couldn't sufficiently large minds always either be made from less matter or merely be larger versions of smaller, identical patterns?

You are in a Chinese room, according to his argument. No one of us is as cruel as all of us.

This for one http://singinst.org/upload/LOGI//LOGI.pdf is in the ballpark of AGI work. Plus FAI work, while not being on AGI per se, is relevant and interesting to a rare conference in the area. Waser is pure drivel.

They...build a high wall around themselves rather than building roads to their neighbors. I can understand self-protection and short-sighted conservatism but extremes aren’t healthy for anyone...repetitively screaming their fear rather than listening to rational advice. Worse, they’re kicking rocks down on us.

If it weren’t for their fear-mongering...AND their arguing for unwise, dangerous actions (because they can’t see the even larger dangers that they are causing), I would ignore them like harmless individuals...rather than [like] junkies who need to do anti-societal/immoral things to support their habits...fear-mongering and manipulating others...

...very good at rhetorical rationalization and who are selfishly, unwilling to honestly interact and cooperate with others. Their fearful, conservative selfishness extends far beyond their “necessary” enslavement of the non-human and dangerous...raising strawmen, reducing to sound bites and other misdirections. They dismiss anyone and anything they don’t like with pejoratives like clueless and confused. Rather than honest engagement they attempt to shut down anyone who doesn’t see the world as they do. And they are very active in trying to proselytize their bad ideas...

In a sense, they are very like out-of-control children. They are bright, well-meaning and without a clue of the likely results of their actions. You certainly can’t hate individuals like that — but you also don’t let them run rampant...

What do you mean no arguments? Just read the above excerpts...what do you think those are, ad hominems and applause lights?

Based on the abstract, it's not worth my time to read it.

Abstract. Insanity is doing the same thing over and over and expecting a different result. “Friendly AI” (FAI) meets these criteria on four separate counts by expecting a good result after: 1) it not only puts all of humanity’s eggs into one basket but relies upon a totally new and untested basket, 2) it allows fear to dictate our lives, 3) it divides the universe into us vs. them, and finally 4) it rejects the value of diversity. In addition, FAI goal initialization relies on being able to correctly calculate a “Coherent Extrapolated Volition of Humanity” (CEV) via some as-yet-undiscovered algorithm. Rational Universal Benevolence (RUB) is based upon established game theory and evolutionary ethics and is simple, safe, stable, self-correcting, and sensitive to current human thinking, intuitions, and feelings. Which strategy would you prefer to rest the fate of humanity upon?

Points 2), 3), and 4) are simply inane.

Maybe he submits papers and conference program comittee find them relevant and interesting enough?

Which invites the question of why clearly incompetent people make up the program committee. His papers look like utter drivel mixed with superstition.

Which paper of his did you read? He has quite a few.

Yep, definitely needs some clarification there.

Humans don't distinguish between the utility for different microscopic states of the world. Nobody cares if air molecule 12445 is shifted 3 microns to the right, since that doesn't have any noticable effects on our experiences. As such, a state (at least for the purposes of that definition of utility) is a macroscopic state.

"~X" means, as in logic, "not X". Since we're interested in the negative utility of the floor being clear, in the above case X is "the airplane's floor being clear" and ~X is "the airplane's floor being opaque but otherwise identical to a human observer".

In reality, you probably aren't going to get a material that is exactly the same structurally as the clear floor, but that shouldn't stop you from applying the idea in principle. After all, you could probably get reasonably close by spray painting the floor.

To steal from Hofstadter, we're interested in the positive utility derived from whatever substrate level changes would result in an inversion of our mind's symbol level understanding of the property or object in question.

I think you need to be more precise about what states and ~ are.

In that formulation, addition of the thing has utility equal to minus the utility of removal of the same thing. And that only if addition/removal can be defined.

About states - there are too many of them, some are macroscopically different but irrelevant to human untility (I am next to sure it is possible to shift some distant galaxies in a way that many sapient being will be able to see different sky and none would care).

The meaningful thing is ratio of utility differentials between some states.

that thing's removal

How is this defined? If an airplane has a clear floor such that its passengers vomit whenever they look down, removing the floor would put them in an even worse position. We want to remove only the property of transparency, which would involve replacing the clear material with an entirely different opaque material that had all other properties identical.

What's troubling to me about the counterfactual is that it doesn't seem to have an objective baseline, a single thing that is ~X, so we are left comparing Y(X) with Y(Z), the utility of thing X instead of thing Z. I'm not sure how valid it is to talk about simply removing properties because the set of higher level properties depends on the arrangement of atoms. It seems like properties are their own thing that can be individually mixed and matched separate from material but they really can't be.

Why not just disallow the casting of more than one downvote by the same person within a five-minute period? How many spite-voters are going to be dedicated enough to wait around for an hour just to blitz someone's karma?

Yeah, I wouldn't have proposed hard limits, I was thinking more of an automatic (i.e. not involving manually poking around in the database) means of allowing the administrators to check on large-scale suspicious voting and reverse it if necessary. (And, as I said, I'm by no means worried about my 16 precious votes (though I'd be starting to get concerned by 160), but this incident reminded me of the general problem and I wanted to check if I had missed any changes to how such things are handled.)

I might support just making all votes public; since on LW they (are supposed to) mean "more/less of this" rather than "I like/don't like you" or "I agree/disagree", I'm not sure I see any reason why that information should not be associated with the people whose opinions they represent, since that is relevant information as well. (Though of course some people prefer going in the other direction to make things consistent, hence the anti-kibitzer. But if the anti-kibitzer can be opt-in, perhaps so should not seeing other people's votes.)

But then, I vaguely remember that having been discussed before, so I'll see if I can locate said discussion(s) before attempting to start another one.

EDIT: Re-framing campaign successful. I no longer endorse the new meaning which has been attributed to this comment.

Well, the probability is computed by an algorithm that is itself imperfect. "I'm 50% confident that the probability is 1/1000" means something like "My computation gives a probably of 1/1000, but I'm only 50% confident that I did it right". For example, if given a complex maths problem about probabilities of getting some card patterns from a deck with twisted rules of drawing and shuffling, you can do that maths, ends up with a probability of 1/5 that you'll get the pattern, but not be confident you didn't make a mistake in applying the laws of probability, so you'll only give a 50% confidence to that answer.

And there is also the difference between belief and belief in belief. I can something "I believe the probability to be of 1/1000, but I'm only 50% confident that this is my real belief, and not just a belief in belief".

It sounds to me like he is describing his distribution over probabilities, and estimates at least 50% of the mass of his distribution is between 1/1,000 and 1/1,000,000. Is that a convenient way to store or deal with probabilities? Not really, no, but I can see why someone would pick it.

If you can have a 95% confidence interval, why can't you have a >50% confidence interval as well?

I basically agree that the part of the original comment that you quote doesn't make any sense at all, and am not attempting to come to the defence of confidence intervals over probabilities, but it does feel like there should be some way of giving statements of probability and indicating how sure one is about the statement at the same time. I think, in some sense, I want to be able to say how likely I think it is that I will get new information that will cause me to update away from my current estimate, or give a second-derivative of my uncertainty, if you will.

Let's say we have two bags, one contains 1 million normal coins, one contains 500,000 2-headed coins and 500,000 2-tailed coins. Now, I draw a coin from the first bag and toss it - I have a 50% chance of getting a head. I draw a coin from the second bag and toss it - I also have a 50% chance of getting a head, but it does feel like there's some meaningful difference between the two situations. I will admit, though, that I have basically no idea how to formalise this - I assume somebody, somewhere, does.

I agree. Perhaps he means to say that his opinion is based on very little evidence and is "just a hunch".

I do think that in fitting a model to data, you can give meaningful confidence intervals for parameters of those models which correspond to probabilities (e.g. p(heads) for a particular coin flipping device). But that's not relevant here.

He's arguing that it's dangerous to Pascal's Mug people. This surely affects your probability that the guy who's pulling a Pascal's mugging on you is really faking.