LessWrong 2.0 Reader

the public seems pretty bought-in on AI risk being a real issue and is interested in regulation.

Some people are more concerned about S-risk than extinction risk, and I certainly don't want to dismiss them or imply that their concerns are mistaken or invalid, but I just find it a lot less likely that the AI project will lead to massive human suffering than its leading to human extinction.

There's a huge gulf between people's expressing concern about AI to pollsters and the kind of regulations and shutdowns that would actually avert extinction. The people (including the "safety" people) whose careers would be set back by many years if they had to find employment outside the AI field and the people who've invested a few hundred billion into AI are a powerful lobbying group in direction opposition to the members of the general public who tell pollsters they are concerned.

I don't actually know enough about the authoritarian countries (e.g., Russia, China, Iran) to predict with any confidence how likely they are to prevent their populations from contributing to human extinction through AI. I can't help but notice though that so far it is the US and the UK that have done the most so far to advance the AI project. Also, the government's deciding to shut down movements and technological trends is much more normalized and accepted in Russia, China and Iran than it is in the West, particularly the US.

I don't have any prescriptions really. I just think that the OP (titled "why the 2024 election matters, the AI risk case for Harris, & what you can do to help", currently standing at 23 points) is badly thought out and badly reasoned, and I wish I had called for readers to downvote it because its main effect on LW was probably to add some politics-mindkill without adding any useful insight.

So it doesn't make much sense to value emotions

I think this is a non sequitur. Everything you value can be described as just <dismissive reductionist description>, so the fact that emotions can too isn't a good argument against valuing them. And in this case, the dismissive reductionist explanation misses a crucial property: emotions are accompanied by (or identical with, depending on definitions) valenced qualia.

Emotions are hardwired stereotyped syndromes of hardwired blunt-force cognitive actions. E.g. fear makes your heart beat faster and puts an expression on your face and makes you consider negative outcomes more and maybe makes you pay attention to your surroundings. So it doesn't make much sense to value emotions, but emotions are good ways of telling that you value something; e.g. if you feel fear in response to X, probably X causes something you don't want, or if you feel happy when / after doing Y, probably Y causes / involves something you want.

That's a different question than the one I meant. Let me clarify:

Basically I was asking you what you think the probability is that Trump would win the election (as of a week before the election, since I think that matters) now that you know how the election turned out.

An analogous question would be the following:

Suppose I have two unfair coins. One coin is biased to land on heads 90% of the time (call it H-coin) and the other is biased to land on tails 90% of the times (T-coin). These two coins look the same to you on the outside. I choose one of the coins, then ask you how likely it is that the coin I chose will land on heads. You don't know whether the coin I'm holding is H-coin or T-coin, so you answer 50% (50%=0.5*.90=+0.5*0.10). I then flip the coin and it lands on heads. Now I ask you, knowing that the coin landed on heads, now how likely do you think it was that it would land on heads when I first tossed it? (I mean the same question by "Knowing how the election turned out, how likely do you think it was a week before the election that Trump would win?").

(Spoilers: I'd be interested in knowing your answer to this question before you read my comment on your "The value of a vote in the 2024 presidential election" EA Forum post that you linked to [EA(p) · GW(p)] to avoid getting biased by my answer/thoughts.)

1. Kamala Harris did run a bad campaign. She was 'super popular' at the start of the campaign (assuming you can trust the polls, though you mostly can't), and 'super unpopular' losing definitively at the end of it. On September 17th, she was ahead by 2 points in polls, and in a little more than a month and a half she was down by that much in the vote. She lost so much ground. She had no good ads, no good policy positions, and was completely unconvincing to people who weren't guaranteed to vote for her from the start. She had tons of money to get out all of this, but it was all wasted.

The fact that other incumbent parties did badly is not in fact proof that she was simply doomed, because there were so many people willing to give her a chance. It was her choice to run as the candidate who 'couldn't think of a single thing' (not sure of exact quote) that she would do differently than Biden. Not a single thing!

Also, voters already punished Trump for Covid related stuff and blamed him. She was running against a person who was the Covid incumbent! And she couldn't think of a single way to take advantage of that. No one believed her that inflation was Trump's fault because she didn't even make a real case for it. It was a bad campaign.

Not taking policy positions is not a good campaign when you are mostly known for bad ones. She didn't run away very well from her unpopular positions from the past despite trying to be seen as moderate now.

I think the map you used is highly misleading. Just because there are some states that swung even more against her, doesn't mean she did well in the others. You can say that losing so many supporters in clearly left states like California doesn't matter, and neither does losing so many supporters in clearly right states like Texas, but thinking both that it doesn't matter in terms of it being a negative, and that it does matter enough that you should 'correct' the data by it is obviously bad.

2.Some polls were bad, some were not. Ho hum. But that Iowa poll was really something else. (I don't have a particular opinion on why she screwed up, aside from the fact that no one wants to be that far off if they have any pride.) She should have separately told people she thought the poll was wrong if she thought it was, did she do that? (I genuinely don't know.) I do think you should ignore her if she doesn't fix her methodology to account for nonresponse bias, because very few people actually answer polls. An intereting way might be to run a poll that just asks something like 'are you male or female?' or 'are you a democrat of Republican?' and so on so you can figure out those variables for the given election on both separate polls and on the 'who are you voting for' polls. If those numbers don't match, something is weird about the polls.

I think it is important to note that people thought the polls would be closer this time by a lot than before (because otherwise everyone would have predicted a landslide due to them being close.) You said, "Some people went into the 2024 election fearing that pollsters had not adequately corrected for the sources of bias that had plagued them in 2016 and 2020." but I mostly heard the opposite from those who weren't staunch supporters of Trump. I think the idea of how corrections had gone before we got the results was mostly partisan. Many people were sure they had been fully fixed (or overcorrected) for bias and this was not true, so people act like they are clearly off (which they were). Most people genuinely thought this was a much closer race than it turned out to be.

The margin of being off was smaller than in the past trump elections, I'll agree, but I think it is mostly the bias people are keying on rather than the absolute error. The polls have been heavily biased on average for the past three presidential cycles, and this time was still clearly biased (even if less so). With absolute error but no bias, you can just take more or larger polls, but with bias, especially an unknowable amount of bias, it is very hard to just improve things. Also, the 'moderate' bias is still larger than 2000, 2004, 2008, and 2012.

My personal theory is that the polls are mostly biased against Trump personally because it is more difficult to get good numbers on him due to interacting strangely with the electorate as compared to previous Republicans (perhaps because he isn't really a member of the same party they were), but obviously we don't actually know why. If the Trump realignment sticks around, perhaps they'll do better correcting for it later.

I do think part of the bias is the pollsters reacting to uncertainty about how to correct for things by going with the results they prefer, but I don't personally think that is the main issue here.

3.Your claim that 'Theo' was just lucky because neighbor polls are nonsense doesn't seem accurate. For one thing, neighbor polls aren't nonsense. They actually give you a lot more information than 'who are you voting for'. (Though they are speculative.) You can easily correct for how many neighbors someone has too and where they live using data on where people live, and you can also just ask 'what percentage of your neighbors are likely to vote for' to correct for the fact that it is different percentages of support.

As a separate point, a lot of people think the validity of neighbor polls comes from people believing that the respondents are largely revealing their own personal vote, though I have some issues with that explanation.

So, one bad poll with an extreme definition of 'neighbor' negates neighbor voting and many bad polls don't negate traditional? Also, Theo already had access to the normal polls as did everyone else. Even if a neighbor poll for some reason exaggerates the difference, as long as it is in the right direction, it is still evidence of what direction the polls are wrong in.

Keep in mind that the chance of Trump winning was much higher than traditional polls said. Just because Theo won with his bets doesn't mean you should believe he'd be right again, but claiming that it is 'just lucky' is a bad idea epistemologically, because you don't know what information he had that you don't.

4.I agree, we don't know whether or not the campaigns spent money wisely. The strengths and weaknesses of the candidates seemed to not rely much on the amount of money they spent, which likely does indicate they were somewhat wasteful on both sides, but it is hard to tell.

5.Is Trump a good candidate or a bad one? In some ways both. He is very charismatic in the sense of making everyone pay attention to him, which motivates both his potential supporters and potential foes to both become actual supporters and foes respectively. He also acts in ways his opponents find hard to counter, but turn off a significant number of people. An election with Trump in it is an election about Trump, whether that is good or bad for his chances.

I think it would be fairer to say Trump got unlucky with election that he lost than that he was lucky to win this one. Trump was the covid incumbent who got kicked out because of it despite having an otherwise successful first term.

We don't usually call a bad opponent luck in this manner. Harris was a quasi-incumbent from a badly performing administration who was herself a laughingstock for most of the term. She was partially chosen as a reaction to Trump! (So he made his own luck! if this is luck.)

His opponent in 2016 was obviously a bad candidate too, but again, that isn't so much 'luck'. Look closely at the graph for Clinton. Her unfavorability went way up when Trump ran against her. This is also a good example of a candidate making their own 'luck'. He was effective in his campaign to make people dislike her more.

6.Yeah, money isn't the biggest deal, but it probably did help Kamala. She isn't any good at drawing attention just by existing like Trump, so she really needed it. Most people aren't always the center of attention, so money almost always does matter to an extent.

7.I agree that your opinion of Americans shouldn't really change much by being a few points different than expected in a vote either way, especially since each individual person making the judgement is almost 50% likely to be wrong anyway! If the candidates weren't identically as good, at least as many as the lower of the two were 'wrong' (if you assume one correct choice regardless of person reasons) and it could easily be everyone who didn't vote for the lower. If they were identically as good, then it can't be that voting for one of them over the other should matter to your opinion of them. I have an opinion on which candidate was 'wrong' of course, but it doesn't really matter to the point (though I am freely willing to admit that it is the opposite of yours).

That makes sense, thanks.

Some people went into the 2024 election fearing that pollsters had not adequately corrected for the sources of bias that had plagued them in 2016 and 2020.

I mostly heard the opposite, that they had overcorrected.

This seems pretty cool! The data augmentation technique proposed seems simple and effective. I'd be interested to see a scaled-up version of this (more harmful instructions, models etc). Also would be cool to see some interpretability studies to understand how the internal mechanisms change from 'deep' alignment (and compare this to previous work, such as https://arxiv.org/abs/2311.12786, https://arxiv.org/abs/2401.01967) 

As it often does when I write, this ended up being pretty long (and not especially well written by the standards I wish I lived up to).

I'm sure I did misunderstand part of what you are saying (that we do misunderstand easily was the biggest part of what we appear to agree on), but also, my disagreements aren't necessarily things you don't actually mention yourself. I think we disagree mostly on what outcomes the advice itself will give if adopted overly eagerly, because I see the bad way of implementing them as being the natural outcome. Again, I think your 8th point is basically the thrust of my criticism. There is no script you can actually follow to truly understand people, because people are not scripted.

Note: I like to think I am very smart and good at understanding, but in reality I think I am in some ways especially likely to misunderstand and to be misunderstood. (Possible reason: Maybe I think strangely as compared to other people?) You can't necessarily expect me to come at things from a similar angle as other people, and since advice is usually intended as implicitly altering the current state of things, I don't necessarily have a handle on that.

Importantly, since they were notes, I took them linearly, and didn't necessarily notice if my points were addressed sufficiently later.

Also, I view disagreements as part of searching for truth, not for trying to convince people you are right. Some of my distaste is that it feels like the advice is being given for persuasion more than truthseeking? (Honestly, persuasion feels a little dirty to me, though I try to ignore that since I believe there isn't actually anything inherently wrong with persuasion, and in many cases it is actually good.) Perhaps my writing would be better / make more sense if I was more interested in persuading people?

An important note on my motives for the comment is that I went through with posting it when I think I didn't do particularly well (there were obvious problems) in part to see how you would respond to it. I don't generally think my commenting actually helps so I mostly don't, but I've been trying out doing it more lately. There are also plenty of problems with this response I am making.

Perhaps it would have been useful for me to refer to what I was writing about by number more often.

Some of the points do themselves seem a bit disrespectful to me as well. (Later note: You actually mention changing this later and the new version on Karma is fine.) Like your suggestion for how to change the mind of religious people (though I don't actually remember what I found disrespectful about it at this moment). (I am not personally religious, but I find that people often try to act in these spaces like religious people are automatically wrong which grates on me.)

Watching someone else having a conversation is obviously very slow, but there is actually a lot of information in any given conversation.

Random take: The first video is about Karma, which I do have an opinion on. I believe that the traditional explanation of Karma is highly unlikely, but Karma exists if you think of it as "You have to live with who you are. If you suck, living with yourself sucks. If you're really good, living with yourself is pretty great." See also, "If you are in hell, it's a you thing. If you are in heaven, it's also a you thing." There are some things extreme enough where that isn't really true, like when being actively tortured, but your mind is what determines how your life goes even more than what events actually happen in the normal case, and it does still effect how you react to the worst (and best) things. (People sometimes use the story about a traveler asking someone what the upcoming town is like, and the person just asking the traveler what people in the previous place were like, while answering 'much the same' for multiple travelers with different outlooks and I do think this is somewhat true.)

Also, doing bad things can often lead to direct or indirect retaliation, and good to direct or indirect reward. Indirect could definitely 'feel' like Karma.

I think that the actual key to a successful conversation is to keep in mind what the person you are talking to actually wants from the conversation, and I would guess what people mostly want from a random conversation is for the other person to think they are important (whenever they don't have an explicit personal goal from the conversation). I pretty much always want to get at the truth as my personal goal because I'm obsessive that way, but I usually have that goal as an attempt at being helpful.

It seems to work for him getting his way, and nothing he does is too bad, but the conversational tactics seem a bit off to me. (Only a bit.) It seems like he is pushing his own ideas too much on someone else who is not ready for the conversation (though they are happy enough to talk).

No, I don't know any way to make sure your conversation partner is ready for the conversation. A lot of evidence for your position is not available in a synchronous thing like a conversation, and I believe that any persuasion should attempt to be through giving them things they can think through later when not under time pressure. He didn't exactly not do that, but he also didn't do that. "You must decide now" (before the end of the conversation) seemed to be a bit of an undercurrent to things. (A classic salesman tactic, but I don't like it. And sure, the salesman will pivot toward being willing to talk to you again later if you don't bite on that most of the time, but that doesn't mean they weren't pressuring you to decide quickly.)

The comparison between 'Karma' and 'Santa' seems highly disrespectful and unhelpful. They are very different things and the analogy seems designed to make things unclear more than clearing them up. In other words, I think it is meant to confuse the issue rather than give genuine insight. You could object that part of the Santa story is literally Karma (the naughty list) but I don't think that makes the analogy work.

I don't really get the impression he was actually willing to be convinced himself. He said at one point that he was willing to, and maybe in the abstract he is, but he never seemed to seek information against his own position. Note that I don't think I would necessarily be able to tell, and I actually disapprove of 'mindreading' quite strongly.

The fact that I am strongly against 'mindreading' and actually resort to it myself automatically is actually one of the points I was trying to make about how easy it is to misuse conversational tactics. I was genuinely trying to understand what he was doing, (in service of making a response based on it) and I automatically ended up thinking he was doing the opposite of what he claimed, just based on vibes without any real evidence.

You could argue I am so against it because I notice myself doing it, and maybe it is true, but I find it infuriating when others do it badly. I don't actually have any issues with them guessing what I'm doing correctly, though I'm unlikely to always be right about that either (just more than other people about me).

He also didn't seem entirely open that he was pushing for a specific position throughout the entire conversation, when he definitely was. This wasn't a case of just helping someone update on the information they have (though there was genuinely a large amount of that too.) (People do need help updating and it can be a valuable service, but for it to really be helpful, it needs to not be skewed itself.)

The second video (about convincing someone to support trans stuff) seems pretty strange. This video seems completely different from the previous one; more propaganda than anything. Clearly an activist (and I generally dislike activists of any stripe.). (Emotional response: Activists are icky.) Also an obviously hot culture war issue (which I have an opinion on though I don't think said opinion is relevant to this discussion). It's also very edited down which makes it feel even more different.

The main tactic seems like trying to force a person to have an emotional reaction through manipulative personal stories (though he claims otherwise and there are other explanations). But he seemed to do it over and over again, so this time I am pretty sure he isn't being entirely honest about that (even though I still disapprove of mindreading like I am doing here.). I feel like he is a bad person (though not unusually so for an activist.)

The alternate explanation, which does work, is just that people like to tell stories about themselves when talking about any subject. I clearly reference myself many times in this response and my original response. I'm not saying I'm being fair in my conclusions.

Do you really see those two videos as similar? While there are some similarities, they feel quite different to me! I didn't love it, but the first video was about talking through the other person's points and having a genuine conversation. The latter was about taking advantage of their conversation partner's points for the next emotional reaction. In other words, the latter video felt a lot more like tricking someone while the former was a conversation.

Moving past the videos to the rest of the response.
Yes, the switch to the longer way of rephrasing that includes explicitly accepting that you might be wrong seems much, much better. Obviously, it is best for the person to really believe they might be wrong, and saying it both helps an honest participant remember that, and should make it easier for the person they are talking with to correct them. Saying the words isn't enough, but I like it a lot better than before.

Obviously, I'm not rephrasing your points because that still isn't how I believe it should be done, but if there is a key point this way of asking about it can be very useful. Or, to rephrase, rephrasing is a tool to be used on points that seem to be particularly important to check your understanding of.

I don't remember exactly what you said in point 4 before you changed it, but I don't particularly read point 5 as being anti personal experience in the way my comment indicates. I have no idea why I would possibly write that about point 5 so I assume you are correct in your assumption.

Since I only vaguely remember it, my memory only contains the conclusion I came to which we both agree can be faulty. But the way I remember it, the old point 4 is very clearly a direct attack on personal experience in general rather than on distinguishing between faulty and reliable personal experience. From past experience, this could be attributable to many things, including just not reading a few words along the way.

I don't really have any issues with your new point 4 (and it is clearly taken from that first video.) That is very obviously a good approach for convincing people of things that doesn't rely on anything I find distasteful. It seems very clearly like what you are saying you are going for and I think it works very well.

For the record, I think 'working definition' is no more different from 'mathematical definition' than 'theoretical definition' is from 'mathematical definition' because I am using 'theoretical definition' in a colloquial way. I was definitely not saying mathematical definitions or formal definitions are useful when talking to a layperson. (Side note: I've been paying attention to 'rationalists' of this sort for about 20 years now, but I am not one. I tend to resist becoming part of groups.) I do generally think that unless you are in the field itself that 'formal definitions' are not helpful since they take far too much time and effort that could be used on other things (and formal definitions are often harder to understand even afterward in many cases), and mathematical definitions are unnatural for most people even after they understand them.

I do not want people spending more time on definitions in conversation unless it is directly relevant, but think remembering that there are different kinds of brief definitions seems important to me.

I perhaps overreacted to the mention of Bayes Rule. It's valid enough for describing things in probalistic circumstances, but people in this community try to stretch it to things that aren't probability theory related and it's become a bit of a pet peeve for me. I have never once used Bayes Rule to update my own beliefs (unless you include using it formally to decide what answer to give on a probability question a few times in school), but people act like that is how it is done.

In the paper on 'Erotectic' reasoning, ... includes a pretty weird bit of jargon on their very first example (first full paragraph of second page) which makes it hard to understand their point. And not only do they not explain, it isn't even something I could look up with a web search because all explanations are paywalled seemingly. They claim it is a well-known example, but it clearly isn't.

As best as I can tell, the example is really just them abusing language. Because 'or else' is closer to 'exclusive or' but they are pretending it is just 'or'. (It is a clear misuse to pretend it doesn't.) I don't know philosophy jargon well, but misstating the premise is not clever. In this case, every word of the premise mattered, and they intentionally picked incorrect ones. Their poor writing wasted a great deal of time. And yes, I am actually upset at them for it. I kept looping back around to being upset about their actions and wanting to yell at them rather than considering what they were writing about. (Which is an important point I suppose, if the person you are conversing with is upset with you, things are reasonably likely to go badly regardless of whether your points are good or bad.) I think it is the most upset I've been reading a formal paper (though I mostly have only really read a small number of AI and/or Math ones.)

In the end I could tell I wasn't going to stop if I kept reading, so I quit without understanding what they were writing about. (I can definitely be a bit overboard sometimes.) All I got was that they think there is some way to ask questions that works with the basic reasoning people normally use and leads to deductively valid reasoning. I have no idea what method of questioning they are in favor of or why they think it works. (I do think the example could have been trivially changed and not bothered me.) I do think my emotional reaction is a prime example of a reason resolving disagreements often doesn't work (and even why 'fake disagreements' where the parties actually agree on the matter can fail to be resolved).

To really stress point 8 it should be point 1. I was just saying it needed to be stressed more. I did notice you saying it was important and I was agreeing with you for the most part. Generally you evaluate points based on what came before, not based on what came after (though it does happen). It's funny, people often believe they are disagreeing when they are just focusing on things they actually agree on in a different manner.

On a related note, it's not like I'm ordering this stuff in order of how important I think it is. Sometimes things fit better in a different order than importance (this is obviously in order of what I am responding to.) (Also, revising this response on a global scale would take far too long given how long it already takes me to write comments. It might be worth writing shorter but better in the same amount of time, but I don't seem inclined to it.)

You know what, since I wrote that I had a lot of disagreements, I really should have pointed out that not all of the things I was writing were disagreements! I think my writing often comes off as more negative than I mean it (and not because other people are reading it badly).

On the note of it being a minimum viable product, I think those are very easy to badly. You aim for what you personally think is the minimum... when you already know the other stuff you are trying to say. It is then often missing many things that are actually necessary for it to work, which makes it just wrong. I get the idea, perfectionism is slow, a waste of resources, and even impossible, but aiming for the actual minimum is a bad idea. It is often useful advice for startups, but we do not want to accept the same rate of failure as a startup business! Virtually all of them fail. We should aim more for the golden mean in something like a formal post like you made. (A high rate of failure in comments/feedback seems fine though since even mostly failed comments can spark something useful.)

As far as quoting the first sentence of each thing I am responding to, that does sound like a useful idea, and I should do it, but I don't think I am going to anyway. For some reason I dread doing it (especially at this point). I also don't even know how to make a quote on lesswrong, much less a partial one. I know I don't necessarily signpost well exactly what I am responding to. (Plus, I actually write this in plaintext in notepad rather than the comment area. I am paranoid about losing comments written on a web interface since it takes me so long to write them.)

Thanks!  These are thoughtful points. See some clarifications below:
 

AGI could be very catastrophic even when it stops existing a year later.

You're right. I'm not even covering all the other bad stuff that could happen in the short-term, that we might still be able to prevent, like AGI triggering global nuclear war.

What I'm referring to is unpreventable convergence on extinction.
 

If AGI makes earth uninhabitable in a trillion years, that could be a good outcome nonetheless.

Agreed that could be a good outcome if it could be attainable.

In practice, the convergence reasoning is about total human extinction happening within 500 years after 'AGI' has been introduced into the environment (with very very little probability remainder above that).

In theory of course, to converge toward 100% chance, you are reasoning about going across a timeline of potentially infinite span.
 

I don't know whether that covers "humans can survive on mars with a space-suit",

Yes, it does cover that. Whatever technological means we could think of shielding ourselves, or 'AGI' could come up with to create as (temporary) barriers against the human-toxic landscape it creates, still would not be enough.
 

if humans evolve/change to handle situations that they currently do not survive under

Unfortunately, this is not workable. The mismatch between the (expanding) set of conditions needed for maintaining/increasing configurations of the AGI artificial hardware and for our human organic wetware is too great. 

Also, if you try entirely changing our underlying substrate to the artificial substrate, you've basically removed the human and are left with 'AGI'. The lossy scans of human brains ported onto hardware would no longer feel as 'humans' can feel, and will be further changed/selected for to fit with their artificial substrate. This is because what humans and feel and express as emotions is grounded in the distributed and locally context-dependent functioning of organic molecules (eg. hormones) in our body.