Posts
Comments
I wish you would have just directly made this post about this specific thing that happened rather than try to generalize from one example. Or found more examples to show a pattern we could engage with.
There are so, so many examples of things like this that happen all the time in the world. I used two hypothetical examples in the post. I thought that would suffice.
It's also false; there were lots of replies.
There were comments on Facebook, to be sure, but I never saw anyone (except me) reply to my comment here on LessWrong, ever after (what felt like) several days.
For anyone curious, you can view the original comment here.
You might be surprised!
You might want to add (1.5) also evaluate whether what's going on is that some group of people wants to be referred to differently, and then (2') generally don't resist in that case even if no harm is apparent, because (a) maybe there's harm you haven't noticed and (b) giving people what they want is usually good. I'd certainly be on board with that. (I suspect Scott would too.)
I think this is pretty much my argument. I think Scott wouldn't agree because he wrote:
On the other hand, the people who want to be the first person in a new cascade, like USC’s social work department, are contemptible. And the people who join when it’s only reached 1% or 5%, out of enthusiastic conformity or pre-emptive fear, are pathetic.
(none of this applies to things being done for good reasons - banning actually harmful things - I’m just skeptical that this process gets used for that very often)
The roots of "Black" go back further than 1966. For example, here are two excerpts from Martin Luther King's "I Have a Dream" speech in 1963 (emphasis mine):
When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all men — yes, Black men as well as white men — would be guaranteed the unalienable rights of life, liberty and the pursuit of happiness.
And when this happens, and when we allow freedom ring, when we let it ring from every village and every hamlet, from every state and every city, we will be able to speed up that day when all of God's children, Black men and white men, Jews and Gentiles, Protestants and Catholics, will be able to join hands and sing in the words of the old Negro spiritual: Free at last. Free at last. Thank God almighty, we are free at last.
The relevant question, if you ask me, is which word was used racistly less often. My guess is "Black". That's what I mean when I say words become taboo because they are used offensively; they become associated with, e.g., racism.
"Black" also had the virtue of being a label of endogenous origin. Per point (4) in the OP.
Are these two reasons (being used racistly less often and endogenous origin) good enough reasons to change the word we use for this group of people? Yes, I argue. The benefits greatly outweigh the costs.
Per point (1) in the OP, the costs of changing are low. Per point (2), the cost of changing the words we use is a cost we willingly incur all the time, for much less good reasons. Per point (3), it's not a cost we'll have to incur very often, as endogenous group labels seem to have incredible longevity.
I hope you find this argument addresses the relevant points and doesn't skirt any important issues. I tried my best to address the crux of the matter head on while being as concise as possible.
I think this is the crux of the matter:
Was the process in this case a bad thing overall, as we should probably expect on Scott's model? (Bad: risk of mis-classifying people as racist whose only sin was not to adjust their language quickly enough; inconvenience during the transition; awkwardness after the transition of reading material written before it. Good: morale-boosting effects on black people of feeling that they were using a term of their own choosing and taking more control of their own destiny; if SC/KT was correct about "negro" bringing along unwanted associations etc., then some degree of escape from those associations.)
My contention is that changing the words we use for minority groups is not a bad thing overall because the costs are low and the benefits are high. This is what the OP attempted to establish with points (1) through (4).
(I elaborated more in a separate comment.)
I commented on Scott’s blog post with a link to this post.
The post makes the claim hyperstitious cascades are bad, where previously innocent words that noone took offense to become taboo
A major claim I’m making is that this has never actually happened in history, and certainly not in any of the examples Scott uses. Words become taboo because they are used offensively.
Out of the two options, this is closer to my view:
"this is completely and horribly incorrect in approach and model"
I think Scott’s model of how changes in the words we use for minority groups happen is just factually inaccurate and unrealistic. Changes are generally slow, gradual, long-lasting, and are primarily advocated for in good faith by conscientious members of the minority group in question.
My contention is that this model of the process is basically just wrong for the examples of minority group labels that have actually caught on.
There's a difference between "labels should never change" (which you say Scott is saying, but he isn't) and "the thing where a previously harmless label becomes offensive, not because it was really offensive all along, but because someone has decided to try to make it offensive and then there's a positive feedback loop, generally does more harm than good" (which he is saying).
Per (4) in the OP, I think this process that Scott describes is simply an incorrect model of why some words for minority groups come to be seen as derogatory and why the acceptable words change. His account of how the label "Black" was popularized, for example, just seems factually incorrect from what I can glean from some cursory reading online,
Potential solutions to foreseeable problems with biological superintelligence include: a) only upgrading particularly moral and trustworthy humans or b) ensuring that upgrading is widely accessible, so that lots of people can do it.
I guess we could say governance remains a problem with biological superintelligence? As it does with normal humans, just more so.
Beautifully written! Great job! I really enjoyed reading this story.
in comparison to a morally purified version of SimplexAI,we might be the baddies."
Did you link to the wrong thing here or is there some reference to generative grammar I'm not getting?
Most of the writing on ethics in our society and its cultural forebears over the last few millennia has been written by moral absolutists: people who believed that there is one, true, and correct set of ethics, and were either trying to figure out what it is, or more often thought they already have, and are now trying to persuade others.
This is not my understanding of moral absolutism. One definition from a University of Texas blog:
Moral absolutism asserts that there are certain universal moral principles by which all peoples’ actions may be judged. It is a form of deontology.
Another, similar definition from New World Encyclopedia:
Moral absolutism in this second sense is often held as opposed to consequentialism. ... By contrast, moral absolutism holds that some actions are absolutely wrong; they could never be right no matter what consequences of failing to do them might be. So, an absolutist would say that it is morally wrong to torture a child in order to save an entire nation. Absolutism says that some actions are wrong whatever the consequences. Or again, moral absolutism about lying would say that the lying is always wrong, whatever the consequences.
It seems too long for a comment. Also, it uses markdown formatting.
Again that's not Scotts point. Scott is concerned about deliberate attempts to rapidly make a perfectly innocent word taboo, causing bother and potential ostracism to everyone for no reason, not natural long term evolution of words.
I don't think such an attempt has ever happened and succeeded. I'm open to counterexamples, though.
The problem isn't that most of the black people in the USA got together and said they prefer to be called black. It's that due to a single bad actor making up a fake history for an innocent word, lots of old grandpas get ostracised by their grandkids for being racist.
I think Scott's account of the history of the term "Black" is dubious.
If someone in the US uses the word Jew and they're not obviously Jewish, they sound antisemitic.
This seems not universal and highly context-dependent.
Honestly, saying his examples ("asian" and "field work") are worse than yours ("black" and "gay") is very close to strawman arguing.
Well, my examples are both real and non-fringe, whereas "Asian" and "field work" are fictional and fringe, respectively. So, I think "gay" and "Black" are more central examples.
Scott also seems annoyed by "Black", but doesn't explain why he's (seemingly) annoyed.
There's a bit more here than I can readily respond to right now, but let me know if you think I've avoided the crux of the matter and you'd like me to address it in a future comment.
This doesn't apply to more central cases like "gay" and "Black".
Fair! I should have said 1,000 years to make the point more clear-cut.
It would be much more helpful if Scott used a real example rather than a fictional one. I don't think his fictional example is very realistic.
Thanks!
Thanks for posting this. I am still a bit fuzzy on what exactly the Superalignment plan is, or if there even is a firm plan at this stage. Hope we can learn more soon.
I think that me not wearing shoes at university is evidence that I might also disdain sports, but not evidence that I might steal.
it is not actually the case that violating one specific social norm for specific reason is a substantial update that someone is a Breaking Social Boundaries Type Pokemon in general.
If I can attempt to synthesize these two points into a single point: don't assume weird people are evil.
If someone walks around barefoot in an urban environment, that's a good clue they might also be weird in other ways. But weird ≠ evil.
Principled non-conformity is a thing. Human diversity is a thing. Eccentricity is a thing.
If weirdness indicated evil, then LessWrong would be a hive of scum and villainy.
Uncritically enforcing rules and conformity to an idea of normalcy is not good. It has done great harm.
Longtermism question: has anyone ever proposed a discount rate on the moral value of future lives? By analogy to discount rates used in finance and investing.
This could account for the uncertainty in predicting the existence of future people. Or serve as a compromise between views like neartermism and longtermism, or pro-natalism and anti-natalism.
Now he’s free to run for governor of California in 2026:
I was thinking about it because I think the state is in a very bad place, particularly when it comes to the cost of living and specifically the cost of housing. And if that doesn’t get fixed, I think the state is going to devolve into a very unpleasant place. Like one thing that I have really come to believe is that you cannot have social justice without economic justice, and economic justice in California feels unattainable. And I think it would take someone with no loyalties to sort of very powerful interest groups. I would not be indebted to other groups, and so maybe I could try a couple of variable things, just on this issue.
...
I don’t think I’d have enough experience to do it, because maybe I could do like a few things that would be really good, but I wouldn’t know how to deal with the thousands of things that also just needed to happen.
And more importantly than that to me personally, I wanted to spend my time trying to make sure we get artificial intelligence built in a really good way, which I think is like, to me personally, the most important problem in the world and not something I was willing to set aside to run for office.
Prediction market: https://manifold.markets/firstuserhere/will-sam-altman-run-for-the-governo
William Nordhaus estimates that firms recover maybe 2% of the value they create by developing new technologies.
Isn’t this the wrong metric? 2% of the value of a new technology might be a lot of money, far in excess of the R&D cost required to create it.
I think you are way overestimating your ability to tell who is trans and way underestimating the ability of trans people to pass as cis. Sometimes, you just can’t tell.
What on Earth? Why does it require being “devious” to be in the closet? If you were given a choice between lifelong celibacy and loneliness, on the one hand, or, on the other hand, seriously endangering yourself, risking being imprisoned or institutionalized, and ruining your life (economically and socially) by having relationships and disclosing them, would it make you “devious” to choose a third option and keep your relationships secret?
Were Jews who hid from the Nazis “devious”? Were people who helped them hide “devious”? Only in a sense that drains the word “devious” of its negative moral connotation.
The documentary “Before Stonewall” covers what gay life was like in the 40s, 50s, and 60s. I would recommend it.
The phrase “change sex” projects an anti-trans aura. (Not as much as using a slur, but it still makes me wince.) People say “transition” these days.
Another factor worth considering is that many trans people “pass” as cis, so you wouldn’t necessarily know someone is trans just by looking at them.
Does your town have a local PFLAG chapter? Another LGBT organization? If so, there might be trans people involved there.
He said:
At a high level you can think of Gemini as combining some of the strengths of AlphaGo-type systems with the amazing language capabilities of the large models
What do you think he meant by "AlphaGo-type systems"? I could be wrong, but I interpreted that as a reference to RL.
This seems super important to the argument! Do you know if it's been discussed in detail anywhere else?
We are on track to build many superhuman AI systems. Unless something unexpectedly good happens, eventually we will build one that has a failure of inner alignment. And then it will kill us all. Does the probability of any given system failing inner alignment really matter?
Yes, because if the first superhuman AGI is aligned, and if it performs a pivotal act to prevent misaligned AGI from being created, then we will avert existential catastrophe.
If there is a 99.99% chance of that happening, then we should be quite sanguine about AI x-risk. On the other hand, if there is only a 0.01% chance, then we should be very worried.
I have a question about "AGI Ruin: A List of Lethalities".
These two sentences from Section B.2 stuck out to me as the most important in the post:
...outer optimization even on a very exact, very simple loss function doesn't produce inner optimization in that direction.
...on the current optimization paradigm there is no general idea of how to get particular inner properties into a system, or verify that they're there, rather than just observable outer ones you can run a loss function over.
My question is: supposing this is all true, what is the probability of failure of inner alignment? Is it 0.01%, 99.99%, 50%...? And how do we know how likely failure is?
It seems like there is a gulf between "it's not guaranteed to work" and "it's almost certain to fail".
I don't know if anyone still reads comments on this post from over a year ago. Here goes nothing.
I am trying to understand the argument(s) as deeply and faithfully as I can. These two sentences from Section B.2 stuck out to me as the most important in the post (from the point of view of my understanding):
...outer optimization even on a very exact, very simple loss function doesn't produce inner optimization in that direction.
...on the current optimization paradigm there is no general idea of how to get particular inner properties into a system, or verify that they're there, rather than just observable outer ones you can run a loss function over.
My first question is: supposing this is all true, what is the probability of failure of inner alignment? Is it 0.01%, 99.99%, 50%...? And how do we know how likely failure is?
It seems like there is a gulf between "it's not guaranteed to work" and "it's almost certain to fail".
There is a strong argument that the term is bad and misleading. I will concede that.
Wolfram's article is very confusing indeed.
Most important sentence:
A reward function reshapes an agent's cognition to be more like the sort of cognition that got rewarded in the training process.
Wow. That is a tremendous insight. Thank you.
On another topic: you quote Yudkowsky in 2008 expressing skepticism of deep learning. I remember him in 2016 or 2017 still expressing skepticism, though much more mildly. Does anyone else recall this? Better yet, can you link to an example? [Edit: it might have been more like 2014 or 2015. Don’t remember exactly.]
I guess sort of the point of this post is that, in the broadest sense, the political critique of so-called “TESCREAL” lacks imagination — about the possible connections between these -isms and social justice.
I don’t think there’s anything inherently disparaging about the acronym.
What?
Thanks for your comment.
...note that indefinite life extension, reversing the aging process, etc, have never become a public priority in any polity.
Is this really strong evidence for anything? For example, the Methuselah Foundation was founded in 2001 and the SENS Research Foundation was founded in 2009. Calico was founded in 2013. Altos Labs was founded in 2021. All this to say, the science of radical life extension is extremely new. There hasn't been much time for life extension to become a political cause.
One motivation of the left is to lift up ordinary people, but another motivation is to bring down the privileged. The second motivation is the one that easily turns against projects for transcending the human condition.
Is your argument that, in the same way leftists oppose the rich and powerful, they also oppose transhumans? I think they oppose the idea of only the rich and powerful getting to become transhuman. To the extent they oppose a world in which anyone can become transhuman, I think it has to do with fears related to eugenics, rather than considerations of wealth, power, or privilege.
Yup. See: "love bombing".
"Someone gives a lot of compliments to me but I don't think they're being genuine"
Au contraire. This is a common tactic of manipulation and abuse.
"I feel 'low-value'"
I think the point is that they were treated as low-value by their bosses.
Wish I knew why this post is getting downvoted to karma hell! :(
I meant "extract" more figuratively than literally. For example, GPT-4 seems to have acquired some ability to do moral reasoning in accordance with human values. This is one way to (very indirectly) "extract" information from the human brain.
Extract from the brain into, say, weights in an artificial neural network, lines of code, a natural language "constitution", or something of that nature.
...I think the human brain’s intrinsic-cost-like-thing is probably hundreds of lines of pseudocode, or maybe low thousands, certainly not millions. (And the part that’s relevant for AGI is just a fraction of that.) Unfortunately, I also think nobody knows what those lines are. I would feel better if they did.
So, the human brain's pseudo-intrinsic cost is not intractably complex, on your view, but difficult to extract.
I think this functionality already exists.
Next to “Latest Posts” on the front page there is a “Customize Feed” button. You can set the topics “AI” and “AI risk” to “Hidden” and set the topic “Rationality” to “Promoted”.
Hope that helps.