Posts
Comments
Using the pronoun people ask you to use has become a proxy for all sorts of other tolerant/benevolent attitudes towards that person and the way they want to live their life, and to an even greater extent, refusing to do that is a proxy for thinking they should be ignored, or possibly reviled, or possibly killed.
There's an interesting mechanic here, a hyperstitious cascade. In certain educational environments, people are taught to use approved language with protected-class members. In that environment, anyone who uses forbidden language is, therefore, some kind of troublemaker. That then makes it somewhat less illegitimate for the most sensitive of those protected-class members to say they feel threatened when someone uses forbidden language. Which then makes it all the more important to teach people to use approved language, and have harsher enforcement on it. If this goes far enough, then we get to where one can make the case that unpunished usage of forbidden language constitutes a hostile environment, which would therefore drive out the protected classes and hence violate civil rights law.
I would expect that some amount of good safety research is of the form, "We tried several ways of persuading several leading AI models how to give accurate instructions for breeding antibiotic-resistant bacteria. Here are the ways that succeeded, here are some first-level workarounds, here's how we beat those workarounds...": in other words, stuff that would be dangerous to publish. In the most extreme cases, a mere title ("Telling the AI it's writing a play defeats all existing safety RLHF" or "Claude + Coverity finds zero-day RCE exploits in many codebases") could be dangerous.
That said, some large amount should be publishable, and 5 papers does seem low.
Though maybe they're not making an effort to distinguish what's safe to publish from what's not, and erring towards assuming the latter? (Maybe someone set a policy of "Before publishing any safety research, you have to get Important Person X to look through it and/or go through some big process to ensure publishing it is safe", and the individual researchers are consistently choosing "Meh, I have other work to do, I won't bother with that" and therefore not publishing?)
Any specific knowledge about colostrum? (Mildly surprised it hasn't been mentioned in the thread.) Do breastmilk banks usually supply that, and is it worthwhile?
The idea that ethical statements are anything more than "just expressions of emotion" is, to paraphrase Lucretius, "regarded by the common people as true, by the wise[1] as false, and by rulers as useful."
I figure you think the wise are correct. Well, then. Consider randomly selected paragraphs from Supreme Court justices' opinions. Or consider someone saying "I'd like to throw this guy in jail, but unfortunately, the evidence we have is not admissible in court, and the judicial precedent on rules of evidence is there for a reason—it limits the potential abusiveness of the police, and that's more important than occasionally letting a criminal off—so we have to let him go." Is that an ethical statement? And is it "just an expression of emotion"?
For the record, in an ethical context, when I say a behavior is bad, I mean that (a) an ethical person shouldn't do it (or at least should have an aversion to doing it—extreme circumstances might make it the best option) and (b) ethical people have license to punish it in some way, which, depending on the specifics, might range from "social disapproval" to "the force of the law".
Alarming and dangerous as this view may be, I'd be really surprised if literally everyone who had power ("in charge of anything important") also lacked the self-awareness to see it.
I think there are lots of people in power who are amoral, and this is indeed dangerous, and does indeed frequently lead to them harming people they rule over.
However, I don't think most of them become amoral by reading emotivist philosophy or by independently coming to the conclusion that ethical statements are "just expressions of emotion". What makes rulers frequently immoral? Some have hypothesized that there's an evolved response to higher social status, to become more psychopathic. Some have said that being psychopathic makes people more likely to succeed at the fight to become a ruler. It's also possible that they notice that, in their powerful position, they're unlikely to face consequences for bad things they do, and... they either motivatedly find reasons to drop their ethical principles, or never held them in the first place.
There's a philosophy called "emotivism" that seems to be along these lines. "Emotivism is a meta-ethical view that claims that ethical sentences do not express propositions but emotional attitudes."
I can see a couple of ways to read it (not having looked too closely). The first is "Everyone's ethical statements are actually just expressions of emotion. And, as we all know, emotions are frequently illogical and inappropriate to the situation. Therefore, everything anyone has ever said or will say about ethics is untrustworthy, and can reasonably be dismissed." This strikes me as alarming, and dangerous if any adherents were in charge of anything important.
The second reading is something like, "When humans implement ethical judgments—e.g. deciding that the thief deserves punishment—we make our emotions into whatever is appropriate to carry out the actions we've decided upon (e.g. anger towards the thief). Emotions are an output of the final judgment, and are always a necessary component of applying the judgment. However, the entire process leading up to the final judgment isn't necessarily emotional; we can try, and expect the best of us to usually succeed, at making that process conform to principles like logical consistency." That I would be on board with. But... that seems like a "well, duh" which I expect most people would agree with, and if that was what the emotivists meant, I don't see why they would express themselves the way they seem to.
I think a proper human morality somehow accounts for disgust having actually been an important part of how it was birthed.
I'm not sure if people maintain consistent distinctions between legal philosophy, ethics, and morality. But for whatever it is that governs our response to crimes, I think anger / desire-for-revenge is a more important part of it. Also the impulse to respond to threats ("Criminal on the streets! Who's he coming for next?"), which I guess is fear and/or anger.
Come to think of it, if I try to think of things that people declare "immoral" that seem to come from disgust rather than fear or anger, I think of restrictions on sexual behavior (e.g. homosexuality, promiscuity) and drugs, which I think the law shouldn't touch (except in forms where someone was injured nonconsensually, in which case revenge-anger comes into play). As emotions go, I think I'd distrust disgust more than the others.
The problem that you seem to be reaching for is a real one. You may find enlightening Leslie Lamport's "Buridan's principle":
A discrete decision based upon an input having a continuous range of values cannot be made within a bounded length of time.
The full paper discusses a similar situation:
Buridan’s Principle has appeared as a fundamental problem in computer design. In computing, a device that makes a discrete (usually binary) decision based upon a continuous input value is called an arbiter, and Buridan’s Principle is usually known as the Arbiter Problem [1].
...
If, as is usually the case, the peripheral device’s setting of the flag is not synchronized with the computer’s execution, then the computer’s binary decision is based upon an input having a continuous range of values. Buridan’s Principle asserts that the decision cannot be made in a bounded length of time. However, the computer must make that decision before beginning its next instruction, which generally happens in a fixed length of time.
The computer is thus trying to do something that is impossible. Just as the driver at the railroad crossing has a finite probability of being hit by the train, the computer has a finite probability of not making its decision in time. The physical manifestation of the computer’s indecision is that bad voltage levels are propagated. For example, if a 0 is represented by a zero voltage and a 1 is represented by +5 volts, then some wire might have a level of 2.5 volts. This leads to errors, because a 2.5 volt level could be interpreted as a 0 by some circuits and a 1 by others. The computer stops acting like a digital device and starts acting like a continuous (analog) one, with unpredictable results.
The Arbiter Problem is a classic example of Buridan’s Principle. The problem is not one of making the “right” decision, since it makes little difference if the interrupt is handled after the current instruction or after the following one; the problem is simply making a decision. The Arbiter Problem went unrecognized for a number of years because engineers did not believe that their binary circuit elements could ever produce “1/2’s”. The problem is solved in modern computers by allowing enough time for deciding so the probability of not reaching a decision soon enough is much smaller than the probability of other types of failure. For example, rather than deciding whether to interrupt execution after the current instruction, the computer can decide whether to interrupt it after the third succeeding instruction. With proper circuit design, the probability of not having reached a decision by time t is an exponentially decreasing function of t, so allowing a little extra time for the decision can make the probability of failure negligible.
...
Buridan’s Principle might lead one to suspect that a digital computer is an impossibility, since every step in its execution requires making discrete decisions within a fixed length of time. However, those decisions are normally based upon a discontinuous set of inputs. Whenever the value of a memory register is tested, each bit will be represented by a voltage whose value lies within two separate ranges—the range of values representing a zero or the range representing a one. Intermediate voltages are not possible because the register is never examined while it is in an intermediate state—for example, while a bit is changing from zero to one. The Arbiter Problem arises when the computer must interact asynchronously with an external device, since synchronization is required to prevent the computer from seeing an intermediate voltage level by reading a bit while the device is changing it. A similar problem occurs in interactions between the computer and its environment that require analog to digital conversion, such as video input.
The full paper is probably worth reading.
The paradox arises for people who lack a concept of "known unknowns" as distinct from "unknown unknowns". If our knowledge of x can only be in the state of "we know what x is and everything about it" or "we don't know anything about x and aren't even aware that anything like x exists", then the reasoning is all correct. However, for many things, that's a false binary: there are a lot of intermediate states between "zero knowledge of the concept of x" and "100% knowledge of x".
Yeah, learning by reading at home definitely has a huge effect in many cases. In Terence Tao's education, he was allowed to progress through multiple years of a subject per year (and to do so at different rates in different subjects), and since the classes he attended were normal ones, I think his academic progression must have been essentially determined by his ability to teach himself at home via textbooks. Unless perhaps they let him e.g. attend 7th grade science 2 days a week and 6th grade science the rest? I should learn more about his life.
The educational setup can also feed into the reading aspect. During my childhood, on a few occasions, I did explicitly think, "Well, I would like to read more of this math stuff (at home), but on the other hand, each thing I learn by reading at home is another thing I'll have to sit through the teacher telling me, being bored because I already know it", and actually decided to not read certain advanced math stuff because of that. (Years later, I changed my mind and chose to learn calculus from my sister's textbook around 8th grade—which did, in fact, cause me to be bored sitting through BC Calculus eventually.) This could, of course, be solved by letting kids easily skip past stuff by taking a test to prove they've already learned it.
Maybe with the objection that the time coefficient can be different for different school subjects, because some of them are more focused on understanding things, and others are more focused on memorizing things
Possibly. It's also the case that IQ is an aggregated measure of a set of cognitive subtests, and the underlying capabilities they measure can probably be factored out into things like working memory, spatial reasoning, etc., which are probably all correlated but imperfectly so; then if some of those are more useful for some subjects than others, you'll expect some variance in progression between subjects. And you certainly observe that the ultra-gifted kids, while generally above average at everything, are often significantly more ahead in math than in language, or vice versa (some of this is probably due to where they choose to spend their time, but I think a nonzero amount is innate advantage).
Among the various ways to take up the extra time of the rapid learners, probably the best one is "don't go faster, go wider".
The term of art, for doing this within a single subject, is "enrichment". And yeah, if you can do it, it fits nicely into schedules. "Taking more classes" is a more general approach. There are administrative obstacles to the latter: K-12 schools seem unlikely to permit a kid to skip half the sessions of one class so he can attend half the sessions of another class (and make up any gaps by reading the textbooks). Colleges are more likely to permit this by default, due to often not having attendance requirements, though one must beware of double-booking exams.
(Note: I am not saying that this is optimal for the rapid learner. The optimal thing for the rapid learner would be to... learn faster, obviously.
I think the best setup—can't find the citation—is believed to be "taking a class with equally gifted children of the same age, paced for them". If you don't have that, then skipping grades (ideally per-subject) would address knowledge gaps; taking a class paced for at least somewhat gifted kids (possibly called an "advanced" class, or a class at a high-tier college) would partly address the learning speed gap, and enrichment would also address the learning speed gap, to a variable extent depending on the details.
A more realistic example would be a math textbook, where each chapter is followed by exercises, some of them marked as "optional, too difficult"
A specific way of doing this, which I think would be good for education to move towards, is to have a programming component: have some of those optional exercises be "Write programs to implement the concepts from this chapter".
But if I had to guess, I would guess that the gifted kids who stay within the confines of school will probably lose most of their advantage, and the ones who focus on something else (competitions, books, online courses, personal projects) will probably keep it.
Subjects Not Permitted Acceleration. [...] With few exceptions, they have very jaded views of their education. Two dropped out of high school and a number have dropped out of university. Several more have had ongoing difficulties at university, not because of lack of ability but because they have found it difficult to commit to undergraduate study that is less than stimulating. These young people had consoled themselves through the wilderness years of undemanding and repetitive school curriculum with the promise that university would be different—exciting, intellectually rigorous, vibrant—and when it was not, as the first year of university often is not, it seemed to be the last straw.
Some have begun to seriously doubt that they are, indeed, highly gifted. The impostor syndrome is readily validated with gifted students if they are given only work that does not require them to strive for success. It is difficult to maintain the belief that one can meet and overcome challenges if one never has the opportunity to test oneself.
Versus:
Young People Who Have [skipped 3 or more grades by the end of high school]. [...] In every case, these young people have experienced positive short-term and long-term academic and socioaffective outcomes. The pressure to underachieve for peer acceptance lessened significantly or disappeared after the first acceleration. Despite being some years younger than their classmates, the majority topped their state in specific academic subjects, won prestigious academic prizes, or represented their country or state in Math, Physics, or Chemistry Olympiads. The majority entered college between ages 11 and 15. Several won scholarships to attend prestigious universities in Australia or overseas. All have graduated with extremely high grades and, in most cases, university prizes for exemplary achievement. All 17 are characterized by a passionate love of learning and almost all have gone on to obtain their Ph.D.s.
Though one could say this is more of an attitude and habit and "ever bothered to figure out study skills" thing, than a "you've permanently lost your advantage" thing. If you took one of those jaded dropouts (of 160+ IQ) and, at age 30, threw them into a job where they had to do some serious and challenging scientific work... There's a chance that their attitude and habits would make them fail and get fired within the first few months, that chance depending on how severe and how ingrained they are. But if they did ok enough to not get fired, then I expect that, within a year, they would be pulling ahead of a hypothetical 120 IQ counterpart for whom everything had gone great and who started with slightly more knowledge.
I'll give a citation on learning speed to show the extent of the problem, at least in early years (bold added):
Observation and investigation prove that in the matter of their intellectual work these children are customarily wasting much time in the elementary schools. We know from measurements made over a three-year period that a child of 140 IQ can master all the mental work provided in the elementary school, as established, in half the time allowed him. Therefore, one-half the time which he spends at school could be utilized in doing something more than the curriculum calls for. A child of 170 IQ can do all the studies that are at present required of him, with top "marks," in about one-fourth the time he is compelled to spend at school. What, then, are these pupils doing in the ordinary school setup while the teacher teaches the other children who need the lessons?
No exhaustive discussion of time-wasting can be undertaken here, except to say briefly that these exceptional pupils are running errands, idling, engaging in "busy work," or devising childish tasks of their own, such as learning to read backward—since they can already read forward very fluently. Many are the devices invented by busy teachers to "take up" the extra time of these rapid learners, but few of these devices have the appropriate character that can be built only on psychological insight into the nature and the needs of gifted children.
Note that this is above what the mere "intelligence quotient" seems to predict—i.e. at a given chronological age, IQ 140 children have 1.4x the mental age of those at IQ 100, and IQ 170 have 1.7x that mental age. So why would you get 2x and 4x learning speeds respectively?
One guess is that, at least within this elementary-school age range, higher mental age also means they've figured out better strategies for paying attention, noticing when you've understood something vs when you need to go back and re-read, etc.—which other kids may eventually figure out too. Another guess is that, for at least some kids, it seems that one of the things that manifests as higher intelligence is an increased need for intellectual stimulation; so in general, it may be that the higher-IQ kids are more naturally inclined to pay attention when the teacher is saying new things, and more inclined to read the textbook and think back to it, so there's less need for self-control, and so they're less handicapped by the lack of it in early years.
I don't know how far up in age the above extends. I do expect the lower-IQ learning rate to catch up somewhat in later years, perhaps bringing the difference down to the IQ ratio. (The learning-rate difference certainly doesn't disappear, though; I believe it's common that if an exceptionally gifted kid gets accelerated into college classes with adults of equal knowledge—several of whom are probably moderately gifted—she'll still finish at the top of her class.)
There's also a failure mode of focusing on "which arguments are the best" instead of "what is actually true". I don't understand this failure mode very well, except that I've seen myself and others fall into it. Falling into it looks like focusing a lot on specific arguments, and spending a lot of time working out what was meant by the words, rather than feeling comfortable adjusting arguments to fit better into your own ontology and to fit better with your own beliefs.
The most obvious way of addressing this, "just feel more comfortable adjusting arguments to fit better into your own ontology and to fit better with your own beliefs", has its own failure mode, where you end up attacking a strawman that you think is a better argument than what they made, defeating it, and thinking you've solved the issue when you haven't. People have complained about this failure mode of steelmanning a couple of times. At a fixed level of knowledge and thought about the subject, it seems one can only trade off one danger against the other.
However, if you're planning to increase your knowledge and time-spent-thinking about the subject, then during that time it's better to focus on the ideas than on who-said-or-meant-what; the latter is instrumentally useful as a source of ideas.
So, there’s no need to worry about the dictionary police enforcing how you should use a word, but understanding how “acceptance” is commonly used and comparing them to definitions found in common advice related to “acceptance” might help us
I see you practice acceptance. ;-)
There was also a character, Kotomine Kirei, who was brought up with good ethics and tried to follow them, but ultimately realized that the only thing that pleased him was causing other people pain... and there's an alternate universe work in which he runs a shop that sells insanely spicy mapo tofu. I suppose he could have gotten into the BDSM business as well. Drill sergeant? Interrogator? (That might not work, but there probably would be people who thought it did.)
I dropped out after 10th grade. I messed around at home, doing some math and programming, for ~6 years, then started working programming jobs at age 22 (nearly 10 years ago). I'd say results were decent.
A friend of mine dropped out after 11th grade. He has gone back and forth between messing around (to some extent with math and programming; in later years, with meditation) and working programming jobs, and I think is currently doing well with such a job. Probably also decent.
(And neither of us went to college, although I think my friend may have audited some classes.)
they use a relative ELO system
ELO itself is a relative system, defined by "If [your rating] - [their rating] is X, then we can compute your expected score [where win=1, draw=0.5, loss=0] as a function of X (specifically )."
that is detached from the FIDE ELO
Looking at the Wiki, one of the complaints is actually that, as the population of rated human players changes, the meaning of a given rating may change. If you could time-teleport an ELO 2400 player from 1950 into today, they might be significantly different from today's ELO 2400 players. Whereas if you have a copy of Version N of a given chess engine, and you're consistent about the time (or, I guess, machine cycles or instructions executed or something) that you allow it, then it will perform at the same level eternally. Now, that being the case, if you want to keep the predictions of "how do these fare against humans" up to date, you do want to periodically take a certain chess engine (or maybe several) and have a bunch of humans play against it to reestablish the correspondence.
Also, I'm sure that the underlying model with ELO isn't exactly correct. It asserts that, if player A beats player B 64% of the time, and player B beats player C 64% of the time, then player A must beat player C 76% of the time; and if we throw D into the mix, who C beats 64% of the time, then A and B must beat D 85% and 76% of the time, respectively. It would be a miracle if that turned out to be exactly and always true in practice. So it's more of a kludge that's meant to work "well enough".
... Actually, as I read more, the underlying validity of the ELO model does seem like a serious problem. Apparently FIDE rules say that any rating difference exceeding 400 (91% chance of victory) is to be treated as a difference of 400. So even among humans in practice, the model is acknowledged to break down.
This is expensive to calculate
Far less expensive to make computers play 100 games than to make humans play 100 games. Unless you're using a supercomputer. Which is a valid choice, but it probably makes more sense in most cases to focus on chess engines that run on your laptop, and maybe do a few tests against supercomputers at the end if you feel like it.
and the error bar likely increases as you use more intermediate engines.
It does, though to what degree depends on what the errors are like. If you're talking about uncorrelated errors due to measurement noise, then adding up N errors of the same size (i.e. standard deviation) would give you an error of √N times that size. And if you want to lower the error, you can always run more games.
However, if there are correlated errors, due to substantial underlying wrongness of the Elo model (or of its application to this scenario), then the total error may get pretty big. ... I found a thread talking about FIDE rating vs human online chess ratings, wherein it seems that 1 online chess ELO point (from a weighted average of online classical and blitz ratings) = 0.86 FIDE ELO points, which would imply that e.g. if you beat someone 64% of the time in FIDE tournaments, then you'd beat them 66% of the time in online chess. I think tournaments tend to give players more time to think, which tends to lead to more draws, so that makes some sense...
But it also raises possibilities like, "Perhaps computers make mistakes in different ways"—actually, this is certainly true; a paper (which was attempting to correspond FIDE to CCRL ratings by analyzing the frequency and severity of mistakes, which is one dimension of chess expertise) indicates that the expected mistakes humans make are about 2x as bad as those chess engines make at similar rating levels. Anyway, it seems plausible that that would lead to different ... mechanics.
Here are the problems with computer chess ELO ratings that Wiki talks about. Some come from the drawishness of high-level play, which is also felt at high-level human play:
Human–computer chess matches between 1997 (Deep Blue versus Garry Kasparov) and 2006 demonstrated that chess computers are capable of defeating even the strongest human players. However, chess engine ratings are difficult to quantify, due to variable factors such as the time control and the hardware the program runs on, and also the fact that chess is not a fair game. The existence and magnitude of the first-move advantage in chess becomes very important at the computer level. Beyond some skill threshold, an engine with White should be able to force a draw on demand from the starting position even against perfect play, simply because White begins with too big an advantage to lose compared to the small magnitude of the errors it is likely to make. Consequently, such an engine is more or less guaranteed to score at least 25% even against perfect play. Differences in skill beyond a certain point could only be picked up if one does not begin from the usual starting position, but instead chooses a starting position that is only barely not lost for one side. Because of these factors, ratings depend on pairings and the openings selected.[48] Published engine rating lists such as CCRL are based on engine-only games on standard hardware configurations and are not directly comparable to FIDE ratings.
It probably takes years before a tumor grows big enough for normal methods to detect it.
There exist fast-growing cancers. I figure that if the fungi theory is correct, then probably a good amount of this is caused by the specific fungus (and perhaps what part of the body that fungus targets), and most of the rest comes from the target's immune system (not sure what else would contribute significantly). If transmission and mild infections are common, and if, say, 1% of cancers are fast-growing, I feel like there should be lots of cases where an immunocompromised person picks up a fast-growing cancer fungus at a hospital or something and, within a few years, gets diagnosable cancer. Enough that it should have been noticed. I don't have numbers for this, but that's my suspicion.
Or, for example... How often do couples get the same type of cancer? I found this:
METHODS
The authors identified 25,670 cancer-free married couples in northern California who were followed for up to 31 years for the development of cancer. In Cox proportional hazards analysis, the development of cancer in a spouse was treated as a time-dependent, independent variable, and spouse-with/spouse-without risk ratios were determined, controlling for age and gender. For selected concordant espoused pairs, additional explanatory information was sought in their medical records.
RESULTS
There was no excess concordance for all cancers combined; the spouse-with/spouse-without risk ratio was 0.97 (95% confidence interval, 0.90–1.05). Statistically significant husband-wife associations were found only for cancer of the tongue and stomach and for non-Hodgkin lymphoma. Except for cancer of the penis/endometrium and testis/vulva, based on one couple with each combination, gender specific cancers did not aggregate within married couples. Established and suspected risk factors, not necessarily related to the marriage, were found for some individuals who had concordance with their spouses.
CONCLUSIONS
Little spousal concordance for cancer occurrence was found. The study of spousal aggregation does not appear useful in identifying unsuspected environmental causes of cancer in heterogeneous populations in urban areas of affluent Western countries. A cohort study would have to be much larger than this one to detect weak spousal concordance reliably..
Also, for whatever Claude's opinion is worth:
Q: How often do couples get the same type of cancer?
While it's not extremely common for couples to get the same type of cancer, it does happen occasionally. This phenomenon has been studied, and there are several factors to consider:
- Shared environmental factors: Couples often share the same living environment, diet, and lifestyle habits, which can expose them to similar cancer risk factors.
- Similar behaviors: Shared behaviors like smoking, alcohol consumption, or sun exposure can increase risk for certain cancers in both partners.
- Infectious agents: Some cancers are caused by infectious agents (like HPV for cervical cancer), which can be transmitted between partners.
- Age-related risks: As couples age together, they may face similar age-related cancer risks.
- Genetic factors: While not directly shared between couples, people might choose partners with similar genetic backgrounds, potentially influencing cancer risk.
- Coincidence: Given the prevalence of cancer, some couples will develop the same cancer by chance.
- Screening effect: When one partner is diagnosed, the other may be more likely to get screened, potentially leading to a diagnosis of a cancer that might have otherwise gone undetected.
Studies on this topic have shown:
- A slight increase in cancer risk for partners of cancer patients, but this varies by cancer type.
- Higher correlations for smoking-related cancers, suggesting shared lifestyle factors play a role.
- Increased risk for cancers with infectious causes, like stomach cancer (H. pylori) or liver cancer (hepatitis viruses).
It's important to note that while interesting, these occurrences are not common enough to be considered a significant public health concern.
My first significant thought (which came up a bit in the AIs' output) is that it would seem that, if fungi cause cancer, then the fungi would at least sometimes be transmitted from one person to another, and if you weren't aware of the fungi, then this would look like cancer being transmitted from one to the other. Yet I think this has basically never been observed.[1]
One could try supposing that each fungus is only rarely able to infect people—only the few individuals that are unusually vulnerable to it. But, well. I imagine that would generally include anyone whose immune system is crippled. Surely there have been enough cases of people with cancer next to old, immunocompromised people in a hospital, with sufficient mistakes in hygiene that the one would have infected the other. Maybe there are additional requirements for an individual to be infected (the fungus has a favorite temperature? Acidity? Salt level?)... but even taking that into account, I think there should have been enough cases that we would have noticed. (If the chance of an individual being infectable by a given fungus is so low that we never see transmission, then how is it that, er, 1/6th of all deaths are caused by cancer? There would have to be zillions of different fungi, each of which is able to infect only a tiny number of people... which surely would have led to natural selection for much better infectivity by now?)
Incidentally, I think it is known that there are some viruses (like HPV) that cause (or greatly heighten the risk of) cancer. It's plausible that fungi play a significantly larger role of this type than people realize. But for it to be the primary cause seems implausible.
The strongest evidence is that they found cancers that seem to have no mutations.
This seems worth digging into.
- ^
There are a few cases of cancers where it's known that the actual cancer cells themselves go from organism to organism: https://en.wikipedia.org/wiki/Clonally_transmissible_cancer
I note that the date on the tweet is June 28, the day after the Trump-Biden debate. It mentions the office of President and concludes with arguments in favor of having an 80-year-old president with no serious arguments against.
I give 95% odds the tweet was causally downstream of the debate: either directly inspired, or a result of arguing with people about old age and cognitive decline, for whom the subject came up because of the debate.
I'm not entirely sure if, or how, it was meant to be a comment on the debate. It could be that he wanted to downplay any perceived downsides to having his favored candidate as president. It could be that the topic was in the air and he had a take and wrote it up. (It is in-character for him to double down on his contrarian take, expressing doubts about an entire field of research in the process—for all that he's said about "used to be extremely confident". [To be sure, his criticisms are often valid, but going from "here are valid criticisms of his opponents" to "the opposite of what his opponents believe is the truth", or "you should throw out everything you thought you believed and just listen to the arguments he's making", is a risky leap.]) Who knows.
On the subject itself, he mentions slow memory recall, but not faulty memory, nor dementia, Alzheimer's, Parkinson's, and so on. Those are real and substantial problems. Perhaps a more rigorous thesis would be "Old people who are not affected by dementia, Alzheimer's, etc., including any subclinical levels thereof, can be intellectually very strong", and one can think of examples bearing this out. However, it's also true that the background probability of an 80-year-old having one or more of these issues is fairly high.
In context, I think "average people" means "the individual consumers paying $20/month to OpenAI". Who do you mean by "many"? I doubt that posters at /r/localllama are representative of "average people" by the above definition.
That is, your frame here is something like "planning is hard therefore you should distrust alignment plans".
But you could just as easily frame this as "abstract reasoning about unfamiliar domains is hard therefore you should distrust doom arguments".
I think received wisdom in cryptography is "don't roll your own crypto system". I think this comes from a bunch of overconfident people doing this and then other people discovering major flaws in what they did, repeatedly.
The lesson is not "Reasoning about a crypto system you haven't built yet is hard, and therefore it's equally reasonable to say 'a new system will work well' and 'a new system will work badly'." Instead, it's "Your new system will probably work badly."
I think the underlying model is that there are lots of different ways for your new crypto system to be flawed, and you have to get all of them right, or else the optimizing intelligence of your rivals (ideally) or the malware industry (less ideally) will find the security hole and exploit it. If there are ten things you need to get right, and you have a 30% chance of screwing up each one, then the chance of complete success is 2.8%. Therefore, one could say that, if there's a general fog of "It's hard to reason about these things before building them, such that it's hard to say in advance that the chance of failure in each thing is below 30%", then that points asymmetrically towards overall failure.
I think Raemon's model (pretty certainly Eliezer's) is indeed that an alignment plan is, in large parts, like a security system in that there are lots of potential weaknesses any one of which could torpedo the whole system, and those weaknesses will be sought out by an optimizing intelligence. Perhaps your model is different?
The original definition of IQ, intelligence quotient, is mental age (as determined by cognitive test scores) divided by chronological age (and then multiplied by 100). A 6-year-old with the test scores of the average 9-year-old thus has an IQ of 150 by the ratio IQ definition.
People then found that IQ scores roughly followed a normal distribution, and subsequent tests defined IQ scores in terms of standard deviations from the mean. This makes it more convenient to evaluate adults, since test scores stop going up past a certain age in adulthood (I've seen some tests go up to age 21). However, when you get too many standard deviations away from the mean, such that there's no way the test was normed on that many people, it makes sense to return to the ratio IQ definition.
So an IQ 300 human would theoretically, at age 6, have the cognitive test scores of the average 18-year-old. How would we predict what would happen in later years? I guess we could compare them to IQ 200 humans (of which we have a few), so that the IQ 300 12-year-old would be like the IQ 200 18-year-old. But when they reached 18, we wouldn't have anything to compare them against.
I think that's the most you can extract from the underlying model.
When is "unfalsifiability of X is evidence against X" incorrect?'
In some sense this must be at least half the time, because if X is unfalsifiable, then not-X is also unfalsifiable, and it makes little sense to have this rule constitute evidence against X and also evidence against not-X.
I would generally say that falsifiability doesn't imply anything about truth value. It's more like "this is a hypothesis that scientific investigation can't make progress on". Also, it's probably worth tracking the category of "hypotheses that you haven't figured out how to test empirically, but you haven't thought very hard about it yet".
There may be useful heuristics about people who make unfalsifiable claims. Some of which are probably pretty context-dependent.
Not sure if you intended this precise angle on it, but laying it out explicitly: If you compare a paid subscriber vs other readers, the former seems more likely to share your values and such, as well as have a higher prior probability on a thing you said being a good thing, and therefore less likely to e.g. take a sentence out of context, interpret it uncharitably, and spread outrage-bait. So posts with higher risk of negative interpretations are better fits for the paying audience.
My first guess is that there's a certain subpopulation for whom, if you treat their problems casually, joke about them, etc., then they'll get offended or hurt and cry while pointing a finger at you (perhaps literally), and thereby bring a bunch of social opprobrium upon you; and the acquaintances don't know enough to distinguish you from that subpopulation, and therefore treat you as though you might be one of them. It's a "safe" strategy they're following; the cost to them if they're wrong is relatively small, compared to the cost to them making the opposite error. (And perhaps they've done that all their life and don't have any practice joking about rough problems.)
(Take all guesses I make, about people I don't understand, with a big grain of salt.)
You might enjoy Gilbert and Sullivan's satirical light opera "Patience". In it, there's a poet that all the women of a village have fallen deeply in love with, except one woman, named Patience, who doesn't like him; and unfortunately the poet seems interested in Patience and not in any of the other women, who are therefore miserable and moping about it. Patience has never been in love, doesn't understand it, naively asks questions about it, and is puzzled by the answers. It thus serves as a vehicle to poke fun at various contemporary notions of what love is supposed to be.
MAIDENS.
Twenty love-sick maidens we,
Love-sick all against our will.
Twenty years hence we shall be
Twenty love-sick maidens still!
...
ANGELA Love feeds on hope, they say, or love will die;
MAIDENS Ah, misery!
ANGELA Yet my love lives, although no hope have I!
MAIDENS Ah, misery!
...
MAIDENS All our love is all for one, Yet that love he heedeth not,
He is coy and cares for none,
Sad and sorry is our lot!
Ah, misery!
...
PATIENCE.
I cannot tell what this love may be
That cometh to all but not to me.
It cannot be kind as they'd imply,
Or why do these ladies sigh?
It cannot be joy and rapture deep,
Or why do these gentle ladies weep?
It cannot be blissful as 'tis said,
Or why are their eyes so wondrous red?
...ANGELA. Ah, Patience, if you have never loved, you have never known true happiness! (All sigh.)
PATIENCE. But the truly happy always seem to have so much on their minds. The truly happy never seem quite well.
JANE. There is a transcendentality of delirium – an acute accentuation of supremest ecstasy – which the earthy might easily mistake for indigestion. But it is not indigestion – it is æsthetic transfiguration!
...
PATIENCE.
If love is a thorn, they show no wit
Who foolishly hug and foster it.
If love is a weed, how simple they
Who gather it, day by day!
If love is a nettle that makes you smart,
Then why do you wear it next your heart?
And if it be none of these, say I,
Ah, why do you sit and sob and sigh?
(The biggest missing factor in Patience's model is probably the fact that the maidens' love is unrequited. Though this is complicated by the fact that some people do enjoy fantasizing about not-necessarily-requited love, at least some of the time.)
Later, Patience gets the idea that love must be selfless... And therefore, it's improper for her to love someone who has lots of good qualities, because that would benefit her; instead she should marry an awful person, because living with them is absolutely unselfish. So she agrees to marry that poet, Bunthorne, who is vain, posturing, moody, petty, etc. But then Bunthorne promises to reform himself into a good man. Patience is initially delighted by this, but then realizes the implications.
PATIENCE. Oh, Reginald, I’m so happy! Oh, dear, dear Reginald, I cannot express the joy I feel at this change. It will no longer be a duty to love you, but a pleasure — a rapture — an ecstasy!
BUN. My darling! [embracing her]
PATIENCE. But — oh, horror! [recoiling from him]
BUN. What’s the matter?
PATIENCE. Is it quite certain that you have absolutely reformed — that you are henceforth a perfect being — utterly free from defect of any kind?
BUN. It is quite certain. I have sworn it.
PATIENCE. Then I never can be yours! [crossing to R.C.]
BUN. Why not?
PATIENCE. Love, to be pure, must be absolutely unselfish, and there can be nothing unselfish in loving so perfect a being as you have now become!
BUN. But, stop a bit. I don’t want to change — I’ll relapse — I’ll be as I was —
I would say that there is a place, in proper love relationships, for a thing that might at first glance resemble "unselfishness". But that thing is less "assigning zero value to your own happiness / well-being / etc." and more "assigning a similar value to your partner's utility as to your own", so that e.g. if something costs yourself 10 utils and benefits her 20 utils, you'll do it (and in a healthy relationship, lots of things like this happen in both directions and it's net positive for both). But it's pretty fair to say that general cultural transmission doesn't make things like this clear.
Absolutely. For a quick model of why you get multiplicative results:
- Intelligence—raw intellectual horsepower—might be considered a force-multiplier, whereby you produce more intellectual work per hour spent working.
- Motivation (combined with say, health) determines how much time you spend working. We could quantify it as hours per week.
- Taste determines the quality of the project you choose to work on. We might quantify it as "the expected value, per unit of intellectual work, of the project".
Then you literally multiply those three quantities together and it's the expected value per week of your intellectual work. My mentor says that these are the three most important traits that determine the best scientists.
Also:
- Knowing the importance of the advantages people have makes you better able to judge how well people are likely to do, which lets you make better decisions when e.g. investing in someone's company or deciding who to hire for an important role (or marry).
- Also orients you towards figuring out the difficult-to-see advantages people must have (or must lack), given the level of success that they've achieved and their visible advantages.
- If you're in a position to influence what advantages people end up with—for example, by affecting the genes your children get, or what education and training you arrange for them—then you can estimate how much each of those is worth and prioritize accordingly.
One of the things you probably notice is that having some advantages tends to make other advantages more valuable. Certainly career-wise, several of those things are like, "If you're doing badly on this dimension, then you may be unable to work at all, or be limited to far less valuable roles". For example, if one person's crippling anxiety takes them from 'law firm partner making $1 million' to 'law analyst making $200k', and another person's crippling anxiety takes them from 'bank teller making $50k' to 'unemployed', then, well, from a utilitarian perspective, fixing one person's problems is worth a lot more than the other's. That is probably already acted upon today—the former law partner is more able to pay for therapy/whatever—but it could inform people who are deciding how to allocate scarce resources to young people, such as the student versions of the potential law partner and bank teller.
(Of course, the people who originally wrote about "privilege" would probably disagree in the strongest possible terms with the conclusions of the above lines of reasoning.)
On the angle of demonstrating that you can learn the material and the skills and generally proving your math mettle: Can you study the books, do a sampling of the problems in the back of each chapter until you think you've mastered it, and then take the tests directly, without being signed up for a class? Maybe find old exams, perhaps from other institutions (surely someone somewhere has published an exam on each subject)? Or, for that matter, print out copies of old Putnam contests, set a timer, and see how well you do?
As someone who never entered college in the first place, I consider it a prosocial thing to make college degrees less correlated with competence. Don't add to the tragedy of that commons!
I grew up knowing "privilege" to mean a special right that was granted to you based on your job/role (like free food for those who work at some restaurants) or perhaps granted by authorities due to good behavior (and would be taken away for misusing it). Note also that the word itself, "privi"-"lege", means "private law": a law that applies to you in particular.
Rights and laws are social things, defined by how others treat you. To say that your physical health is a privilege therefore seems like either a category error, or a claim that other people treated you better in a way that gave you your better physical health, which then raises questions like "What made you deserve that treatment?" or perhaps "Is it really because of how other people treated you, or other reasons like genetics or having made healthier life choices?". The latter may then lead to "Yeah, but you grew up being taught better and/or in a situation where healthy choices were more affordable, which are probably partly caused by wealth and are privilege", both of which might be counter-argued in the specific person's case or in general, and so on. Social justice arguments ensue.
"Advantage" seems like a more neutral term, one that doesn't inherently imply fairness-laden claims about how you got it. I would recommend it.
The first, second, and third considerations in such a study would be ruling out other directions of causality:
- Does having a sore throat make people hum less?
- Does being healthy make people happy, and therefore inclined to hum more?
- Does humming correlate with being in a choir, which may also cause respiratory diseases to be spread more often (in the absence of precautions)?
- Does living in close quarters with lots of other people make them likely to stop you from humming, and also make them more likely to pass diseases onto you?
- Does having this happen early in life give you a stronger immune system or at least plenty of immunity to common diseases?
- Do certain cultures have attitudes towards humming and also attitudes to hygiene that are causally relevant?
- ...
I would be extremely, extremely skeptical of any study on the subject other than a randomized intervention.
I had heard, 15+ years ago (visiting neuroscience exhibits somewhere), about experiments involving people who, due to brain damage, can no longer form new memories. And Wiki agrees with what I remember hearing about some cases: that, although they couldn't remember any new events, if you had them practice a skill, they would get good at it, and on future occasions would remain good at it (despite not remembering having learned it). I'd heard that an exception was that they couldn't get good at Tetris.
Takeaway: "Memory" is not a uniform thing, and things that disrupt memory don't necessarily disrupt all of it. So beware of that in any such testing. In fact, given some technique that purportedly blocks memory formation, "Exactly what memory does it block?" is a primary thing to investigate.
One class of variance in cognitive test results is probably, effectively, pseudorandomness.
Suppose there's a problem, and there are five plausible solutions you might try, two of which will work. Then your performance is effectively determined by the order in which you end up trying solutions. And if your skills and knowledge don't give you a strong reason to prefer any of them, then it'll presumably be determined in a pseudorandom way: whichever comes to mind first. Maybe being cold subconsciously reminds you of when you were thinking about stuff connected to Solution B, or discourages you from thinking about Solution C. Thus, you could get a reliably reproducible result that temperature affects your performance on a given test, even if it has no "real" effect on how well your mind works and wouldn't generalize to other tests.
This should be addressable by simply taking more, different, cognitive tests to confirm any effect you think you've found.
Pithy sayings are lossily compressed.
One aspect neither of you have explicitly addressed is the speaking of numbers; speaking, after all, predates writing. We say "one billion, four hundred twenty-eight million, [...]".
Given that that's what we say, the first two pieces of information we need are "one" and "billion". More generally, we need to get the first 1-3 digits (the leftmost comma-separated group), then we need the magnitude, then we can proceed reading off all remaining digits.
Given that the magnitude is not explicitly written down, we get it by counting the digits. If the digits are comma-separated into groups of 3 (and "right-justified", so that if there are 3n+1 or 3n+2 digits, then the extra 1-2 are the leftmost group), then it's generally possible to get the magnitude from your "peripheral vision" (as opposed to counting them one by one) for numbers less than, say, 1 billion, which are what you'd most often encounter; like, "52" vs "52,193" vs "52,193,034", you don't need to count carefully to distinguish those. (It gets harder around 52,193,034,892 vs 52,193,034,892,110, but manually handling those numbers is rare.) So if getting the magnitude is a mostly free operation, then you might as well just present the digits left-to-right for people who read left-to-right.
Now, is it sensible that we speak "one billion, four hundred twenty-eight million, [...]"? Seems fine to me. It presents the magnitude and the most significant digits first (and essentially reminds you of the magnitude every 3 digits), and either the speaker or the listener can cut it off at any point and have an estimate accurate to as many digits as they care for. (That is essentially the use case of "partially running the algorithm" you describe.) I think I'd hate listening to "six hundred sixty three, six hundred twenty-seven thousand, four hundred twenty-eight million, and one billion", or even suffixes of it like "four hundred twenty eight million and one billion". Tell me the important part first!
The other big-endian algorithm is the one I observe myself as usually using. For "321", it is:
- Count the digits (three), and convert that into an order of magnitude (one hundred). (Running total: ??? hundred ???.)
- Read the first digit, multiply it by its order of magnitude (one hundred), and add it to the total. (Running total: three hundred ???.)
- Read the second digit, multiply it by its order of magnitude (ten), and add it to the total. (Running total: three hundred and twenty ???.
- Read the third digit, multiply it by its order of magnitude (one), and add it to the total. (Arriving at three hundred and twenty one.)
I generally agree, except I find words like "multiply" and "add" a bit misleading to use in this context. If I read a number like 3,749,328, then it's not like I take 3 million, and then take 7, multiply by 100,000, and get 700,000, and then perform a general-purpose addition operation and compute the subtotal of 3,700,000. First of all, "multiply by 100,000" is generally more like "Shift left by 5 (in our base-10 representation)"; but moreover, the whole operation is more like a "Set the nth digit of the number to be this". If this were a computer working in base 2, "set nth digit" would be implemented as "mask out the nth bit of the current number [though in this case we know it's already 0 and can skip this step], then take the input bit, shift left by n, and OR it with the current number".
(In this context I find it a bit misleading to say that "One hundred plus twenty yields one hundred and twenty" is performing an addition operation, any more than "x plus y yields x+y" counts as performing addition. Because 100, by place-value notation, means 1 * 100, and 20 means 2 * 10, and 120 means 1 * 100 + 2 * 10, so you really are just restating the input.)
Also, I might switch the order of the first two steps in practice. "Three ... [pauses to count digits] million, seven hundred forty-nine thousand, ...".
Take note of the Underhanded C Contest for inspiration on the problem of auditing code written by an intelligent, untrusted source. I think one takeaway is that, with some ingenuity, one can often put malicious behavior into ok-looking code that maybe contains some innocent mistakes. It seems, then, that fully guarding against malicious code implies fully guarding against all bugs.
Which might be achievable if your style guides put heavy limitations on what kind of code can be written. (The halting problem makes it impossible for a deterministic program to always detect all bugs.) Something perhaps like the JPL C standards:
- Predictable Execution
- Use verifiable loop bounds for all loops meant to be terminating.
- Do not use direct or indirect recursion.
- Do not use dynamic memory allocation after task initialization.
...
Rule 3 (loop bounds)
All loops shall have a statically determinable upper-bound on the maximum number of loop iterations. It shall be possible for a static compliance checking tool to affirm the existence of the bound. An exception is allowed for the use of a single non-terminating loop per task or thread where requests are received and processed. Such a server loop shall be annotated with the C comment: /* @non-terminating@ */
Agreed on most of the above, but on this particular point:
This is pretty much parallel to a common argument for laws against euthanasia, assisted suicide, etc.: the easier it is for someone with terrible medical conditions to arrange to die, [...], or (this isn't quite parallel, but it seems clearly related) to make it appear that they've done so when actually they were just murdered.
I would expect the opposite there. If assisted suicide and stuff is legalized, I expect that to come with high standards of "There should be a notarized signature, multiple witnesses, a video from the person in question stating their intentions, and they walk into a building where some official people first take the person into another room and say 'Are these men coercing you? We can have our security staff subdue them and bring in the police'", etc., designed specifically to make it hard to cover up a murder like that. And the existence of that option should push a chunk of regular suicides in that direction, making it less plausible that someone would commit suicide in the "traditional" way where they give no one any warning, may or may not leave a note, etc.
Yup. Many programmer applicants famously couldn't solve FizzBuzz. Which is probably because:
[skipping several caveats and simplifying assumptions]
Now, when you get those 200 resumes, and hire the best person from the top 200, does that mean you’re hiring the top 0.5%?
“Maybe.”
No. You’re not. Think about what happens to the other 199 that you didn’t hire.
They go look for another job.
That means, in this horribly simplified universe, that the entire world could consist of 1,000,000 programmers, of whom the worst 199 keep applying for every job and never getting them, but the best 999,801 always get jobs as soon as they apply for one. So every time a job is listed the 199 losers apply, as usual, and one guy from the pool of 999,801 applies, and he gets the job, of course, because he’s the best, and now, in this contrived example, every employer thinks they’re getting the top 0.5% when they’re actually getting the top 99.9801%.
Without regard to anything specific to LLMs... Math works the same for all conceivable beings. Beings that live in our universe, of sufficient advancedness, will almost certainly know about hydrogen and other elements, and fundamental constants like Planck lengths. So there will exist commonalities. And then you can build everything else on top of those. If need be, you could describe the way things looked by giving 2D pixel-grid pictures, or describe an apple by starting with elements, molecules, DNA, and so on. (See Contact and That Alien Message for explorations of this type of problem.)
It's unlikely that any LLM resembling those of today would translate the word for an alien fruit into a description of their own DNA-equivalent and their entire biosphere... But maybe a sufficiently good LLM would have that knowledge inside it, and repeatedly querying it could draw that out.
The Wikipedia link for claddagh rings goes to the Project Semicolon page.
The true thing that Sagan's line might be interpreted to mean is "A claim which is very unlikely on priors needs very strong evidence to end up with a posterior probability close to 1." "Extraordinary evidence" would ideally have been stated as "extraordinarily strong evidence", but that makes the line a bit clunkier. Unfortunately, there is often a tradeoff between accuracy and pithiness. Many pithy sayings require a bit of interpretation/reconstruction to get the correct underlying idea. I think anyone who invokes a catchphrase should be aware of this, though I don't know how many people share this perspective.
Are there in fact a significant number of people who take it at the face value of "extraordinary evidence" and think it must mean it was obtained via super-advanced technology or something?
I think either I don't know exactly what defines a "subculture", or there needs to be a qualifier before "subculture". Might "people who are enthusiastic about X sport / hobby / profession" be a subculture? Because I think lots of those can be highly successful while remaining what they are. (Perhaps you'd say that hobbies that succeed get eaten in the Geeks/MOPs/Sociopaths way, but that's less so for professions.)
A "subculture of those dealing with X problem" sounds much more likely to fit what you describe, but that may not be your intent.
imagine a friend from a different country is visiting and will stay with you for a while. You're exchanging some text messages beforehand in order to figure out how to spend your time together. You want to show your friend the city, and you want to be very accommodating and make sure all their preferences will be met. So you simply ask them: "What do you want to do"? And maybe you add "I'm completely fine with anything!" to ensure you're really introducing no constraints whatsoever and you two can do exactly what your friend desires.
An additional angle on situations like this: Your friend may be hoping to choose something that's positively enjoyable for you. Saying "I'm completely fine with anything" may not meet that bar, and doesn't give any hints as to what would. To illustrate directly, compare "There are ten restaurants nearby and I'm fine with any of them" vs "There are ten restaurants nearby, I've been to them all and I love them all". I think there are people who would respond to the second with "Great, I'll look them up and pick my favorite" and would find the first frustrating (and may respond by probing, "Well, are there any that you particularly like?" [And if you really don't care about food, then their hope to find a restaurant you enjoy is destined for frustration.]).
In this case there's also the aspect that, since you live there (likely for some years) and they're from another country, you likely know a lot more about the local offerings than they do (not guaranteed—perhaps you're an ascetic who doesn't explore such things and they're a tourist who has researched your town—but likely), so in a division-of-labor sense it's likely appropriate for you to volunteer info first.
That second aspect is indeed about the pure computational problem. The first aspect is a combination of the computation/search problem and an emotional negotiation element.
A friend of mine mentioned the article, and here's what I wrote.
Some of it seems pretty unfair. The early anecdote about charity work in Bali seems to be used to criticize the EA ethos, when "rich westerners flying to poor countries to do manual labor (and possibly post on social media about how virtuous they're being)" is the classic case of something EAs consider to be an ineffective and wasteful charity. (Though perhaps EAs might not go so far as to expect it to be net harmful.) I don't think most EAs would agree to the "bet the Earth on a 51% chance" scheme. As the author says, "SBF consistently made terrible choices" even according to SBF's own goals, so I don't think one can point to his bad outcome as evidence that his goals were bad, except perhaps via a psychological argument that having his grand goals and being a powerful player led him to think "good-for-me-now and good-for-everyone-always started to merge into one" (and subsequent self-serving bias), which would be a general argument against having grand goals and being a powerful player.
But the thing of "lots of aid money ends up in the hands of corrupt middlemen and oppressive rulers and might make the whole thing net negative" is a good point; the specific things that went wrong are good to know about; and the thing of GiveWell not taking seriously and honestly the harm caused by the aid (which, given how they operate, would indeed mean publicly writing up calculations) is a very good point. They should take "tracking the negative consequences" into their routine practice; e.g. there exists N such that >$N being given to a cause justifies having a person go and investigate what's happening.
I hadn't checked any of the specific claims about GiveWell's charities going wrong, or about what they have or haven't written about the downsides; I basically took the author's word on that.
Looking at Wiki's Undercover Journalism article, one that comes to mind is Nellie Bly's Ten Days in a Mad-House.
[Bly] took an undercover assignment for which she agreed to feign insanity to investigate reports of brutality and neglect at the Women's Lunatic Asylum on Blackwell's Island. [...]
Once admitted to the asylum, Bly abandoned any pretense at mental illness and began to behave as she would normally. The hospital staff seemed unaware that she was no longer "insane" and instead began to report her ordinary actions as symptoms of her illness. Even her pleas to be released were interpreted as further signs of mental illness. Speaking with her fellow patients, Bly was convinced that some were as "sane" as she was.
Bly experienced the deplorable conditions firsthand. The nurses behaved obnoxiously and abusively, telling the patients to shut up, and beating them if they did not. The food consisted of gruel broth, spoiled beef, bread that was little more than dried dough, and dirty undrinkable water. The dangerous patients were tied together with ropes. The patients were made to sit for much of each day on hard benches with scant protection from the cold. Waste was all around the eating places. Rats crawled all around the hospital. [...]
The bath water was rarely changed, with many patients bathing in the same filthy water. Even when the water was eventually changed, the staff did not scrub or clean out the bath, instead throwing the next patient into a stained, dirty tub. The patients also shared bath towels, with healthy patients forced to dry themselves with a towel previously used by patients with skin inflammations, boils, or open sores.
Interestingly...
While physicians and staff worked to explain how she had deceived so many professionals, Bly's report prompted a grand jury to launch its own investigation[9] with Bly assisting. The jury's report resulted in an $850,000 increase in the budget of the Department of Public Charities and Corrections. The grand jury also ensured that future examinations were more thorough such that only the seriously ill were committed to the asylum.
I can't say I'm happy with failure being rewarded with a higher budget. Still, it may have been true that their budget was insufficient to provide sanitary and humane conditions. Anyway, the report itself seems to have been important and worthwhile.
The ones that come to my mind are "Person or Organization X is doing illegal, unethical, or otherwise unpopular practices which they'd rather conceal from the public." Lie that you're ideologically aligned or that you'll keep things confidential, use that to gain access. Then perhaps lie to blackmail them to give up a little more information before finally publishing it all. There might be an ethical line drawn somewhere, but if it's not at "any lying" then I don't know where it is.
Ok, let's consider the case of shadowy influencers like that. It would be nice to know who such people were, sure. If they were up to nefarious things, or openly subscribed to ideologies that justify awful actions, then I'd like to know that. If there was an article that accurately laid out the nefarious things, that would be nice. If the article cherry-picked, presented facts misleadingly, made scurrilous insinuations every few paragraphs (without technically saying anything provably false)—that would be bad, possibly quite bad, but in some sense it's par for the course for a certain tier of political writing.
When I see the combination of making scurrilous insinuations every few paragraphs and doxxing the alleged villain, I think that's where I have to treat it as a deliberate cancellation attempt on the person. If it wasn't actually deliberate, then it was at least "reckless disregard" or something, and I think it should be categorized the same way. If you're going to dox someone, I figure you accept an increased responsibility to be careful about what you say about them. (Presumably for similar reasons, libel laws are stricter about non-public figures. No, I'm not saying it's libel when the statements are "not technically lying"; but it's bad behavior and should be known as such.)
As for the "they're probably robust" aspect... As mentioned in my other comment, even if they predictably "do well" afterwards, that doesn't mean they haven't been significantly harmed. If their influence is a following of 10M people, and the cancellation attempt reduces their influence by 40%, then it is simultaneously true that (a) "They have an audience of 6M people, they're doing fine", and (b) "They've been significantly harmed, and many people in such a position who anticipated this outcome would have a significant incentive to self-censor". It remains a bad thing. It's less bad than doing it to random civilians, sure, but it remains bad.
I don’t trust that the replacements will be actually good
If ∑(people who agree with you on this * how much they agree with you) is high enough, that constitutes market demand for making an actually good replacement, or remaking the original. Then the cost-benefit calculation determines whether the market will incentivize someone (or an AI) to solve the problem.
If you, personally, have preferences that are truly unusual, then that could be a problem. Though I do expect the technology to make individually customized solutions cheaper and more feasible (in the vein of 3D printing). Perhaps, among all the truly unusual preferences you have, the net move will be towards more satisfaction of them.
But one difference between doxxing normal people versus doxxing "influential people" is that influential people typically have enough power to land on their feet when e.g. they lose a job.
It may decrease their influence substantially, though. I'll quote at length from here. It's not about doxxing per se, but it's about cancellation attempts (which doxxing a heretic enables), and about arguments similar to the above:
If you’re a writer, artist or academic who has strayed beyond the narrow bounds of approved discourse, two consequences will be intimately familiar. The first is that it becomes harder to get a hearing about anything. The second is that if you do manage to say anything publicly — especially if you talk about the silencing — it will be taken as proof that you have not been silenced.
This is the logic of witch-ducking. If a woman drowns, she isn’t a witch; if she floats, she is, and must be dispatched some other way. Either way, she ends up dead.
The only counter to this is specific examples. But censorship is usually covert: when you’re passed over to speak at a conference, exhibit in a gallery or apply for a visiting fellowship, you rarely find out. Every now and then, however, the censors tip their hands.
And so, for everyone who says I can’t have been cancelled because they can still hear me, here’s the evidence.
The first time I know I was censored was even before my book criticising trans ideology came out in mid-2021. I had been asked to talk about it on the podcast of Intelligence Squared, a media company that, according to its website, aims to “promote a global conversation”. We had booked a date and time.
But as the date approached I discovered I had been dropped. When I asked why, the response was surprisingly frank: fear of a social-media pile-on, sponsors getting cold feet and younger staff causing grief. The CEO of Intelligence Squared is a former war correspondent who has written a book about his experiences in Kosovo. But at the prospect of platforming a woman whose main message is that humans come in two sexes, his courage apparently ran out.
Next came the Irish Times, my home country’s paper of record. Soon after my book came out a well-known correspondent rang me, said he had stayed up all night to finish it and wanted to write about it. He interviewed me, filed the piece, checked the quotes — and then silence. When I nudged by email, he said the piece had been spiked as it was going to press.
Sometime around then it was the BBC’s turn. I don’t know the exact date because I only found out months later, when I met a presenter from a flagship news programme. Such a shame you couldn’t come on the show, he said, to which I replied I had never been asked. It turned out that he had told a researcher to invite me on, but the researcher hadn’t, instead simply lying that I wasn’t available. I’ve still never been on the BBC to discuss trans issues.
Next came ABC, the Australian state broadcaster, which interviewed me for a radio show about religion and ethics. This time, when I nudged, I was told there had been “technical glitches” with the recording, but they would “love to revisit this one in the future”. They’ve never been back in touch. [... several more examples ...]
Now, the author has a bestselling book, has been on dozens of podcasts, and now works for an advocacy organization that's 100% behind her message (she's not technically listed as a cofounder, but might as well be). She has certainly landed on her feet and has a decent level of reach; yet, clearly, if not for a bunch of incidents like the above—and, as she says, probably a lot more incidents for which she doesn't have specific evidence—then she would have had much greater reach.
In Scott's case... if we consider the counterfactual where there wasn't a NYT article drawing such smears against Scott, then, who knows, maybe today some major news organizations (perhaps the NYT itself!) would have approached him for permission to republish some Slate Star Codex articles on their websites, perhaps specifically some of those on AI during the last ~year when AI became big news. Or offered to interview him for a huge audience on important topics, or something.
So be careful not to underestimate the extent of unseen censorship and cancellation, and therefore the damage done by "naming and shaming" tactics.
In general, I would say:
- Just because someone wasn't successfully canceled, doesn't mean there wasn't a cancellation attempt, nor that most other people in their position would have withstood it
- Just because they're doing well now, doesn't mean they wouldn't have been doing better without the cancellation attempt
- Even if the cancellation attempt itself did end up actually benefiting them, because they had the right personality and skills and position, that doesn't mean this should have been expected ex ante
- (After all, if it's clear in advance to everyone involved that someone is uncancellable, then they're less likely to try)
- Even if it's factually true that someone has the qualities and position to come out ahead after cancellation, they may not know or believe this, and thus the prospect of cancellation may successfully silence them
- Even if they're currently uncancellable and know this, that doesn't mean they'll remain so in the future
- E.g. if they're so good at what they do as to be unfireable, then maybe within a few years they'll be offered a CEO position, at which point any cancel-worthy things they said years ago may limit their career; and if they foresee this, then that incentivizes self-censorship
The point is, cancellation attempts are bad because they create a chilling effect, an environment that incentivizes self-censorship and distorts intellectual discussion. And arguments of the form "Hey, this particular cancellation attempt wasn't that bad because the target did well" fall down to one or more of the above arguments: they still create chilling effects and that still makes them bad.