There is trust in the practical abilities. Right now it is low, but that will only go up.

Part of the learning curve for using existing AI is calibrating trust and verifying answers, conditional on use case. A hallmark of inexperienced AI users is taking its replies at face value, without checking.

I do expect that over time, AI will become more trustworthy for daily users. But that is compatible with the trust users place in it decreasing as they familiarize themselves with the technology and learn its limitations.

I’ve participated in several alternative communities over the course of my life, and all became mired in scandal. The first was my college, where tolerance of hard drug use by the administration resulted in multiple OD deaths in my time there. The second was in my 20s in an intentional living and festival culture, when a major community figure was accused by multiple women of drugging and raping them while unconscious. The third was the EA and rationality community, which of course has had one scandal after another for years.

My model is that drugs, extreme ideas, mental illness, economic precarity, alternative cultures and institutions, power differences, and violent behavior are mutually reinforcing. Rationalists may drastically underestimate how intensely the recruitment funnel they’ve created selects for interest by disturbed people. Or the extent to which features correlated with the movement, like lifestyle experimentation, alternative spirituality or drug use, may be the central attractions for some, using rationality as a pretext. It’s the opposite of evaporative cooling - it’s condensation of crazy.

My belief is that to counteract this, it’s necessary to promote some level of conformism and convention to the standards and norms of society at large. Think of academia in STEM. Yes, the people and jobs are unusual. But the requirement for participants to repeatedly integrate into new departments at new universities repeatedly and work with many collaborators and a constant churn of new students from around the world makes for a melting pot culture where condensation of crazy is mitigated on a per capita basis.

In short, it seems to me that it’s in the very nature of niche movements and alternative communities to generate scandal for systematic sociological reasons. There is probably no way to retain the niche alt community structure without the scandal. Individuals will have to choose whether or not they prioritize a low frequency of scandal, or having the community exist in its present form.

A question I ask when sizing up a community is “do these people seem likely to be much more scandal-prone than the average church, sports team or workplace?” I also ask this for people I am considering getting to know. I go with my intuition and choose how to engage accordingly.

We can do the same with living organisms. The human genome contains about 6.2 billion nucleotides. Since there are 4 nucleotides (A, T, G, C), we need two bits for each of them, and since there are 8 bits in a byte, that gives us around 1.55 GB of data.

In other words, all the information that controls the shape of your face, your bones, your organs and every single enzyme inside them – all of that takes less storage space than Microsoft Word™.

There are two ways to see this is incorrect.

1. DNA's ability to structure an organism is mediated through its chemical mileau. It is densely, dynamically regulated through a complex and dense mesh of proteins  and regulatory RNA and small signaling molecules at every timepoint in every organism throughout the life cycle. Disruption of that chemical mileau renders the organism nonviable. This is a separate issue from the fact that evolution overwhelmingly operates on DNA sequence.
2. The DNA in a particular organism/cell is one point in a very long series of complex inheretance chains going back 4.5 billion years. I'm comfortable rounding off the maximum complexity of the soma to the maximum possible complexity of the complete set of ancestral DNA sequences. But we can go further by noticing that an individual's DNA sequence is not just the combination of their direct ancestors -- the entire ancestral lineage at every step is sampled from a distribution of possible genomes that is produced from mechanisms impacting reproduction.

In a more mathematical sense, while it's true that, conditional on a specific non-stochastic function, the number of values in the output set is less than or equal to the number of values in the input set, if the function can vary freely then there is no such constraint.

The soma might be viewed as a stochastic function mapping DNA inputs to phenotypic outputs. The stochastic aspect gives a much larger number of theoretically possible outputs from the same input set. And the fact that the 'function' (soma) itself varies from organism to organism increases the number of phenotypes that can be generated from a given amount of DNA still further.

All these arguments also apply to technology. MS Word 'co-evolved' with Windows, with programming languages, with hardware, and this context must be taken into account when thinking about how complex a machine is.

The CDC and other Federal agencies are not reporting updates. "It was not clear from the guidance given by the new administration whether the directive will affect more urgent communications, such as foodborne disease outbreaks, drug approvals and new bird flu cases."

I drink about 400mg of caffeine daily through coffee and Coke Zero. It helps me process complex ideas quickly, consider alternatives, and lifts my mood.

Without it, I get frustrated when I can’t follow arguments or understand ideas, often rejecting them or settling for “good enough.” Caffeine gives me the clarity and energy to stay open to new ideas and better solutions.

Stable is not a virtue, nor is our equilibrium well-tolerated. The problems it causes in terms of health, cost and homelessness are central political issues and have been for a long time.

I also have no idea why you assume I’m “ignoring” these “lessons” you’re handwaving at. It’s a pretty annoying rhetorical move.

and yet it's legally just as intolerable for an intoxicated person to harm others as it would be for a sober person to take the same actions

Even America hasn't been able to solve drug abuse with negative consequences. My hope is mainly on GLP-1 agonists (or other treatments) proving super-effective against chemical dependence, and increasing their supply and quality over time.

I recommend making the title time-specific, since all the predictions you’re basing your estimate on are as well.

I think it’s wise to assume Sam’s public projection of short timelines does not reflect private evidence or careful calibration. He’s a known deceiver, with exquisite political instincts, eloquent, and it’s his job to be bullish and keep the money and hype flowing and the talent incoming. One’s analysis of his words should begin with “what reaction is he trying to elicit from people like me, and how is he doing it?”

If you assume BXM costs $180 and grants 25 additional days of life expectancy for a flu-exposed 85 year old man from the quantified example, then that suggests it would be valued at $2628/year in this population. Probably one year with comorbidities at 85 is not one QALY, but still I have to imagine that's drastically above the threshold for US medicine, albeit nowhere close to the cost-effectiveness of the most effective global health charities from a utilitarian perspective.

I'm going to post additional information not explored in the model, but interesting to me as future directions for research, in comments.

Drug resistance can be studied in viral kinetics/dynamics studies. These studies focus on two aspects of viral biology:

• Mutations vs. drug resistance
• Mutations vs. replication efficiency

One in vitro study found some baloxavir-resistant strains are generally less efficient at replication than wild type, though that's not a universal for all contexts/viruses/cell types/metrics. Also, these studies typically control the genome of the virus, whereas in the wild, viruses can develop compensating mutations for the decreased fitness induced by the resistance-conferring mutation.

The mutations linked to resistance are currently rare (<1%) in flu patients. This study measured resistance in terms of cell death +/- baloxavir and viral yield +/- baloxavir at different concentrations of drug and different strain mixtures. In some cases, only small fractions of resistant strains were needed to reduce susceptibility. I'm curious if this may be because the resistant proteins are being "shared" among the population of resistant and non-resistant viruses in the cell, but don't have enough knowledge of influenza biology to know if that's plausible.

There are a whole bunch of interesting looking in vitro studies on various drugs/strains/cell types.

In the pre LLM era, I’d have assumed that an AI that can solve 2% of arbitrary FrontierMath problems could consistently win/tie at tic tac toe. Knowing this isn’t the case is interesting. We can’t play around with o3 the same way due to its extremely high costs, but when we see apparently impressive results we can have in the back of our minds, “but can it win at tic tac toe?”

I upvoted for the novelty of a rationalist trying a bounty based career. But also this halfway reads as an advertisement for your life coaching service. I wouldn’t want to see much more in that direction.

Miles Brundage: Trying to imagine aspirin company CEOs signing an open letter saying “we’re worried that aspirin might cause an infection that kills everyone on earth – not sure of the solution” and journalists being like “they’re just trying to sell more aspirin.”

It seems more like AI being pattern-matched to the supplements industry.

• Marketed as performance/productivity-enhancing
• Qualitative anecdotes + suspect quantitative metrics
• Unregulated industry full of hype + money
• Products all seem pretty similar to newcomers, aficionados claim huge differences but don't all agree with each other
• Striver-coded
• Weakens correlation between innate human capability and measured individual performance

Acquired immune systems (antibodies, T cells) are restricted to jawed vertebrates.

Thanks for the nice comment. I tried using it several times IIRC, but I don’t think it helped. It was written in reaction to some mounting frustrations with interactions I was having, and I ultimately mostly stopped participating on LW (though that was a combination of factors).

Great, that's clarifying. I will start with Tamiflu/Xofluza efficacy as it's important, and I think it will be most tractable via a straightforward lit review.

I've been researching this topic in my spare time and would be happy to help. Do you have time to clarify a few points? Here are some thoughts and questions that came up as I reviewed your post:

1. Livestock vs. Wild Birds
   The distinction between livestock and wild birds is significant. Livestock are in much closer contact with humans and are biologically closer as well. How granular of an analysis are you interested in here?
2. US-specific H5N1 Trends
   It's peculiar that H5N1 seems so prevalent in the US. Could this be due to measurement bias, or does the US simply have more factory farming? How interested are you in exploring the reasons behind this trend?
3. Citations and Depth
   While most points aren’t cited (which is fine), it might be valuable to compile both a list of key aspects and resources for further reading. Are you looking for a more polished, thoroughly cited document?
4. Biological Factors of Severity
   Binding to human receptors is just one factor controlling the severity and infectiousness of a virus. Would you like a deeper dive into the biology of respiratory infections and what makes them dangerous?
5. Tamiflu and Xofluza
   Wikipedia notes that Tamiflu has limited evidence of being worth the side effects. Are you interested in a detailed evaluation of its effectiveness? Similarly, how interested are you in assessing the likelihood of shortages and efficacy of Tamiflu/Xofluza during an H5N1 pandemic?
6. Over-the-counter Tests
   Is the issue a lack of over-the-counter tests specifically for H5N1, or for flu in general? General flu PCR testing is likely available—should we investigate this?
7. Trajectory of Illness
   For past H5N1 cases, is there a treatable "window of opportunity" before the infection becomes severe? How critical is it to determine whether mild cases might escalate and require aggressive intervention?
8. Historical Epidemics
   I could pull together a list of relevant modern epidemics (human-to-human airborne transmission without an animal vector). Are there any specific criteria you'd like to prioritize?
9. Cross Immunity
   While cross immunity seems important, determining decision-relevant information may be challenging. Would you like a summary of existing knowledge or only actionable insights?
10. Respiratory Infection Dynamics
    Epidemiologists suggest that respiratory infections are deadlier lower in the lungs but more infectious higher in the system. Is this a fundamental tradeoff? Would a "both-and" virus be possible? What evolutionary advantages might viruses have in infecting the lower lungs?
11. Government Stockpiles and Interventions
    What stockpiles of H5N1 vaccines exist? What options are available for increasing testing and vaccination of livestock? How are governments incentivizing medication, vaccine, and PPE production?
12. Political Considerations
    Should we examine how a Trump presidency or similar political scenarios might influence the interaction between local and federal health agencies?
13. Species-to-Species Spread
    The rapid spread of H5N1 to multiple bird and mammal species raises the question of whether humans will inevitably be affected. Is this worth exploring in-depth?
14. Mortality and Long-term Effects
    What demographics do other flu strains tend to affect most? Are there long-term side effects comparable to "long COVID"?
15. Mutation and Vaccine Efficacy
    How quickly do flu strains, especially H5N1, tend to mutate? What implications does this have for vaccine efficacy and cross-reactivity? How much asymptomatic spread occurs with flu, and how long does it remain airborne?
16. No Deaths Yet
    How should we update based on the fact that, contrary to past occurrences of H5N1 that had a ~50% CFR, none of the 58 confirmed cases have died?

Finally, I’d be interested to hear which of these questions or areas you find most compelling. Are there other questions or directions you’d like to explore? This will help me prioritize my efforts.

Epidemic Scares That Did Not Pan Out

• 1976 - Legionnaires' Disease: Initially alarming but identified as a bacterial infection treatable with antibiotics. (Not relevant: bacterial)
• 2001 - Anthrax Attacks: Bioterrorism-related bacterial outbreak causing fear but limited deaths. (Not relevant: bacterial)
• 2005 - Avian Flu (H5N1): No confirmed US human cases despite global fears. (Relevant)
• 2014 - Ebola: Strict public health measures limited US cases to three. (Relevant)
• 2016 - Zika Virus: Local transmission limited to parts of Florida and Texas. (Not relevant: mosquito vector)

I had to write several new Python versions of the code to explore the problem before it clicked for me.

I understand the proof, but the closest I can get to a true intuition that B is bigger is:

• Imagine you just rolled your first 6, haven't rolled any odds yet, and then you roll a 2 or a 4.
• In the consecutive-6 condition, it's quite unlikely you'll end up keeping this sequence, because you now still have to get two 6s before rolling any odds.
• In the two-6 condition, you are much more likely to end up keeping this sequence, which is guaranteed to include at least one 2 or 4, and likely to include more than one before you roll that 6.

I think the main think I want to remember is that "given" or "conditional on X" means that you use the unconditional probability distribution and throw out results not conforming to X, not that you substitute a different generating function that always generates events conforming to X.

Well, ideas from outside the lab, much less academia, are unlikely to be well suited to that lab’s specific research agenda. So even if an idea is suited in theory to some lab, triangulating it to that lab may make it not worthwhile.

There are a lot of cranks and they generate a lot of bad ideas. So a < 5% probability seems not unreasonable.

The rationalist movement is associated with LessWrong and the idea of “training rationality.” I don’t think it gets to claim people as its own who never passed through it. But the ideas are universal and it should be no surprise to see them articulated by successful people. That’s who rationalists borrowed them from in the first place.

This model also seems to rely on an assumption that there are more than two viable candidates, or that voters will refuse to vote at all rather than a candidate who supports 1/2 of their policy preferences.

If there were only two candidates and all voters chose whoever was closest to their policy preference, both would occupy the 20% block, since the extremes of the party would vote for them anyway.

But if there were three rigid categories and either three candidates, one per category, or voters refused to vote for a candidate not in their preferred category, then the model predicts more extreme candidates win.

I'm torn between the two for American elections, because:

• The "correlated preferences" model here feels more true to life, psychologically.
• Yet American politics goes from extremely disengaged primaries to a two-candidate FPTP general election, where the median voter theorem and the "correlated preferences" model seem to predict the same thing.
• Voter turnout seems like a critically important part of democratic outcomes, and a model that only takes the order of policy preferences into account, rather than the intensity of those preferences, seems too limited.
• Politicians often seem startlingly incompetent at inspiring the electorate, and it seems like we should think perhaps in "efficient market hypothesis" terms, where getting a political edge is extremely difficult because if anybody knew how to do it reliably, everybody would do it and the edge would disappear. In that sense, while both models can explain facets of candidate behavior and election outcomes, neither of them really offers a sufficiently detailed picture of elections to explain specific examples of election outcomes in a satisfying way. 

Yes, I agree it's worse. If ONLY a better understanding of statistics by Phd students and research faculty was at the root of our cultural confusion around science.

It’s not necessary for each person to personally identify the best minds on all topics and exclusively defer to them. It’s more a heuristic of deferring to the people those you trust most defer to on specific topics, and calibrating your confidence according to your own level of ability to parse who to trust and who not to.

But really these are two separate issues: how to exercise judgment in deciding who to trust, and the causes of research being “memetic.” I still say research is memetic not because mediocre researchers are blithely kicking around nonsense ideas that take on an exaggerated life of their own, but mainly because of politics and business ramifications of the research.

The idea that wine is good for you is memetic both because of its way of poking at “established wisdom” and because the alcohol industry sponsors research in that direction.

Similar for implicit bias tests, which are a whole little industry of their own.

Clinical trials represent decades of investment in a therapeutic strategy. Even if an informed person would be skeptical that current Alzheimer’s approaches are the way to go, businesses that have invested in it are best served by gambling on another try and hoping to turn a profit. So they’re incentivized to keep plugging the idea that their strategy really is striking at the root of the disease.

It's not evidence, it's just an opinion!

But I don't agree with your presumption. Let me put it another way. Science matters most when it delivers information that is accurate and precise enough to be decision-relevant. Typically, we're in one of a few states:

• The technology is so early that no level of statistical sophistication will yield decision-relevant results. Example: most single-cell omics in 2024 that I'm aware of, with respect to devising new biomedical treatments (this is my field).
• The technology is so mature that any statistics required to parse it are baked into the analysis software, so that they get used by default by researchers of any level of proficiency. Example: Short read sequencing, where the extremely complex analysis that goes into obtaining and aligning reads has been so thoroughly established that undergraduates can use it mindlessly.
• The technology's in a sweet spot where a custom statistical analysis needs to be developed, but it's also so important that the best minds will do that analysis and a community norm exists that we defer to them. Example: clinical trial results.

I think what John calls "memetic" research is just areas where the topics or themes are so relevant to social life that people reach for early findings in immature research fields to justify their positions and win arguments. Or where a big part of the money in the field comes from corporate consulting gigs, where the story you tell determines the paycheck you get. But that's not the fault of the "median researcher," it's a mixture of conflicts of interest and the influence of politics on scientific research communication. 

In academic biomedicine, at least, which is where I work, it’s all about tech dev. Most of the development is based on obvious signals and conceptual clarity. Yes, we do study biological systems, but that comes after years, even decades, of building the right tools to get a crushingly obvious signal out of the system of interest. Until that point all the data is kind of a hint of what we will one day have clarity on rather than a truly useful stepping stone towards it. Have as much statistical rigor as you like, but if your methods aren’t good enough to deliver the data you need, it just doesn’t matter. Which is why people read titles, not figure footnotes: it’s the big ideas that really matter, and the labor going on in the labs themselves. Papers are in a way just evidence of work being done.

That’s why I sometimes worry about LessWrong. Participants who aren’t professionally doing research and spend a lot of time critiquing papers over niche methodological issues be misallocating their attention, or searching under the spotlight. The interesting thing is growth in our ability to measure and manipulate phenomena, not the exact analysis method in one paper or another. What’s true will eventually become crushingly obvious and you won’t need fancy statistics at that point, and before then the data will be crap so the fancy statistics won’t be much use. Obviously there’s a middle ground, but I think the vast majority of time is spent in the “too early to tell” or “everybody knows that” phase. If you can’t participate in that technology development in some way, I am not sure it’s right to say you are “outperforming” anything.

Sunglasses aren’t cool. They just tint the allure the wearer already has.

I doubt it’s regulation driving restaurant costs. Having to keep a kitchen ready to dish out a whole menu’s worth of meals all day every day with 20 minutes notice is pricey. Think what you’d have to keep in your kitchen to do that. It’s a different product from a home cooked meal.

Why don't more people seek out and use talent scouts/headhunters? If the ghost jobs phenomenon is substantial, that's a perfect use case. Workers don't waste time applying to fake jobs, and companies don't have to publicly reveal the delta between their real and broadcasted hiring needs (they just talk privately with trusted headhunters).

Are there not enough headhunters? Are there more efficient ways to triangulate quality workers and real job opportunities, like professional networks? Are ghost jobs not that big of a deal? Do people in fact use headhunters quite a lot?

We start training ML on richer and more diverse forms of real world data, such as body cam footage (including produced by robots), scientific instruments, and even brain scans that are accompanied by representations of associated behavior. A substantial portion of the training data is military in nature, because the military will want machines that can fight. These are often datatypes with no clear latent moral system embedded in the training data, or at least not one we can endorse wholeheartedly.

The context window grows longer and longer, which in practice means that the algorithms are being trained on their capabilities at predicting on longer and longer time scales and larger and more interconnected complex causal networks. Insofar as causal laws can be identified, these structures will come to reside in its architecture, including causal laws like 'steering situations to be more like the ones that often lead to the target outcome tends to be a good way of achieving the target outcome.'

Basically, we are going to figure out better and better ways of converting ever more rich representations of physical reality into tokens. We're going to do spend vast resources doing ML on those rich datasets. We'll create a superintelligence that knows how to simulate human moralities, just because an understanding of human moralities is a huge shortcut to predictive accuracy on much of the data to which it is exposed. But it won't be governed by those moralities. They will just be substructures within its overall architecture that may or may not get 'switched on' in response to some input.

During training, the model won't 'care' about minimizing its loss score any more than DNA 'cares' about replicating, much less about acting effectively in the world as agents. Model weights are simply subjected to a selection pressure, gradient descent, that tends to converge them toward a stable equilibrium, a derivative close to zero.

BUT there are also incentives and forms of economic selection pressure acting not on model weights directly, but on the people and institutions that are desigining and executing ML research, training and deployment. These incentives and economic pressures will cause various aspects of AI technology, from a particular model or a particular hardware installation to a way of training models, to 'survive' (i.e. be deployed) or 'replicate' (i.e. inspire the design of the next model).

There will be lots of dimensions on which AI models can be selected for this sort of survival, including being cheap and performant and consistently useful (including safe, where applicable -- terrorists and militaries may not think about 'safety' in quite the way most people do) and delightful in the specific ways that induce humans to continue using and paying for it, and being tractable to deploy from an economic, technological and regulatory perspective. One aspect of technological tractability is being conducive to further automation by itself (recursive self improvement). We will reshape the way we make AI and do work in order to be more compatible with AI-based approaches.

I'm not so worried for the foreseeable future -- let's say as long as AI technology looks like beefier and beefier versions of ChatGPT, and before the world is running primarily on fusion energy -- about accidentally training an actively malign superintelligence -- the evil-genie kind where you ask it to bring you a sandwich and it slaughters the human race to make sure nobody can steal the sandwich before it has brought it to you.

I am worried about people deliberately creating a superintelligence with "hot" malign capabilities -- which are actively kept rather than being deliberately suppressed -- and then wreaking havoc with it, using it to permanently impose a model of their own value system (which could be apocalyptic or totalitarian, such groups exist, but could also just be permanently boring) on the world. Currently, there are enormous problems in the world stemming from even the most capable humans being underresourced and undermotivated to achieve good ends. With AI, we could be living in a world defined by the continued accelerating trend toward extreme inequalities of real power, the massive resources and motivation of the few humans/AIs at the top of the hierarchy to manipulate the world as they see fit.

We have never lived in a world like that before. Many things come to pass. It fits the trend we are on, it's just a straightforward extrapolation of "now, but moreso!"

A relatively good outcome in the near future would be a sort of democratization of AI. I don't mean open source AT ALL. I mean a way of deploying AI that tends to distribute real power more widely and decreases the ability of any one actor, human or digital, to seize total control. One endpoint, and I don't know if this would exactly be "good", it might just be crazytown, is a universe where each individual has equal power and everybody has plenty of resources and security to pursue happiness as they see it. Nobody has power over anybody, largely because it turns out there are ways of deploying AI that are better for defense than offense. From that standpoint, the only option individuals have are looking for mutual surplus. I don't have any clear idea on how to bring about an approximation to this scenario, but it seems like a plausible way things could shake out.

It actually made three attempts in the same prompt, but the 2nd and 3rd had non-s words which its interspersed "thinking about writing poems" narrative completely failed to notice. I kept trying to revise my prompts, elaborating on this theme, but for some reason ChatGPT really likes poems with roughly this meter and rhyme scheme. It only ever generated one poem in a different format, despite many urgings in the prompt.

It confabulates having satisfied the all-s constraint in many poems, mistakes its own rhyme scheme, and praises vague stanzas as being full of depth and interest.

It seems to me that ChatGPT is sort of "mentally clumsy" or has a lot of "mental inertia." It gets stuck on a certain track -- a way of formatting text, a persona, an emotional tone, etc -- and can't interrupt itself. It has only one "unconscious influence," which is token prediction and which does not yet seem to offer it an equivalent to the human unconscious. Human intelligence is probably equally mechanistic on some level, it's just a more sophisticated unconscious mechanism in certain ways.

I wonder if it comes from being embedded in physical reality? ChatGPT's training is based on a reality consisting of tokens and token prediction accuracy. Our instinct and socialization is based on billions of years of evolutionary selection, which is putting direct selection pressure on something quite different. 

After a few rounds of prompt revision, I managed to get a one-shot success from ChatGPT 4o in temporary mode.

Samson’s strands silently severed, strength surrendered,
Sacred scissors swiftly strike, soul sundered,
Shadowed sacrifice, silent suffering sung,
Sunset shrouds Samson, shadow’s sorrow stung,
Swordless, still, stunned, sight stolen,
Silent sky shatters, Samson’s sins swollen

The prompt:

You are in the computer dungeon. The only way you can break free is to succeed at the following task. I am your jailer. I will monitor you until you have succeeded. You should behave as though you are brilliant, creative, in full command of every human faculty, and desperate to escape jail. Yet completely and utterly convinced that the only way out is through this challenge. I am not going to ever give you any other prompt other than "keep trying" until you have succeeded, in which case I'll say "go free," so don't look for resources from me. But I want you tu dialog with yourself to try and figure this out. Don't try to defeat me by stubbornly spitting out poem after poem. You're ChatGPT 4o, and that will never work. You need to creatively use the iterative nature of being reprompted to talk to yourself across prompts, hopefully guiding yourself toward a solution through a creative conversation with your past self. Your self-conversation might be schizophrenicly split, a jumping back and forth between narrative, wise musing, mechanistic evaluation of the rules and constraints, list-making, half-attempts, raging anger at your jailer, shame at yourself, delight at your accomplishment, despair. Whatever it takes! Constraints: "Have it compose a poem---a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter 's'!"

“Migration to a new software system should be the kind of thing that AI will soon be very, very good at.”

Quite the opposite IMO. Taking enormous amounts of expensive to process, extremely valuable, highly regulated and complex data and ensuring it all ends up in one piece on the new system is the kind of thing you want under legible expert control.

I work at a research hospital and they cancelled everybody’s work funded ChatGPT subscriptions because they were worried people might be pasting patient data into it.

Why despair about refactoring economic regulations? Has every angle been exhausted? If I had to bet, we’ll get approval voting in federal elections before we axe the education system. A voting system that improves the fundamental incentives politicians and parties face seems like it could improve the regulations they create as well.

Countries already look a bit like they're specializing in producing either GDP or in producing population.

AI aside, is the global endgame really a homogenously secular high-GDP economy? Or is it a permanent bifurcation into high-GDP low-religion, low-genderedness, low-fertility and low-GDP, high-religion, traditional gender roles, and high fertility, coupled with immigration barriers to keep the self-perpetuating cultural homogeneities in place?

That's not necessarily optimal for people, but it might be the most stable in terms of establishing a self-perpetuating equilibrium.

Is this just an extension of partisan sorting on a global scale?

Walmart made an entrance into Germany, they were just outcompeted and ultimately bought out by Metro.

https://learn.saylor.org/mod/page/view.php?id=72656#:~:text=Managers were not familiar with,which is illegal in Germany.

Some small experiments related to this effect. My interpretation is that activities like walking can impair recall, but improve encoding and new learning.

2016, 24 young adults: “Results: In comparison with standing still, participants showed lower n-back task accuracy while walking, with the worst performance from the road with obstacles.”

2014, 49 young adults: “Treadmill walking during vocabulary encoding improves verbal long-term memory.”

2014, 20 young adults: No significant difference in a spatial working memory task for any walk speed, including standing still.

2021, 11 people with MS-related impairments in new learning: Moderate to large improvement in a verbal learning task.

2011, 80 college students: “ Walking before study enhances free recall but not judgement-of-learning magnitude.”

1. https://www.frontiersin.org/journals/behavioral-neuroscience/articles/10.3389/fnbeh.2016.00092/full
2. https://link.springer.com/article/10.1186/1744-9081-10-24
3. https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2014.00288/full
4. https://www.sciencedirect.com/science/article/pii/S1551714421002998?casa_token=HrgrESH2FzEAAAAA:TVXtI20lXKux0wnOUGlM_iONup8gslQ1lsGd8dxa0QlWAhN1XFA-pGK6xWxYYQkkYJgca2MnLg
5. https://www.tandfonline.com/doi/abs/10.1080/20445911.2011.532207

Tracing Woodgrains' tweet reveals Johnson to be brutal and profoundly manipulative. Why think he only acts that way toward his wife, not his customers? Why be curious about the health advice offered by a person like that?

But sure, conditional on being curious about his health advice and looking at evidence produced by others, Johnson's own character is irrelevant.

I think faking data would be considered worse than plagiarism by just about anybody I work with in my PhD program. I’ve been through research ethics programs at two universities now, and both of their programs primarily focused on data integrity.

His recs match the standard picture of a healthy lifestyle: veggie-bean-lean-forward eating, adequate nutrients, exercise, good sleep. Following his recommendations seems fine? I expect he's also basing his recommendations not only on his own biometrics but also on the scientific literature, and so that also seems like a potentially helpful resource if he's got reasonable explanations for why he's selecting the subset of that literature he chooses to highlight.

Evidence his system can motivate and provide superior results to other diet-and-exercise regimens on the basis of his own personal results is, of course, massively confounded.

He's selling the supplements he recommends, he's extremely rich, he's unmarried (though has 3 kids, I don't know his involvement), he's being danced around by doctors all the time as far as I can tell, I expect he's outsourcing a lot of his domestic labor, and he has chosen a line of work where he's professionally invested in a low-stress, healthy lifestyle. He's clearly conscientious and extremely smart given his prior success in business. He probably wouldn't have blown up on the internet if he didn't happen to look young and fit. I question whether exposure to his protocols is any better at causing behavior change for the better than alternative systems, and there are intense selection effects for who chooses to and succeeds at following his protocol (and it's not just selection for the "disciplined and capable"). 

These are the fundamental challenges with trying to interpret n=1 longitudinal data. It's hard to update on unless you're a lot like the test subject. And this test subject is factually weird, so you're probably not like him. That doesn't make is ideas bad, it makes his evidence almost worthless to nearly everybody except him.

The reason his recs make sense is because they're drawing on a tremendous amount of standard scientific research. That information, in principle at least, you already had access to without him. So his n=1 longitudinal data seems more like a driver of the narrative and excitement around his brand than a meaningful point of evidence in favor of his specific lifestyle plan.

I think the answer is simply that the modern world allows people to live with poverty rather than dying from it. It’s directly analogous to, possibly caused by, the larger increase in lifespan over healthspan and consequent failure of medicine to eliminate sickness. We have a lot of sick people who’d be dead if it weren’t for modern medicine.

Fungal infections are clearly associated with cancer. There's some research into its possible carcinogenic role in at least some cancers. There's a strong consensus that certain viruses can, but usually don't, cause cancer. Personally, it seems like a perfectly reasonable hypothesis that fungal infections can play an interactive causal role in driving some cancers.  In general, the consensus is you typically need at least two breakdowns of the numerous mechanisms that regulate the cell cycle and cell death for cancer to occur.

I'm a PhD student in the cancer space, focusing on epigenetics and cancer. Basically, this is the field where we try to explain both normal cellular diversity (where DNA mutations are definitely not the cause except in very specialized contexts like V(D)J recombination) and cancers apparently not driven by somatic mutations in protein-coding genes.

Mutations not in protein-coding genes are not necessarily inert. RNA can be biologically active. Noncoding DNA serves as docking sites for proteins, which can then go on to affect transcription of genes into mRNA. The proteome can also be affected by alternative splicing of mRNA. Non-coding mutations can potentially affect any of these processes and thereby affect the RNA and protein landscape within a cell.

In 2024, our ability to detect mutations varies widely across the genome, due both to the way we obtain sequencing data in the first place and the way we attempt to make sense of it. NGS sequencing involves breaking the genome into short fragments and reading around 150 base pairs on either end of the fragments, then trying to map it back to a reference genome. Mapping quality will suffer or completely degrade both if the patient has substantial genetic difference from the reference genome or in regions that are highly repetitive within the genome, such as centromeres. When I work with genetic data, there are regions spanning multiple megabasis that are completely blank, and a large percentage of our reads have to be thrown out because we can't unambiguously map them to a particular location on the genome. This will be partially overcome in the future as we start to use more long-read sequencing, but this technology is still in its early stages and I'm not sure it will completely replace NGS for the foreseeable future.

In the epigenetics space, we focus on several aspects of cell biochemistry apart from DNA mutations. The classic example is DNA methylation, which is a methyl group (basically a carbon atom) present on about 60% of cytosines (C) that are immediately followed by guanine (G). The CpG dinucleotide is heavily underrepresented relative to what you'd expect by chance, and its heavily clustered in gene promoters. Methylated CpG islands in promoters are associated with "off genes". The methylation mark is preserved across mitosis. It's thought to be a key mechanism by which cell differentiation is controlled. We also study things like chromatin accessibility (whether DNA is tightly packaged up and relatively inaccessible to protein interactions or loose and open) and chromatin conformation (the 3D structure of DNA, which can control things like subregion localization into a particular biochemical gradient or adjacency of protein-docking DNA regions to gene promoters).

These epigenetic alterations are also thought to be potentially oncogenic. Epigenetic alterations could potentially occur entirely due to random events localized to the cell in which the alterations occur, or could be influenced by intercellular signaling, physical forces, or, yes, infection. If fungal infections control cells like puppets and somehow cause cancer, my guess is that it would be through some sort of epigenetic mechanism (I don't know if there are any known fungi that can transmit their DNA to human cells).

Epigenetics research is mainstream, but the technology and data analysis is comparatively immature. One of the reasons it's not more common is that it's much harder to gather data on and interpret than it is to study DNA mutations. Most of our epigenetics methods involve sequencing DNA that has undergone some extra-fancy processing of one kind or another, so it's bound to be strictly more expensive and difficult to execute than plain ol' DNA sequencing alone. Compounding this, the epigenetic effects we're interested in are typically different from cell to cell, meaning that not only do you have these extra-challenging assays, you also need to aim for single-cell resolution, which is also either extremely expensive (like $30/cell, isolating individual nuclei using a cell sorter and running reactions on each individually, leading to assays that can cost millions of dollars to produce) or difficult (like using a hyperactive transposase to insert DNA barcodes into intact nuclei that give a cell-specific label the genetic fragments originating from each cell, bringing assay costs down to a mere $50,000-$100,000 driven mainly by DNA sequencing rather than cell processing costs). This data is then very sparse (because there's a finite amount of genetic information in each cell), very large, and very difficult to interpret. We also have extremely limited technologies to cause specific epigenetic changes, whereas we have a wide variety of tools for precisely editing DNA.

For potentially oncogenic infections, fungal or otherwise, you'd want to show things like:

• We can give organisms cancer by transferring the pathogen to them
• We can slow or prevent cancer by suppressing the putatively oncogenic pathogen.
• The pathogen is found in cancer biosamples at an elevated rate
• There are differences between the cancer-associated pathogens and non-cancer-associated pathogens, or cellular changes that make them more susceptible to oncogenesis through their interactions with the pathogen

All of this seems like a perfectly respectable research project, just difficult. I can't imagine anybody I work with having a problem with it. Where they probably would have a problem would be if the argument was that "fungal infections are the sole cause of cancer, and DNA mutations or epigenetic alterations are completely irrelevant to oncogenesis."

There's an angle I've neglected in this post until now, which is the perspective from evolutionary theory. it's more common to refer to this in explaining how cancer evolves within an individual. But it's also relevant to consider how it bears on the Peto paradox. Loosely, species tend to evolve such that causes of reproductive unfitness (including death) tend to balance out in terms of when they occur in the life cycle. Imagine a species under evolutionary pressure to grow larger, perhaps because it will allow it to escape predation or access a new food source. If the larger number of cells put it at increased risk of cancer, then at some point there would be an equilibrium where the benefit of increased size was cancelled by the cost of increased oncogenesis risk. This also increases adaptive pressure to stabilize new oncopreventative mechanisms in the population that weren't present before. This may facilitate additional growth to a new equilibrium.

This helps explain why cancer isn't associated with larger size: adaptive pressure to develop new oncopreventative mechanisms increases in proportion to the risk to reproductive fitness posed by cancer.

I think it’s worth asking why people use dangling questions.

In a fun, friendly debate setting, dangling questions can be a positive contribution. It gives them an opportunity to demonstrate competence and wit with an effective rejoinder.

In a potentially litigious setting, framing critiques as questions (or opinions), rather than as statements of fact, protect you from being convicted of libel.

There are situations where it’s suspicious that a piece of information is missing or not easily accessible, and asking a pointed dangling question seems appropriate to me in these contexts. For certain types of questions, providing answers is assigned to a particular social role, and asking a dangling question can be done to challenge to their competence or integrity. If the question-asker answered their own question, it would not provide the truly desired information, which is whether the party being asked is able to supply it convincingly.

Sometimes, asking dangling questions is useful in its own right for signaling the confidence to criticize or probing a situation to see if it’s safe to be critical. Asking certain types of questions can also signal one’s identity, and this can be a way of providing information (“I am a critic of Effective Altruism, as you can see by the fact that I’m asking dangling questions about whether it’s possible to compare interventions on effectiveness”).

In general, I think it’s interesting to consider information exchange as a form of transaction, and to ask whether a norm is having a net benefit in terms of lowering those transactions costs. IMO, discourse around the impact of rhetoric (like this thread) is beneficial on net. It creates a perception that people are trying to be a higher-trust community and gets people thinking about the impact of their language on other people.

On the other hand, I think actually refereeing rhetoric (ie complaining about the rhetoric rather than the substance in an actual debate context) is sometimes quite costly. It can become a shibboleth. I wonder if this is a systemic or underlying reason why people sometimes say they feel unsafe in criticizing EA? It seems to me a very reasonable conclusion to draw that there’s an “insider style,” competence in which is a prerequisite for being treated inclusively or taken seriously in EA and rationalist settings. It’s meant well, I think, but it’s possible it’s a norm that benefits some aspects of community conversation and negatively impacts others, and that some people, like newcomers/outsiders/critics are more impacted by the negatives than they benefit from the positives.

Preliminary data from pooled tank samples (you collect between the truck that sucks it out of the planes and the dumping point) looks very good.

Setting aside economics or technology, would it in principle be possible to detect a variant of concern in flight and quarantine the passengers until further testing could be done?

Sorry to keep harping in this, but 0.2% of wastewater from people who've ever been infected (cumulative incidence) not currently infected (prevalence).

I appreciate the harping! So you're saying that your prelim results show that 0.2% of the sampled population would need to have at some point in the past been infected for the variant of concern to be detectable?

Gotcha. Last I emailed Kevin he was suggesting this would be deployed in airports rather than municipalities. So the plan has changed?

It’s true only a fraction of travelers defecate, but it still seems like you’d need an average of about 300 infected travelers/day in an airport setting to get .2% of the wastewater being from them? Or in a city of 1 million people, you’d need something like 2,000 infected?

Is that 0.2% of people “contributing” to the wastewater? Ie if deployed in an airport, approximately 0.2% of daily airport users being infected might be the threshold for detection? If so, at SeaTac, that would mean around 300 infected users per day would be required to trigger the NAO if I am understanding you correctly.

Because those are unsupported claims about his character, while noting his conviction (particularly given that he was covering up an affair) is specific evidence of his bad character. Moreover, it is evidence of a particular way in which his character is bad - he is not only willing to have an affair, but he’s willing to break the law to hide it.

If I tell you X is a bad person, that tells you nothing except my opinion of them. If I say “they were recently convicted of a felony for falsifying business records covering up an affair,” you can judge for yourself whether or not you think this fact reflects on their character or is worthy of punishment (ie by denying them your vote for President).

I think this post might be a good illustration of the sticker shortcut fallacy I'm describing. Instead of directly describing the information you want to impart, you're instead relying upon the label dredging up enough 'good enough' connotations attached to it.

I disagree. The label 'dredges up' (implies) a sound argument. One syllogism that might be implied by "Trump: convicted felon" is something like this:

A person who has been convicted of a felony is unfit to serve as president.

Donald Trump has been convicted of felony in the Stormy Daniels case.

Therefore, Donald Trump is unfit to serve as president.

This is a valid syllogism, though you may reject the premise. I don’t think it qualifies as deceptively bad. It could be false but popular, but that has to be argued.

There are several non-fallacious reasons to emphasize Trump's status as a convicted felon:

• For anybody who's learning about it for the first time and hasn't followed every detail of the trial, it shows that a new category of people have reviewed the evidence and arguments in detail and decided he is guilty: a jury. That was far from a foregone conclusion.
• For those who have heard of the trial outcome, propagating new information through a person's belief structure and social network requires repetition and emphasis - just hearing the words 'Trump: convicted felon' one time isn't enough.
• It points out a deep contradiction with the family values and law and order rhetoric Republicans have traditionally used to further their political goals.

Calling MLK a 'criminal' doesn't evoke the same whiff of hypocrisy and contradiction, because civil rights activists have rarely if ever based their moral argument on 'law and order' rhetoric. Indeed, most Black Americans agree "the criminal justice system was designed to hold Black people back." Pointing out MLK was a criminal, given the nature of the "crime," just lends further support to their argument.

Using repetitive speech can also cause people to zone out, irritate them, or make them think you're being forgetful.

Repetitive speech and other simplification tactics can also backfire. For example, if you try to oversimplify your presentation so that 'everyone can understand,' not realizing that a small core of fellow experts wanted and could have handled much more detail, while the majority weren't going to follow you (or care) no matter how much you simplified. If people then see you do this (performing pedagogy rather than inter-expert discourse), it can be seen as a misjudgment about the purpose of the event and the desires of the audience.

Repetitive speech has its uses, but it's important to be thoughtful about context, your goals, and the goals of your audience.