Posts
Comments
My guess is that ASI will be faster to adapt to novel weapons and military strategies. Nanotech is likely to speed up the rate at which new weapons are designed and fabricated.
Imagine a world in which a rogue AI can replicate a billion drones, of a somewhat novel design, in a week or so. Existing human institutions aren't likely to adapt fast enough to react competently to that.
I just published a post on Drexler's MSEP software that is relevant to whether people should donate to his project.
two more organizations that seem worthy of consideration
Investing in Eon Systems looks much more promising than donating to Carbon Copies.
I see maybe a 3% chance that they'll succeed at WBE soon enough to provide help with AI x-risk.
The Invention of Lying provides a mostly accurate portrayal of a world where everyone is honest. It feels fairly Hansonian.
No, I don't recall any ethical concerns. Just basic concerns such as the difficulty of finding a boss that I'm comfortable with, having control over my hours, etc.
Oura also has heart rate and VO2 max tracking. Does anyone know of problems with Oura's data?
The primary motive for funding NASA was definitely related to competing with the USSR, but I doubt that it was heavily focused on military applications. It was more along the lines of demonstrating the general superiority of the US system, in order to get neutral countries to side with us because we were on track to win the cold war.
Manifold estimates an 81% chance of ASI by 2036, using a definition that looks fairly weak and subjective to me.
I've bid the brain emulation market back up a bit.
Brain emulation looks closer than your summary table indicates.
Manifold estimates a 48% chance by 2039.
Eon Systems is hiring for work on brain emulation.
We can only value lives at $10 million when we have limited opportunities to make that trade, or we’d go bankrupt.
I'm suspicious of the implication that we have many such opportunities. But a quick check suggests says it's very dependent on guesses as to how many lives are saved be treatments.
I did a crude check for lives saved by cancer treatments. Optimistic estimates suggest that lives are being saved at less than $1 million per life. Robin Hanson's writings have implied that the average medical treatments is orders of magnitude less effective than that.
Could last year's revamping of OpenAI's board have been influenced by government pressure to accept some government-approved board members? Nakasone's appointment is looking more interesting after reading this post.
Soaking seeds overnight seems to be a good way to reduce phytic acid.
oral probiotics in general might just all be temporary.
The solution to concerns about it being temporary is to take them daily. I take Seed Daily Synbiotic. My gut is probably better as a result, but I don't have evidence that is at all rigorous.
The beginning of this comment is how Lintern expands on that claim. But it sounds like you have an objection that isn't well addressed there.
If cancer merely involved one bad feature, I could imagine software analogies that involved a large variety of mistakes producing that one bad feature.
The hallmarks of cancer indicate that all cancers have a number of bad features in common that look sufficiently unrelated to each other that it seems hard to imagine large sets of unrelated mutations all producing those same hallmarks. Lintern lists many other features that could be considered additional hallmarks.
When I try to imagine software problems that seem analogous to cancer, I come up with problems such as spam where there's an optimizer that's generating the problems.
I'm unclear whether you're imagining software problems that I haven't thought of, or whether you're modeling cancer differently from me.
Maybe? It doesn't seem very common for infectious diseases to remain in one area. It depends a lot on how they are transmitted. It's also not unusual for a non-infectious disease to have significant geographical patterns. There are cancers which are concentrated in particular areas, but there seem to be guesses for those patterns that don't depend on fungal infections.
Thanks. You've convinced me that Lintern overstates the evidence of mutation-free cancer cells.
But you seem to have missed really obvious consequences of the fungi theory, like, "wouldn't it be infectious then",
I very much did not miss that.
containing some potentially pretty dangerous advice like "don't do chemotherapy".
Where did I say that?
Enough that it should have been noticed.
My guess is that almost nobody looks for this kind of connection.
Even if they do notice it, they likely conclude that pathogens are just another small influence on cancer risk.
Because radiation cannot spread a fungus
Anything that causes cell damage and inflammation has effects that sometimes make cells more vulnerable to pathogens.
How would transmission be detected? It probably takes years before a tumor grows big enough for normal methods to detect it.
I assume that transmission is common, mild infections are common, and they rarely become harmful tumors.
Here are some conversations that I had with AIs, mainly Claude 3.5 Sonnet:
Perplexity, in response to "arguments against Mark Lintern's theory that fungal infections are the primary cause of cancer?":
- Effectiveness of current treatments: Many existing cancer treatments that do not specifically target fungal infections have shown efficacy in treating various types of cancer, which may contradict the theory that fungi are the primary cause [1].
That stands out as strange. The "effectiveness" of current cancer treatments is what led me to search for what is wrong with cancer research.
me: What are the strongest arguments against Mark Lintern's claim that fungal infections are the primary cause of cancer?
Claude:
Mark Lintern's claim that fungal infections are the primary cause of cancer is not widely accepted in the scientific community. While it's important to consider alternative hypotheses, there are several strong arguments against this claim:
Lack of robust scientific evidence: - There is a scarcity of peer-reviewed, well-designed studies supporting the fungal infection hypothesis as a primary cause of cancer. - The vast majority of cancer research points to genetic mutations, environmental factors, and other well-established risk factors.
Inconsistency with epidemiological data: - Cancer incidence patterns and risk factors observed in large-scale population studies don't align well with patterns of fungal infections. - Known risk factors like smoking, UV radiation, and certain chemicals have much stronger correlations with cancer incidence.
Cancer genomics: - Extensive genomic studies, such as The Cancer Genome Atlas, have identified numerous genetic mutations and alterations in cancer cells that are not explained by fungal infections. - These genetic changes are often directly linked to cancer-related processes.
Mechanism of action: - While some fungi can produce carcinogenic compounds, there's no clear mechanism by which fungi could consistently cause the specific genetic alterations observed in various types of cancer.
Immunocompromised patients: - Patients with compromised immune systems are highly susceptible to fungal infections. If fungi were the primary cause of cancer, we would expect to see much higher cancer rates in these populations, which is not the case.
Treatment efficacy: - Cancer treatments targeting cellular mechanisms and genetic alterations have shown significant efficacy, which would be unlikely if fungi were the primary cause.
Experimental evidence: - Laboratory studies inducing cancer in animal models through genetic alterations or carcinogen exposure don't require fungal involvement.
Tissue culture studies: - Cancer cells can be grown and studied in sterile laboratory conditions without any fungal presence, demonstrating that fungi are not necessary for cancer cell survival or proliferation.
Hereditary cancers: - The existence of hereditary cancer syndromes, where specific genetic mutations dramatically increase cancer risk, is difficult to explain with a fungal infection hypothesis.
Cellular and molecular biology: - Our understanding of cancer at the cellular and molecular level, including oncogene activation and tumor suppressor inactivation, doesn't require fungal involvement to explain cancer development.
Inconsistency across cancer types: - Different cancers have distinct molecular profiles and risk factors. A single cause (fungi) is unlikely to explain this diversity.
Successful prevention strategies: - Many cancer prevention strategies (e.g., smoking cessation, sun protection) are effective without addressing fungal infections.
Lack of fungal signatures: - Sophisticated genomic analyses of tumors have not revealed consistent fungal genetic material or signatures across cancer types.
Evolutionary perspective: - Cancer is observed across many animal species, including those with very different susceptibilities to fungal infections than humans.
Clinical observations: - Clinicians and pathologists routinely examine cancer tissues and rarely report finding evidence of widespread fungal involvement.
While it's true that some fungi can produce carcinogenic compounds and that certain fungal infections (like chronic candidiasis) have been associated with slightly increased risk of specific cancers, the evidence overwhelmingly suggests that fungi are not the primary cause of most cancers. The scientific consensus supports a multifactorial model of cancer causation, involving genetic, environmental, and lifestyle factors, with somatic mutations playing a central role in most cases.
It's important to approach such claims critically and rely on well-established scientific evidence and consensus when considering cancer causes and treatments.
Most of Claude's points seem weak, often assuming a rather controversial belief that fungi are reliably detected when they're present. The point that looks strongest to me is the one about Immunocompromised patients.
me: What's the best source of evidence for cancer rates in immunocompromised patients?
Claude:
To find the best evidence for cancer rates in immunocompromised patients, we should look at large-scale epidemiological studies, systematic reviews, and meta-analyses. Some of the most reliable sources include:
- HIV/AIDS patient cohorts: The HIV/AIDS Cancer Match Study, conducted by the National Cancer Institute, is one of the most comprehensive sources. It links HIV/AIDS and cancer registry data in the United States.
Key resource: Engels, E. A., et al. (2008). Trends in cancer risk among people with AIDS in the United States 1980--2002. AIDS, 22(16), 2101-2108.
- Transplant recipient studies: The Transplant Cancer Match Study, also by the National Cancer Institute, is an authoritative source for cancer incidence in organ transplant recipients.
Key resource: Engels, E. A., et al. (2011). Spectrum of cancer risk among US solid organ transplant recipients. JAMA, 306(17), 1891-1901.
- Systematic reviews and meta-analyses: These provide comprehensive overviews of multiple studies.
Example: Grulich, A. E., et al. (2007). Incidence of cancers in people with HIV/AIDS compared with immunosuppressed transplant recipients: a meta-analysis. The Lancet, 370(9581), 59-67.
- Large national databases: Countries with comprehensive health registries, like the Nordic countries, often provide valuable data.
Example: Vajdic, C. M., & van Leeuwen, M. T. (2009). Cancer incidence and risk factors after solid organ transplantation. International Journal of Cancer, 125(8), 1747-1754.
Collaborative international studies: The International Immunocompromised Host Society (ICHS) often publishes comprehensive data.
Specific immunodeficiency disorder studies: For rare immunodeficiency disorders, studies like the USIDNET (United States Immunodeficiency Network) can be valuable.
World Health Organization (WHO) reports: The WHO occasionally publishes comprehensive reports on cancer in immunocompromised populations.
Centers for Disease Control and Prevention (CDC): The CDC maintains databases and publishes reports on cancer incidence in HIV-infected individuals.
When evaluating these sources, consider: - Sample size - Duration of follow-up - Methodology for cancer detection and classification - Adjustment for confounding factors - Recency of the data
It's important to note that cancer rates can vary depending on the specific cause and nature of immunosuppression, so looking at multiple sources covering different immunocompromised populations can provide a more comprehensive picture.
me: What does the Somatic Mutation Theory of cancer predict about the kinds of mutations seen in the DNA of cancer cells? What kind of evidence concerning those mutations would falsify the theory?
Claude:
The Somatic Mutation Theory (SMT) of cancer makes several predictions about the kinds of mutations seen in the DNA of cancer cells. Here are the key predictions and the types of evidence that could potentially falsify the theory:
Predictions of the Somatic Mutation Theory:
Accumulation of mutations: SMT predicts that cancer cells will have accumulated multiple mutations over time.
Driver mutations: Some mutations (called driver mutations) will be causally implicated in cancer development, affecting genes that control cell growth, division, or death.
Passenger mutations: Other mutations (called passenger mutations) will be present but not directly contributing to cancer development.
Clonal expansion: Cells with advantageous mutations will undergo clonal expansion, leading to a population of cancer cells with shared mutations.
Heterogeneity: Different areas of a tumor may have different sets of mutations due to ongoing mutation and selection processes.
Specific gene types: Mutations will often affect proto-oncogenes (promoting cell growth) and tumor suppressor genes (inhibiting cell growth).
Functional changes: Mutations will lead to functional changes in proteins that drive cancer-related processes (e.g., increased proliferation, evading apoptosis).
Evidence that could potentially falsify the theory:
Lack of mutations: If cancer cells consistently showed no increase in mutations compared to normal cells, this would challenge SMT.
Random distribution: If mutations in cancer cells were completely random and showed no enrichment in cancer-related genes, this would be contrary to SMT predictions.
Reversibility without genetic changes: If advanced cancers could be consistently and permanently reversed to normal cells without altering their genetic material, this would challenge SMT.
Lack of clonality: If cancer cells within a tumor showed no evidence of clonal relationships, this would be inconsistent with SMT.
Identical mutations across cancer types: If all cancers, regardless of tissue origin or type, always had the exact same set of mutations, this would be difficult to explain with SMT.
Non-genetic transmission: If cancer could be reliably transmitted between individuals or cells without any transfer of genetic material or induction of new mutations, this would challenge SMT.
Perfect correlation with environmental factors: If cancer incidence was perfectly explained by environmental factors alone, with no role for random mutations or genetic susceptibility, this would be inconsistent with SMT.
Lack of genomic instability: If cancer cells showed no signs of genomic instability or increased mutation rates compared to normal cells, this would be contrary to many versions of SMT.
Mutations without consequences: If the identified mutations in cancer cells could be shown to have no functional consequences at all, this would challenge a key aspect of SMT.
It's important to note that while these findings would challenge SMT, they would need to be consistently replicated and thoroughly verified before the theory would be considered falsified. Many of these potential falsifying observations have not been seen; instead, the accumulating evidence has generally supported the main tenets of the Somatic Mutation Theory, though it has also led to refinements and expansions of the theory over time.
me: What does the Cancer Genome Atlas say about whether mutations in cancer cells tend to be in cancer-related genes?
Claude:
The Cancer Genome Atlas (TCGA) has provided extensive data on the genomic characteristics of various cancer types, offering significant insights into the patterns of mutations in cancer cells. Regarding your specific question about whether mutations in cancer cells tend to be in cancer-related genes, TCGA data strongly supports this notion. Here's a summary of key findings:
- Enrichment in cancer-related genes: TCGA data shows that mutations in cancer cells are indeed enriched in cancer-related genes. These include:
- Oncogenes (e.g., KRAS, BRAF) - Tumor suppressor genes (e.g., TP53, PTEN) - DNA repair genes (e.g., BRCA1, BRCA2) - Cell cycle regulators (e.g., RB1, CDKN2A)
Recurrent mutations: Certain genes are found to be recurrently mutated across multiple cancer types, suggesting their importance in cancer development. For example, TP53 is mutated in a high percentage of various cancers.
Cancer-type specific patterns: While some genes are commonly mutated across cancer types, TCGA has also revealed cancer-type specific mutation patterns. For instance: - BRAF mutations are common in melanoma
- APC mutations are frequent in colorectal cancer - EGFR mutations are often seen in certain lung cancers
Pathway-level alterations: TCGA data shows that even when specific genes are not mutated, the pathways they belong to are often altered. This supports the idea that cancer-related processes are being targeted by mutations.
Driver vs. passenger mutations: TCGA analyses have helped distinguish between driver mutations (in cancer-related genes) and passenger mutations. Driver mutations are less common but more impactful, often occurring in cancer-related genes.
Mutational signatures: TCGA has revealed mutational signatures that are associated with specific cancer types or carcinogenic exposures, many of which affect cancer-related genes more frequently.
Fusion genes: TCGA has identified numerous fusion genes in various cancers, often involving known cancer-related genes, further supporting the concentration of impactful mutations in cancer-relevant genomic regions.
Copy number alterations: Beyond point mutations, TCGA data shows that copy number alterations (amplifications and deletions) also frequently affect cancer-related genes.
Epigenetic alterations: While not mutations per se, TCGA has shown that epigenetic changes (like DNA methylation) often affect the expression of cancer-related genes.
Mutual exclusivity: TCGA data has revealed patterns of mutual exclusivity among mutations in certain cancer-related genes, suggesting that altering one component of a pathway is often sufficient for cancer development.
It's important to note that while TCGA data strongly supports the concentration of mutations in cancer-related genes, it also reveals the complexity of cancer genomics:
- Not all mutations in cancer cells are in known cancer-related genes. - The functional impact of many mutations is still not fully understood. - There's significant heterogeneity both within and between cancer types. - Some cancers show a high mutational burden across the genome, while others have fewer, more specific alterations.
In conclusion, TCGA data provides robust evidence that mutations in cancer cells tend to be enriched in cancer-related genes, supporting key aspects of the Somatic Mutation Theory of cancer. However, it also highlights the complexity of cancer genomics and the need for continued research to fully understand the role of various genetic alterations in cancer development and progression.
me: I've seen reports (via Mark Lintern) that a number of cancers involve no mutations. How do supporters of the Somatic Mutation Theory respond to such claims?
Claude:
The claim that some cancers involve no mutations is indeed a challenge to the Somatic Mutation Theory (SMT) of cancer. This topic has been a subject of debate in the cancer research community. Here's how supporters of SMT typically respond to such claims:
Questioning the methodology: - SMT supporters often argue that the studies claiming "mutation-free" cancers may have limitations in their detection methods. - They suggest that more sensitive techniques or broader genomic analyses might reveal mutations that were initially missed.
Epigenetic alterations: - Some SMT proponents argue that while there might not be detectable DNA sequence mutations, epigenetic changes (which can alter gene expression without changing the DNA sequence) could be present. - They consider epigenetic alterations as an extension of the mutation concept, still fitting within a broader interpretation of SMT.
Inherited genetic factors: - In some cases, cancer might develop due to inherited genetic factors rather than new somatic mutations. SMT supporters argue this still fits within the genetic basis of cancer.
Chromosomal abnormalities: - Some cancers might have large-scale chromosomal abnormalities (like aneuploidy) rather than point mutations. These are still genetic changes consistent with SMT.
Viral oncogenesis: - In cancers caused by viruses, the viral genes might drive cancer development without new mutations in the host genome. SMT supporters argue this still involves genetic alterations, albeit from an external source.
Rare exceptions: - SMT proponents often argue that even if some cancers truly lack mutations, these are likely rare exceptions that don't invalidate the theory for the vast majority of cancers.
Limitations of current knowledge: - They might argue that our current understanding of cancer-related genes is incomplete, and mutations in yet-unknown cancer-related genes might be present.
Challenging the "mutation-free" claim: - Some supporters question whether truly "mutation-free" cancers exist, suggesting that more comprehensive analyses would likely reveal some level of genetic alteration.
Fusion genes and other structural variations: - Some genetic alterations, like gene fusions, might not be detected by methods looking for point mutations but still represent genetic changes consistent with SMT.
Mosaicism and low-frequency mutations: - They argue that some mutations might be present in only a small subset of cells or at very low frequencies, making them difficult to detect with standard methods.
Expanding the definition of "mutation": - Some researchers propose broadening the definition of mutations to include other types of genetic and epigenetic alterations, thus maintaining the core of SMT while accommodating new findings.
Integrating with other theories: - Many SMT supporters advocate for an integrated view of cancer development, incorporating aspects of other theories (like the metabolic theory or tissue organization field theory) while maintaining the central role of genetic alterations.
It's worth noting that the cancer research community continues to debate and refine theories based on new evidence. While SMT remains a dominant paradigm, many researchers acknowledge that it may not fully explain all aspects of cancer development and progression. The field is moving towards more nuanced and integrated models of cancer biology that can account for the complex and diverse nature of the disease.
This comment describes some relevant research.
From Somatic Mutation Theory - Why it's Wrong for Most Cancers:
It should come as no surprise, therefore, that somatic mutations are questioned as representing "the" cause for the majority of cancers [10,11] and it should be noted that some cancers are not associated with any mutations whatsoever.
Importantly, a detailed analysis of 31,717 cancer cases and 26,136 cancer-free controls from 13 genome-wide association studies [48] revealed that "the vast majority, if not all, of aberrations that were observed in the cancer-affected cohort were also seen in cancer-free subjects, although at lower frequency" [47]. Thus, the notion that somatic mutations are necessarily harmful and can lead to cancer is not borne out by this study and further affirms the hypothesis that mutations observed in cancers are not the triggering event but more likely a means for the clonal replication of already transformed cancer cells.
From Speciation Theory of Carcinogenesis Explains Karyotypic Individuality and Long Latencies of Cancers:
However, despite 65 years of research on the mutation theory, there is still no proof for even one set of mutations that is able to convert a normal cell to a cancer cell.
Because the prevailing mutation theory has dominated the search for the genetic causes of cancer since the discovery of gene mutation in 1927 [111], consistent mutations or consistent karyotypic abnormalities with specific mutations were expected [15,21,22]. Instead, individual mutations [43,44] and individual karyotypes [2,9,112,113] were found, of which over 68,000 are listed in the NCI-Mitelman database of cancers [6].
From Cancer Treatment: A Systems Approach
When the first results of The Cancer Genome Project were reported, many scientists were shocked to learn that most patients, including those with the same type of cancer, did not share the same cancer-related mutations.
The DNA Theory predicts that cancer will be more common in cells that replicate more frequently and/or in organs that have more cells. Lintern's theory suggests more cancer in organs that have more surface area that are accessible by fungi. I think the evidence favors Lintern here, but probably not very strongly.
Itraconazole therapy in a pancreatic adenocarcinoma patient: A case report: an example of a patient with a terminal cancer diagnosis recovering in apparent response to getting an anti-fungal drug.
Case studies with salvestrol treatment: a class of natural anti-fungals helped cure some "terminal" cancer patients. 6 more case studies. New clinical study with phytonutrients (phytoalexins) a randomized controlled trial.
Fungal diseases mimicking primary lung cancer: radiologic--pathologic correlation: cancer and fungal infections look similar. Somehow, this is rarely taken as evidence that they're the same thing.
How a plant's anti-fungal defence may protect against cancer.
From Finding the "Missing 50%" of Invasive Candidiasis:
Blood cultures are limited for diagnosing invasive candidiasis by poor sensitivity and slow turn-around time.
An important drawback is that the difficulty of beating the market fluctuates depending on factors such as who the other traders are, and what kind of news is moving the markets.
I'm holding a modest long position in NVIDIA (smaller than my position in Google), and expect to keep it for at least a few more months. I expect I only need NVIDIA margins to hold up for another 3 or 4 years for it to be a good investment now.
It will likely become a bubble before too long, but it doesn't feel like one yet.
It currently looks like the free version of ChatGPT is good enough that I wouldn't get much benefit from a subscription. I have little idea how long this will remain true.
The more complex the rules get, the harder it gets to enforce them.
If the threshold is used merely for deciding who needs to report to regulators, then it seems appropriate to use the simple rule. We should care mainly that it applies to the most powerful models at any one time, not that it applies to a fixed capability level.
For a threshold that's designed to enforce a long-term pause, it's going to be hard to do more than slow capability progress without restrictions on GPU production.
My favorite power-related stock is CSIQ (Canadian Solar).
I also have positions in lithium mining companies (for grid storage), and construction companies that have some focus on power grids (e.g. MYRG).
Uranium bets are harder to evaluate.
You seem to assume we should endorse something like average utilitarianism. Bostrom and I consider total utilitarianism to be closer to the best moral framework. See Parfit's writings if you want deep discussion of this topic.
I can't recall any clear predictions or advice, just a general presumption that it will be used wisely.
Given utopian medicine, Gwern's points seem not very important.
He predicts that it will be possible to do things like engineer away sadness. He doesn't devote much attention to convincing skeptics that such engineering will be possible. He seems more interested in questions of whether we should classify the results as utopian.
What evidence do you have about how much time it takes per day to maintain the effect after the end of the 2 weeks?
The part about "securities with huge variance" is somewhat widely used. See how much EA charities get from crypto and tech startup stock donations.
It's unclear whether the perfectly anti-correlated pair improves this kind of strategy. I guess you're trying to make the strategy more appealing to risk-averse investors? That sounds like it maybe should work, but is hard because risk-averse investors don't want to be early adopters of a new strategy?
Doesn't this depend on what we value?
In particular, you appear to assume that we care about events outside of our lightcone in roughly the way we care about events in our near future. I'm guessing a good deal of skepticism of ECL is a result of people not caring much about distant events.
I had nitrous oxide once at a dentist. It is a dissociative anesthetic. It may have caused something like selective amnesia. I remember that the dentist was drilling, but I have no clear memory of pain associated with it. It's a bit hard to evaluate exactly what it does, but it definitely has some benefits. Maybe the pain seemed too distant from me to be worth my attention?
A much higher fraction of the benefits of prediction markets are public goods.
Most forms of insurance did took a good deal of time and effort before they were widely accepted. It's unclear whether there's a dramatic difference in the rate of adoption of prediction markets compared to insurance.
I'm reaffirming my relatively extensive review of this post.
The simbox idea seems like a valuable guide for safely testing AIs, even if the rest of the post turns out to be wrong.
Here's my too-terse summary of the post's most important (and more controversial) proposal: have the AI grow up in an artificial society, learning self-empowerment and learning to model other agents. Use something like retargeting the search to convert the AI's goals from self-empowerment to empowering other agents.
I'm reaffirming my relatively long review of Drexler's full QNR paper.
Drexler's QNR proposal seems like it would, if implemented, guide AI toward more comprehensible systems. It might modestly speed up capabilities advances, while being somewhat more effective at making alignment easier.
Alas, the full paper is long, and not an easy read. I don't think I've managed to summarize its strengths well enough to persuade many people to read it.
This post didn't feel particularly important when I first read it.
Yet I notice that I've been acting on the post's advice since reading it. E.g. being more optimistic about drug companies that measure a wide variety of biomarkers.
I wasn't consciously doing that because I updated due to the post. I'm unsure to what extent the post changed me via subconscious influence, versus deriving the ideas independently.
Exchanges require more capital to move the price closer to the extremes than to move it closer to 50%.
This post is one of the best available explanations of what has been wrong with the approach used by Eliezer and people associated with him.
I had a pretty favorable recollection of the post from when I first read it. Rereading it convinced me that I still managed to underestimate it.
In my first pass at reviewing posts from 2022, I had some trouble deciding which post best explained shard theory. Now that I've reread this post during my second pass, I've decided this is the most important shard theory post. Not because it explains shard theory best, but because it explains what important implications shard theory has for alignment research.
I keep being tempted to think that the first human-level AGIs will be utility maximizers. This post reminds me that maximization is perilous. So we ought to wait until we've brought greater-than-human wisdom to bear on deciding what to maximize before attempting to implement an entity that maximizes a utility function.
Oops. I misread which questions you were comparing.
Now that I've read the full questions in the actual paper, it looks like some of the difference is due to "within 100 years" versus at any time horizon.
I consider it far-fetched that much of the risk is over 100 years away, but it's logically possible, and Robin Hanson might endorse a similar response.
I don't quite see this logical contradiction that your Twitter poll asks about.
I wouldn't be surprised if the answers reflect framing effects. But the answers seem logically consistent if we assume that some people believe that severe disempowerment is good.
The Fed can stimulate nominal demand at the ZLB. But (outside of times when it's correcting the results of overly tight monetary conditions) that means mostly more inflation, and has strongly diminishing returns on increased real consumption.
Eventually the economy would reach a new equilibrium (which presumably would contain the same amount of private consumption as the old equilibrium).
I expect less consumption in the new equilibrium.
The Fed has limited power to affect real demand. Fed stimulus is only helpful if there's unemployment due to something like deflation.
I realize now that some of this post was influenced by a post that I'd forgotten reading: Causal confusion as an argument against the scaling hypothesis, which does a better job of explaining what I meant by causal modeling being hard.
I agree there's something strange about Loyal's strategy.
But it's not like all aging researchers act like they back Loyal's approach. Intervene Immune has been getting good biomarker results in human trials by taking nearly the opposite approach: raising IGF-1 levels for a while.
I wrote a longer discussion about IGF-1 and aging in my review of Morgan Levine's book True Age.
If someone comes into the hospital
That's a bad criterion to use.
See Robin Hanson's Buy Health proposal for a better option.
Is this the post you're looking for?
I've got a Mercedes with an Active Blind Spot Assist that eliminates the need to worry about this.
I understand how we can avoid trusting an AI if we've got a specification that the proof checker understands.
Where I expect to need an AI is for generating the right specifications.