Posts

Less Wrong automated systems are inadvertently Censoring me 2024-02-21T12:57:16.955Z
A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is 2024-02-07T21:49:46.331Z
The Math of Suspicious Coincidences 2024-02-07T13:32:35.513Z
Brute Force Manufactured Consensus is Hiding the Crime of the Century 2024-02-03T20:36:59.806Z
Without Fundamental Advances, Rebellion and Coup d'État are the Inevitable Outcomes of Dictators & Monarchs Trying to Control Large, Capable Countries 2024-01-31T10:14:02.042Z
"AI Alignment" is a Dangerously Overloaded Term 2023-12-15T14:34:29.850Z
Could Germany have won World War I with high probability given the benefit of hindsight? 2023-11-27T22:52:42.066Z
Could World War I have been prevented given the benefit of hindsight? 2023-11-27T22:39:15.866Z
“Why can’t you just turn it off?” 2023-11-19T14:46:18.427Z
On Overhangs and Technological Change 2023-11-05T22:58:51.306Z
Stuxnet, not Skynet: Humanity's disempowerment by AI 2023-11-04T22:23:55.428Z
Architects of Our Own Demise: We Should Stop Developing AI 2023-10-26T00:36:05.126Z
Roko's Shortform 2020-10-14T17:30:47.334Z
Covid-19 Points of Leverage, Travel Bans and Eradication 2020-03-19T09:08:28.846Z
Ubiquitous Far-Ultraviolet Light Could Control the Spread of Covid-19 and Other Pandemics 2020-03-18T12:44:42.756Z
$100 for the best article on efficient charty - the winner is ... 2010-12-12T15:02:06.007Z
$100 for the best article on efficient charity: the finalists 2010-12-07T21:15:31.102Z
$100 for the best article on efficient charity -- Submit your articles 2010-12-02T20:57:31.410Z
Superintelligent AI mentioned as a possible risk by Bill Gates 2010-11-28T11:51:50.475Z
$100 for the best article on efficient charity -- deadline Wednesday 1st December 2010-11-24T22:31:57.215Z
Competition to write the best stand-alone article on efficient charity 2010-11-21T16:57:35.003Z
Public Choice and the Altruist's Burden 2010-07-22T21:34:52.740Z
Politicians stymie human colonization of space to save make-work jobs 2010-07-18T12:57:47.388Z
Financial incentives don't get rid of bias? Prize for best answer. 2010-07-15T13:24:59.276Z
A proposal for a cryogenic grave for cryonics 2010-07-06T19:01:36.898Z
MWI, copies and probability 2010-06-25T16:46:08.379Z
Poll: What value extra copies? 2010-06-22T12:15:54.408Z
Aspergers Survey Re-results 2010-05-29T16:58:34.925Z
Shock Level 5: Big Worlds and Modal Realism 2010-05-25T23:19:44.391Z
The Tragedy of the Social Epistemology Commons 2010-05-21T12:42:38.103Z
The Social Coprocessor Model 2010-05-14T17:10:15.475Z
Aspergers Poll Results: LW is nerdier than the Math Olympiad? 2010-05-13T14:24:24.783Z
Do you have High-Functioning Asperger's Syndrome? 2010-05-10T23:55:45.936Z
What is missing from rationality? 2010-04-27T12:32:06.806Z
Report from Humanity+ UK 2010 2010-04-25T12:33:33.170Z
Ugh fields 2010-04-12T17:06:18.510Z
Anthropic answers to logical uncertainties? 2010-04-06T17:51:49.486Z
What is Rationality? 2010-04-01T20:14:09.309Z
David Pearce on Hedonic Moral realism 2010-02-03T17:27:31.982Z
Strong moral realism, meta-ethics and pseudo-questions. 2010-01-31T20:20:47.159Z
Simon Conway Morris: "Aliens are likely to look and behave like us". 2010-01-25T14:16:18.752Z
London meetup: "The Friendly AI Problem" 2010-01-19T23:35:47.131Z
Savulescu: "Genetically enhance humanity or face extinction" 2010-01-10T00:26:56.846Z
Max Tegmark on our place in history: "We're Not Insignificant After All" 2010-01-04T00:02:04.868Z
Help Roko become a better rationalist! 2009-12-02T08:23:37.643Z
11 core rationalist skills 2009-12-02T08:09:05.922Z
Being saner about gender and rationality 2009-07-20T07:17:13.855Z
How likely is a failure of nuclear deterrence? 2009-07-15T00:01:28.640Z
Our society lacks good self-preservation mechanisms 2009-07-12T09:26:23.365Z
The enemy within 2009-07-05T15:08:05.874Z

Comments

Comment by Roko on CFAR Takeaways: Andrew Critch · 2024-03-07T16:59:59.762Z · LW · GW

Ability to go sleep... Sleep deprivation is one of the greatest effect sizes for IQ

Interesting!

Being alive is so much fun. Sleep is death, I don't want to sleep!

Comment by Roko on Less Wrong automated systems are inadvertently Censoring me · 2024-02-21T17:58:46.661Z · LW · GW

I not only feel censored, I am censored in the sense that my ability to speak is being taken away. The causality seems to be people downvoting --> negative karma --> algo prevents posting, but that's still censorship

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-21T12:27:40.061Z · LW · GW

The point of probabilities is to quantify uncertainty, not to wait until you are omnipotent and have all the data needed to reach certainty

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-14T23:56:12.306Z · LW · GW

it's the largest wet market in central China

Got a source for that?

My impression of Huanan Seafood Market is that it contained only a very small number of animals that are even potential candidates for the virus, and is mostly fish (seafood).

Wikipedia contradicts itself on this, claiming circa 100 animals in one section and then circa 10,000 in a different section. Do you know what is going on here?

https://en.wikipedia.org/wiki/Huanan_Seafood_Wholesale_Market#Facility_and_operations

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-14T23:49:02.690Z · LW · GW

I don't think you understand probability theory

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-14T13:09:37.849Z · LW · GW

I'm open to sponsorship to do further research at $200/hr. DM me if you're interested.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-14T11:36:27.758Z · LW · GW

Credibly dismiss? What?

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-14T11:36:11.312Z · LW · GW

Given that they were already caught in two separate coverups, it is not only not implausible it is highly likely that some kind of cover-up of the early cases was attempted. The only question is whether they succeeded to the extent of making it look like the wet market was the origin.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-11T17:29:46.083Z · LW · GW

hmmm interesting

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-10T20:11:09.492Z · LW · GW

The whole point of probability theory is to make decisions when you do not have "perfect data sources"

Why even talk about probabilistic reasoning if you won't use it until the data is "perfect" and you are omniscient?

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-10T20:06:11.518Z · LW · GW

Related:

https://twitter.com/R_H_Ebright/status/1756308105103601707

Comment by Roko on OpenAI wants to raise 5-7 trillion · 2024-02-09T16:57:37.648Z · LW · GW

It is not a misprint.

This kind of strategy is rational once you're sure that The Singularity is going to happen and it's just a matter of waiting out Moore's Law, there are benefits to being first.

Comment by Roko on More Hyphenation · 2024-02-09T12:05:24.098Z · LW · GW

Yes. I have noticed that I prefer hyphens, and now that I think about why it's because they make writing less ambiguous.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T20:31:19.683Z · LW · GW

pandemics are arguably more likely to originate in large cities because population density facilitates spread, large wildlife markets are more likely, and they serve as major travel hubs

China was 63% Urban in 2020.

https://www.statista.com/statistics/270162/urbanization-in-china/

Say it's 2x more likely in Urban areas. It doesn't really make that much difference. Plus, there is some probability of it going to Vietnam or other SE Asian countries (Vietnam is closer to Yunnan than Wuhan is).

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T16:47:52.713Z · LW · GW

thx

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T11:00:47.191Z · LW · GW

It's not specifically DEFUSE, it's DEFUSE and all possible related dangerous GoF work which became possible post 2017

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T10:57:07.329Z · LW · GW

It doesn't specifically have to be DEFUSE, it just has to be some work which started after the following key events:

circa 2011: technology becomes available for dangerous GoF and people start discussing it

circa 2018: ban on GoF is lifted

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T10:26:35.294Z · LW · GW

WIV scientists' behavior after leak (talking about adding fcs to coronavirus in december, going to dinner, publishing ratg13)

wait, they talked about adding a FCS? Where?

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T10:25:42.688Z · LW · GW

proline at fcs, otherwise suboptimal fcs, out of frame insertion,

I have never heard of this, where can I find out more?

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T10:22:41.219Z · LW · GW

But we are updating on the timing.

Under the null hypothesis we assign equal probability to each year between 1980 and 2060, and they add up to 1. So there is an assumption there that a pandemic will definitely occur starting in china.

We should make the same assumption under the alternate hypothesis. The only difference is under AH there's a lab leak. So we just adjust the way the probability is allocated by year. It still has to add up to 100%.

So, maybe we'll have a uniform background of 0.1% per year between 1980 and 2060, and then after the 2011 events where people started talking about GoF it increases a bit as GoF is at least possible, then it increases again in 2017 when GoF is funded and greenlit, and after that each year it decreases a little bit, think of it as a hazard rate, once it has happened once people will start being cautious again.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T09:13:31.213Z · LW · GW

It has to be conditional on a massive global pandemic starting in that country at all, to make a fair comparison with the 2/80 calculation under the null hypothesis.

But say we break it down into two parts. (1) probability that the GoF research does have the potential to cause a pandemic and (2) distribution in time of the pandemic after research starts.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T09:01:11.384Z · LW · GW

I'm avoiding that as I don't understand the data provenance/cover-up potential.

The point of this post is to process just the "clean" data - stuff that interested parties such as WIV, Ecohealth and WHO could not have changed or affected.

Of course others should try to look into that and work out what's going on.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T02:44:04.157Z · LW · GW

What pieces of circumstantial evidence are unlikely under the Lab Leak hypothesis?

I don't think evidence should generally be two-sided. E.g. imagine a game that's an amateur versus Magnus Carlsen, but you don't know who is white and who is black. If you look at the outcomes of moves as individual bits, it will be very one-sided.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T00:25:54.623Z · LW · GW

please elaborate ....

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-08T00:02:38.347Z · LW · GW

e.g. this bet

https://longbets.org/9/

also contributes to the idea that the prior cannot be that small since Rees bet on it in 2017.

Comment by Roko on A Back-Of-The-Envelope Calculation On How Unlikely The Circumstantial Evidence Around Covid-19 Is · 2024-02-07T23:58:10.620Z · LW · GW

Under the alternate hypothesis, location, warnings, timing and specific features are all much more likely. It's probably something like 0.6^4 ~= 0.12

Priors for a lab leak vs a natural spillover are a bit harder, but we have examples of lab leak such as foot and mouth disease in the UK, other leaks of covid, etc. I think a reasonable prior for a lab leak is between 1% and 30% but priors are of course not something we can expect everyone to agree on.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-07T19:07:38.983Z · LW · GW

some people truly are beyond help lol

Comment by Roko on Don't sleep on Coordination Takeoffs · 2024-02-07T18:02:24.494Z · LW · GW

I think the most likely source for a coordination singularity is crypto, not prediction markets.

PMs will not get you out of bad NEs.

Comment by Roko on Don't sleep on Coordination Takeoffs · 2024-02-07T16:47:34.684Z · LW · GW

. I do not stack with the people doing more crypto-focused stuff

why not?

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-07T15:41:14.813Z · LW · GW

That just shows different intel orgs have different ideas.

We know from declassified docs that the spate of UFO sightings in the 1940s and 1950s were caused by various US spy balloon projects, but different parts of the US military and intel are very heavily siloed from each other so even most of the military thought the UFOs were real.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-07T15:32:17.610Z · LW · GW

Regarding cases at the Wuhan Central Hospital and HPHICWM, patients with a history of exposure at Huanan Market could not have been “cherry picked” before anyone had identified the market as an epidemiologic risk factor. Hence, there was a genuine preponderance of early COVID-19 cases associated with Huanan Market.

I see. So the claim is that these early cases were reported via channels that could not have been messed with, and they could not have been cherry picked because it was not known what the disease was.

But I suppose it is still possible that maybe these records were actually messed with, or that someone from WIV or SKVL or whatever deliberately infected the market with covid-19 as they knew it would make for a fairly bulletproof cover of an actual leak that they already knew about. Or something else weird like that - maybe intel did it on purpose for some convoluted set of reasons, knowing that it would create an ambiguous situation with some people pointing at a lab leak and some people pointing at the market.

Weird things happen in the murky world of human conflicts.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-07T14:48:49.091Z · LW · GW

It is weird, but there are two different groups (China, Daszak/Ecohealth) with different incentives. In fact there are more than two different groups - Wuhan local authorities and Chinese central party officials had different incetives.

The Chinese initially wanted to cover it up, but they couldn't keep that up forever. Daszak and Echohealth (and likely parts of the US intelligence agencies and military) desperately wanted this to not be a lab leak, as shown by FOIA'd emails from early 2020.

So, who faked what? Who molested the data before I get to see it?

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-07T10:34:46.546Z · LW · GW

I've always wanted to see some hard data on this. All the wet markets in China and Vietnam, numbers of animals per month, etc. That kind of model would be extremely useful in pinning down just how unlucky an innocent WIV would be.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-06T13:18:00.811Z · LW · GW

That does sound interesting. I think I have a solution.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-06T11:51:29.817Z · LW · GW

It appears to me that all evidence for the claim that the virus originated in the wet market pass through Chinese government sources

This is my concern.

Data from Wuhan in early 2020 is "dirty". It had to pass through Chinese government hands, and then maybe also got filtered by the 2021 WHO investigation which was headed by... well you know who ;-)

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-05T22:50:25.108Z · LW · GW

The full arguments for a proposition can be arbitrarily long. You have to select a finite subset to even engage with.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-05T20:57:54.615Z · LW · GW

yes.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-05T19:27:51.782Z · LW · GW

I think it is implausible that an unrelated coronavirus with a FCS would magically make itself very genetically similar to existing sarbecoronaviruses. Of course it could make itself phenotypically similar (like whales are to sharks) but the genome would look very different.

https://en.wikipedia.org/wiki/RaTG13#Phylogenetics

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-05T19:16:48.324Z · LW · GW

we should expect the frontpage to be curated to be more conforming to mainstream viewpoints

I believe that that is the case and may be appropriate for LW

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-05T19:15:55.080Z · LW · GW

FYI: I think it's entirely reasonable to leave this post as a personal blogpost if the LW mod team feel that it is too "spicy" to associate with the LW brand.

If in a few years it turns out that the thesis here is proven correct one can discuss how to handle controversial but plausibly true posts.

Comment by Roko on Davidad's Provably Safe AI Architecture - ARIA's Programme Thesis · 2024-02-05T16:37:32.998Z · LW · GW

because of your attachment to specific model of causality known to ten people in entire world, you risk to notice this too late.

you're thinking about this the wrong way. AGI governance will not operate like human governance.

Comment by Roko on Safe Stasis Fallacy · 2024-02-05T15:47:24.375Z · LW · GW

I think the risk with AI safety is whilst it is not an explicitly pro-tyranny ideology, there is an increasing need for governance and control, and so by blocking off governance by Friendly AGI we will not get less AI risk but more tyranny from human government structures (which are inherently corrupt and tyrannical because humans are untrustworthy and traditional human government structures are riddled with brokenness)

Comment by Roko on Davidad's Provably Safe AI Architecture - ARIA's Programme Thesis · 2024-02-05T13:07:51.316Z · LW · GW

Humans already do this, except we have made it politically incorrect to talk about the ways in which human-generated Goodhearting make the world worse (race, gender, politics etc)

Comment by Roko on Davidad's Provably Safe AI Architecture - ARIA's Programme Thesis · 2024-02-04T20:58:40.340Z · LW · GW

I think the first step will be using AGIs to come up with better plans.

Comment by Roko on Davidad's Provably Safe AI Architecture - ARIA's Programme Thesis · 2024-02-04T20:58:06.965Z · LW · GW

The current paradigm of AI research makes it hard to make really pure tool AIs.

We have software tools, like Wolfram Alpha, and we have LLM-derived systems.

This is probably the set of tools we will either win or die with

Comment by Roko on Most experts believe COVID-19 was probably not a lab leak · 2024-02-04T17:07:31.389Z · LW · GW

How many lab similar to Wuhan in crowded cities

None. Wuhan is the only BSL4 lab in China, and it is the only place that did bat coronavirus gain of function research. And Shi Zhengli's group at WIV is the premier group in China that studies bat coronaviruses.

Comment by Roko on Most experts believe COVID-19 was probably not a lab leak · 2024-02-04T17:05:25.382Z · LW · GW

Daszak's comments say that much of the work could be done at WIV which could include the FCS. The comment is genuine AFAIK but you'll have to chase it up yourself.

The work on BsmBI is somewhat compelling because the people who suspected it published before DEFUSE was unearthed, and then DEFUSE was found to contain an order for BsmBI. So, they sort of predicted this. And there are reasons to make the virus out of relatively uniformly long segments - it's convenient, the tools and techniques have maximum lengths they can handle, and you want to minimize the total amount of work. So you use roughly equal length segments. However this is a fairly complicated series of claims and I don't see it as being the big win for lab-leak - the big win is DEFUSE itself.

But zoom out a bit: we can just notice that all of this stuff only collects around Wuhan, WIV and associated facilities like State Key Laboratory of Virology at Wuhan University.

There's no equivalent of this in all the other cities in China and I am pretty sure that if you look through the top 20 cities by population you won't find something like DEFUSE. I tried this a bit with chatGPT, Google Scholar, Google Search etc. There simply are not 50 other Daszaks out there doing GoF bat coronavirus research in every other city in China. Wuhan is THE PLACE where this happens.

Comment by Roko on Most experts believe COVID-19 was probably not a lab leak · 2024-02-04T16:48:58.795Z · LW · GW

I'm curious what makes you think COVID-19 was created in a lab and not a natural virus leaked while they were studying it.

Sorry, I misread this.

I don't have a strong opinion. I think it's plausible that it was a lab leak of a natural virus, though I will note that the more technical people are more skeptical of this, claiming that covid-19 was unusually good at spreading through humans even from the start, which is unlikely for a fully natural spillover.

If I knew more about the details of virology, I would have a stronger opinion.

Comment by Roko on Most experts believe COVID-19 was probably not a lab leak · 2024-02-04T16:45:19.661Z · LW · GW

Conditional on COVID-19 having escaped from a lab, why do you think it was created there rather than being a natural virus they were studying in that lab?

I don't have a strong opinion on that question.

Comment by Roko on Brute Force Manufactured Consensus is Hiding the Crime of the Century · 2024-02-04T14:13:37.079Z · LW · GW

Apparently that is the classical way to do things, though naming stuff after people is no longer done in modern biosciences.