Posts
Comments
I don't think the "US censorship community" had any problem banning things shared on TikTok. They could just say "If you don't ban the Bin Ladin letter discussion, we take it as a reason to invigorate our banning efforts".
Other important companies that are affected:
Riot Games is 100% owned by Tencent.
Tencent has a 40% stake in Epic Games. Given that Epic Games is working on their own app store, being able to pressure them to censor apps is likely of interest.
Eliezer wrote The 5-Second Level.
If you identify a bad 5-Second step, 30 seconds give you six training runs where you can go through the step to train it.
One core problem with AI is that it's not just "people" who make up new things behind teh scenes but AI itself that will make up new things.
TikTok continues to be Chinese spyware. It also continues to be an increasing point of vulnerability for China to put its thumb on American culture, politics and opinion.
As long as it's legal for anyone to be a data broker and sell out a lot of data about the American population to anyone willing to pay the price, which includes China, shutting it down for being spyware makes little sense.
If you truly care about data not reaching China you need GDPR-type data privacy legislation and enforce that legislation.
One, continued demands that we ‘prove’ that China or ByteDance has its finger on the scale. I say that the information elsewhere in this post, in the absence of counterexamples, is very strong evidence. At minimum, they are fixing content balance issues they dislike and allowing those they like, with an algorithm that snowballs.
TikTok banned discussion of Bin Ladin's letter not because the CCP disliked it but because the US national security establishment disliked the letter. The same is true for them banning Glenn Greenwald.
While they might change their content moderation policies in the future in the moment those decisions are made to appear nonthreatening.
I am sympathetic to ‘rushed’ objections when bills are so long there is no time to read and understand them. This does not seem to be one of those cases.
If you think you understand the implications, which other popular social media company is partly under Chinese control and thus targeted by the bill if the government would argue that it violates national security concerns?
Tencent owns around 38% of Discord's shares.
This law makes it easy to pressure Discord to do whatever the White House wants them to do by threatening to ban them otherwise.
Transposons activity is not downregulated in the placenta while being downregulated in most other cells. If there are too many transposons in the DNA that likely makes the placenta fail during the pregnancy. As a result, the amount of variance in transposon counts you see when sequencing born people is more limited than that of embryos.
If you do IVF you care about the pregnancy not terminating after three or four months.
The evidence is more of a theoretical argument. In the selfish gene frame, transposons (even when they aren't technically genes) "benefit" from copying themselves even when it reduces the total fitness of the organism.
You need a mechanism that prevents people with too many transposons from procreating to account for transposons counts not just growing indefinitely.
In the absence of a process that regulates a quantity, you have variance. Lack of observed variance in sequenced adult genomes is evidence of a mechanism for regulation existing.
Source?
By the mid-2010s, you could genotype all the parts of a person’s DNA most likely to differ from other people’s for under $100. At that price point, it became possible to gather genomes from hundreds of thousands of people and assemble them into giant databases that researchers could access.
I think there's a good chance that transposon count in the DNA differs between people, shotgun sequencing can't tell you how many times a given sequence appears.
There's a tradition in biology to consider things that can't be easily measured as irrelevant, but that doesn't mean it's true.
If you have a nonstandard unicode character in your password and an attacker tries to crack a password based on the assumption that no nonstandard unicode character is in the password, they can't crack your password no matter how much compute they throw at it.
Given that you decide where in your string you place it, the attacker would have to test for the special character being at multiple different positions which adds a lot of additional entropy.
How often do you do this per week?
An AGI can kill you even if it's not beyond what you consider to be "too frightening".
The grading isn't on a scale.
Do you have any source for the technology being an improvement in accuracy over polygraphs?
Status is a way to have power. Aligning an agent to be power-maximizing is qualitatively different from what we want from AI which we want to align to care about our own ends.
Caveat: people differ in body odor based on genetics, hormones, and armpit microbiome. I personally am privileged to not smell bad, therefore I don't shower until my skin or hair starts to feel icky (a few days).
Experimenting with this requires a source of trustworthy feedback. Only try this if you have a friend who's opinion you trust that you can ask whether or not you smell bad.
Practically it's also worth noting that specific emotional states such as going to an event with a lot of social anxiety can make your body sweet in a way that's smelly even if you normally don't.
Yes, this story sounds like the default way EA Funds operates and not like an outlier.
Scheming is one type of long-term planning. Even if a AI is not directly able to do that kind of long-term planning an AI that works on increasing it's on capabilities might adopt it later.
Beyond that not all scheming would result in the AI resisting direct shutdown. We have currently "AI" getting shutdown for price fixing in the real estate sector. If someone would create an LLM for that purpose that person is likely interested in the AI not admitting to doing price fixing directly while they are still interested in profit maximization. There are going to be a lot of contexts where economic pressures demands a profit maximizing AI that will deny that it violates any laws.
Just because an AI doesn't engage in simple plans does not mean it won't do more complex ones. Especially in those cases where the economic incentives misallign with the intent of regulations.
Humans usually don't dream in response to queries but when their minds are more idle. Besides you not being able to rule it out, why do you think it happens in the AI case?
Legal fight are the result of products that the market wants being made available even when it violates license rights. Companies then just pay the what the court tells them is a reasonable price for the patent violation.
It might be that in practice the friction makes certain products not available instead of just increasing their prices a bit, but I would want to hear from an industry insider that this is a significant problem to believe that.
I don't see why genetic drift would increase this particular mutation. On average I expect genetic drift to destroy the body's ability to produce substances and not increase their production of them.
If the selection pressure is due to certain risk-taking behavior, it's not clear that we see a removal of selection pressure. In humans, we certainly don't have an environment which is similar to those cavefish as far as the selection pressures go.
13: established product areas can have a minefield of IP such that the obvious combinations of features would be legally prohibited from being sold without unavailable license deals.
Do you have links to documented cases where products seem to be unavailable for these reasons?
When the government wants electric car production it has a choice to invest directly into electric car R&D or to subventionize electric cars. The United States spends much more money on subventions than on R&D.
I would guess that if you ask the lawmakers they would say something like "Electric car companies are much better at investing money into developing better cars than the government happens to be, so the subventions are the more effective policy tool than paying the same amount of money for basic research".
I don't think this is a pure matter of incentives but simply of the ideology of privatization. It might be wrong and in reality direct government R&D is more effective than subventions but that's an argument you have to make more directly.
Usually, the FDA is framed as a correction to misalignments between the interests of consumers and pharmaceutical companies. People want effective medicine but pharmaceutical companies can give them sugar and sawdust for cheap and it is difficult for patients to find out.
The Ranbaxxy case suggests that the FDA isn't that good at preventing that from happening. They mostly work on a basis where they trust pharmaceutical companies to do what they are saying they do.
It's unlikely that you would be turned down in EA for being Russian but that's not the same thing as being located in Russia.
When it comes to funding projects in Russia, both sanctions from Western governments and potential problems of governance with the Russian state matter. Putin understands that AI is very important so there's Russian government interest in AI projects in Russia.
There's no software solution but when you actually see such criticism you can vote it up strongly. If we have enough experienced people in this community who have the karma to cast strong votes and willingness to do it, the problem is solvable.
Have you done any calibration training?
There's a lot of model uncertainty that makes it so that 1 in 80 million is way too high.
It's equivalent to watching a season of a TV show
If you watch a debate the way you would a TV show where you aren't critically evaluating everything, that's not a good basis for forming beliefs about the real world.
Youtube videos tend to encourage you to consume them in that way, but that doesn't make it a good epistemic practice and that's part of the reason why scientific debates usually happen in text.
But gain of function is a new invention - it only really started in 2011 and funding was banned in 2014, then the moratorium was lifted in 2017. The 2011-2014 period had little or no coronavirus gain of function work as far as I am aware. So coronavirus gain of function from a lab could only have occurred after say 2010 and was most likely after 2017 when it had the combination of technology and funding.
Ralph Baric's lab was doing work that he thought would fall under the gain-of-function ban in 2014. He published the paper where Fauci said in front of congress that it contained no gain-of-function while Fauci gave the paper the file title when he mailed it around of "Baric, Shi et al - Nature medicine - SARS Gain of function.pdf".
While his funding was stopped temporarily the NIH decided not to honor the moratorium and continued funding his research through the EcoHealth grant in the official time of the moratorium.
This assumes that the moratorium actually prevented gain-of-function research on Coronaviruses in any meaningful sense and that the NIH honored it. That does not seem to have happened.
So they ask their economics experts about how to efficiently encourage the relevant tech development, and they decide to temporarily subsidize different routes to hydrogen at slightly below their current extra costs.
I don't think that's a good model. Instead of asking economics experts, they ask lobbyists about what's necessary for the technology to be developed.
The lobbyists from BP and Shell tell them: We are happy to switch from fossil fuels to green energy and to do that we need your help by implementic policies X, Y and Z.
Modelling the way these policies are made in Westerns states or even in China without lobbyists is going to lead to poor understanding of the mechanisms.
If you have a nonstandard unicode character in your password and an attacker tries to crack a password based on the assumption that no nonstandard unicode character is in the password, they can't crack your password no matter how much compute they throw at it.
Given that you decide where in your string you place it, the attacker would have to test for the special character being at multiple different positions which adds a lot of additional entropy.
Japan has a high-trust culture but China has a low-trust culture. From what I know Chinese do care about what other people think.
You might want to add a single nonstandard Unicode character at a random position to get a lot of additional entropy.
I'm not as familiar with the details but I don't see why a subpoena would have a stronger force of law than the COVID-19 Origin Act of 2023 when it comes to making information public. If they send a subpoena to the Director of Intelligence a subpoena I would expect him to claim executive privilege.
and they would do so knowing that it's eventually going to come out that they withheld evidence?!?
They withheld evidence. The only question we don't know is how strong the evidence they withheld happens to be. Congress specifically wrote the section into the COVID-19 Origin Act of 2023 because they know that there's evidence about those three employees from the State Department quote from above.
Again, it's winter, people get sick, that's very weak Bayesian evidence of an outbreak, at best. On priors, how many people at an institute that size get influenza every month during the winter?
Michael Schellenberger et al suggest:
According to multiple U.S. government officials interviewed as part of a lengthy investigation by Public and Racket, the first people infected by the virus, “patients zero,” included Ben Hu, a researcher who led the WIV’s “gain-of-function” research on SARS-like coronaviruses, which increases the infectiousness of viruses.
This suggests that the evidence was strong enough to convince some of the U.S. government officials with access to the evidence to convince them.
According to that article, the three employees of the WIV were not random members of the institute but working on coronavirus gain-of-function. They also seemed to be ill enough to go to the hospital which is not typical with influenza.
That kind of information wasn't in previous government disclosures and wrote the COVID-19 Origin Act of 2023 to make the government disclose it. For some reason, the Director of Intelligence is withholding information for which Congress explicitly asked them.
Given that the Director of Intelligence is willing to withhold evidence that Congress can be certain to exist, they might also be withholding pieces of evidence over which we have no clue, and that might be protected enough that the sources with whom journalists talked didn't know either or were unwilling to share.
months earlier
As far I remember previous reporting was about them being ill in November so not multiple months earlier.
How charitable of you. I was misinformed: I thought rationalists were (generally) not mind-killed.
That's easily solved by reading the post from Eliezer about Politics is the Mind-Killer and understanding the advice it makes.
For me to be wrong means you recollected the thought process that went into a one-sentence snipe
If I had only voiced that position in a comment and nowhere else, that might be true. That's not the case, I have multiple times criticized people for not applying Politics is the Mind-Killer and using political examples to make points that aren't about politics.
Says the guy who often wades into politics and often takes politically-charged stances on LW/EAF. You seem to be correct, it's just sad that the topic you are correct about is the LessWrong community.
I talk about politics when I want to make a point about politics. I usually don't talk about politics when I want to make a point about something else. If you want to make a point about politics it's unavoidable to talk about politics.
The advice of Politics is the Mind-Killer is that you don't talk about politics if you want to make a point that isn't about politics because using political examples makes it harder for the point that isn't about politics to come through. That post does not advise people not to talk about politics even if people who haven't read it sometimes use the title as a catchphrase for the position that one shouldn't talk about politics in general.
It's a useful heuristic that Eliezer proposed. You write a post titled "Let's make the truth easier to find" and in it follow heuristics that make the truth harder to find. If your actual goal would be to "Let's make the truth easier to find" then my feedback would be valuable. Of course, if your goal is to signal that you care about the truth and have certain political positions, then my feedback feels offensive.
The nearest virus to COVID-19 we know to be likely in possession of the WIV was the one from Laos, not ratg13.
Publishing a sequence from a virus that comes from China like ratg13 and not from Laos might be a move they took to make it more plausible that a relative of the virus from the Chinese cave naturally spilled over.
And my audience was LessWrong, which I thought could handle the examples like mature adults.
Part of rationality is not being in denial of reality and there are certain realities about what happens when you talk about politics.
Part of what the sequences are about is to care about reality and you prefer to be in denial of it and ignore the advice that the sequences made. Bringing up that you ignored it felt relevant to me.
I'm certain you gave no thought to the matter. It feels like you're just here to tear people down day after day, month after month, year after year.
Then you are wrong. Contemporary politics is one source of examples but a very bad source as described in the sequences. There's history. In the West, we generally study history to learn from it.
If you know of some non-political examples which have had as much impact on the modern world as the epistemic errors involved in global warming policy and the invasion of Ukraine, by all means tell me.
The decision to invade Iraq in more recent history was driven by bad epistemics. Talking about it does not trigger people's tribal senses the same way as talking about contemporary political conflicts.
If you go further back, there are also plenty of things that happened in the 20th century that were driven by bad epistemics.
Lastly, there's no reason why you have to pick the most consequential examples to make your points. You don't want people to focus on big consequences but want them to focus on the dynamics of truthseeking.
You're saying that 1) it's unlikely to be a lab leak known to US Intel
I'm not sure what you mean with "known to". There's evidence. If we believe reporting that at least some people within the intelligence agencies who were willing to leak information to Schellenberger/Taibbi believe that this evidence feels conclusive to some members of the intelligence agencies about three WIV employees being ill with COVID but it doesn't seem conclusive to other people.
Evidence often is in a form where it looks conclusive to some people but not others, especially when some of the people engage in motivated reasoning.
Second, if they have evidence about WIV members having COVID, (and not, you know, any other respiratory disease in the middle of flu/cold season,) I still don't know why you think you would know that it was withheld from congress.
The COVID-19 Origin Act of 2023 puts a legal obligation on the Director of National Intelligence to disclose among other things:
(C) researchers at the Wuhan Institute of Virology
who fell ill in autumn 2019, including for any such
researcher--
(i) the researcher's name;
(ii) the researcher's symptoms;
(iii) the date of the onset of the
researcher's symptoms;(iv) the researcher's role at the Wuhan
Institute of Virology;
(v) whether the researcher was involved with
or exposed to coronavirus research at the Wuhan
Institute of Virology;
(vi) whether the researcher visited a hospital
while they were ill; and
(vii) a description of any other actions taken
by the researcher that may suggest they were
experiencing a serious illness at the time
The report that was created in response does not provide that information.
I think from the reporting we have, we can be relatively certain that the intelligence agencies have reports of three WIV employees falling ill. How strong the evidence happens to be that this was COVID-19 is not publically known but would be publically known if the Director of National Intelligence had followed the law.
There has to be some reason why the Director of National Intelligence decided not to follow the law passed by Congress and release that information in his report. The unwillingness shows that the Director of National Intelligence (as representative of the intelligence communities) is biased.
EDIT: According to the State Department:
The U.S. government has reason to believe that several researchers inside the WIV became sick in autumn 2019, before the first identified case of the outbreak, with symptoms consistent with both COVID-19 and common seasonal illnesses. This raises questions about the credibility of WIV senior researcher Shi Zhengli’s public claim that there was “zero infection” among the WIV’s staff and students of SARS-CoV-2 or SARS-related viruses.
I do think that at least part of the treatment in adequate worlds will be done via prediction-based medicine where a doctor gives the patient predictions about the outcome of various treatments or their absence and then gets his prediction track record evaluated.
In prediction-based medicine, you don't need centralized evaluation of medicine via the FDA or of doctor education via med school but can evaluate on the fly and thus have competition based on treatment outcomes instead of competition based on credentials.
It seems like the part that makes that claim is behind the paywall. Can you quote it?
With the amount of leaking going on, I think it would have leaked if they had picked up through their surveillance efforts acknowledgments of how Chinese leaders admitted a lab leak to each other.
There seems to be some evidence of the unusual activity at the Wuhan Institute of Virology (WIV) with reduced cell phone traffic and possible roadblocks as mentioned in the NIH letter to the EcoHealth Alliance. The US intelligence community should have more details about this.
There also seems to be some evidence pointing towards three employees of the WIV being hospitalized with COVID-19 before the COVID-19 cases that we publically know about. This evidence seems to be illegally withheld from Congress. I see no reason for the intelligence agencies to violate the law and not disclose that evidence if they aren't interested in discouraging people from believing in the lab leak hypothesis.
A foundational text in some academic field might take 17 hours to read, but you would still expect someone to have read it before making a priori wild claims that contradicted the expert consensus of that field very radically.
There's a good reason why the foundational text in an academic field would be a text and not a video. Evaluating arguments is easier to do when they are done via text.
I think that makes familiarising yourself with those arguments (whether from the debate or another equivalent-or-better source)
I don't think anyone has shown that the debate contains specific arguments that Roko is unfamiliar with.
especially given that most people seem to have been swayed in the opposite direction to your claim
What evidence do you have for that claim?
There's the state department's effort to investigate the lab leak hypothesis by building some Bayesian model which was shut down because it might "open a can of worms". From the memo notes:
Over the past months, members of Former Assistant Secretary Ford’s staff, and some AVC staff members, warned AVC leadership not to pursue an investigation into the origin of COVID-19. Both AVC and ISN staff members stated that AVC would “open a can of worms” if it continued.
There are also the whistleblower complaints that CIA analysts were bribed to rule against lab leak hypothesis.
The FBI analysts came to a different conclusion than the major US intelligence agencies while probably having similar access to evidence.
There are limits of how much the intelligence agencies can bias the findings by putting pressure on analysts. It seems that they didn't totally rule out the lab leak hypothesis because that would be pretty hard to justify.
Did we need to know anything but "Covid is an airborne infectious respiratory virus"?
The virologists did not consider the topic about whether or not coronaviruses are airborne worth studying. They were rather assuming that it isn't airborne and doing their research under safety protocols that don't protect against airborne transmission.
If you actually want to know those things, funding virologists is useless and you instead want to fund epidemiologists that study disease transmission.
It's a visual illustration, no? Visually this looks rather strong (Figures 1 and 2 on pages 16-17).
The visual match can quite easily show you that one match is stronger than another but they don't tell you how good a match in question actually happens to be. There are ways to measure whether something is a good match with numbers.
And I do think that China might have enough research manpower to just routinely reproduce all published findings of this kind of experiments if they want to (I don't know if they actually do that; I would not be too surprised if they do that as a routine though; this depends on what are their actual policies; I have no means to investigate that).
The idea that a country would just spend that much research capital and do that without it leaving any trace in their research publications and other public communication seems farfetched.
Why are you using vague terms like "very high" instead of being specific about the numbers you consider to be "very high"? It would be much easier to follow your claim if you would be more specifc.
It is rather obvious that Covid is not derived directly from this virus (the difference is too big), but it does look to me and to many other readers of this preprint, although not to everyone, that it's likely that people who have synthesized the original Covid have been looking at the published 2008 sequence while doing their work.
Why? How do you expect that synthezing process to look like? What motivation do you imagine for it as a research project?
Why would they synthezise the whole genome instead of just taking one viruses they had in their lab and insert the mutations they want to study? Inserting specific mutations is much easier than synthesizing the whole thing.
But the exact same thing that makes it substantial evidence (there aren't many such labs, whereas there are many opportunities for natural zoonosis which could happen in a wider variety of places) also means that the prior is low.
Why does that mean the prior is low?
I see no reason why I would assume that the prior for a lab to leak a virus with airborne transmission when they handle it under biosafety level II which is not designed to prevent airborne transmission is low.
then there is a clear match with a particular 2008 PNAS paper and the sequence published in connection with that 2008 paper.
I don't think "clear match" is a good term. Whether something is a clear match or not does not depend on other sequences that were published. If you sort them, you can see which of the published sequences is the "nearest match", but that's a different concept than "clear match".
With that nearest match having only 74% identity, that's a lot of distance to the actual SARS-Cov-2 virus.
To me, it looks like a low-quality paper that doesn't deserve much attention and is far from anything worth being called a smoking gun.
To make sure those claims are correct someone would need to independently reproduce its findings. It's not all that straightforward, because people running the sequence database in question has turned the ability to filter by the date of submission off since then
You don't really need that. You can just look at the sequences under discussion.
From where did you get that number?
Fauci is not the only person who created chilling effects. The Chinese also did their best to discourage people from believing in the lab leak hypothesis and there are many others who had their own reasons for discouraging the belief as well.
We are talking about experts in virology or other bio security-relevant domains. Those aren't necessarily experts on all the evidence about the origin of COVID19.