Why Should I Assume CCP AGI is Worse Than USG AGI?

There are some additional reasons, beyond the question of which values would be embedded in the AGI systems, to not prefer AGI development in China, that I haven't seen mentioned here:

• Systemic opacity, state-driven censorship, and state control of the media means AGI development under direct or indirect CCP control would probably be less transparent than in the US, and the world may be less likely to learn about warning shots, wrongheaded decisions, reckless behaviour, etc. True, there was the Manhattan Project, but that was quite long ago; recent examples like the CCP's suppression of information related to the origins of COVID feel more salient and relevant.
• There are more checks and balances in the US than in China, which you may think could e.g., positively influence regulation; or if there's a government project, help incentivise responsible decisions there; or if someone attempts to concentrate power using some early AGI, stop that from happening. E.g., in the West voters have some degree of influence over the government, there's the free press, the judiciary, an ecosystem of nonprofits, and so on. In China, the CCP doesn't have total control, but much more so than Western governments do.

I think it's also very rare that people are actually faced with a choice between "AGI in the US" versus "AGI in China". A more accurate but still flawed model of the choice people are sometimes faced with is "AGI in the US" versus "AGI in the US and in China", or even "AGI in the US, and in China 6-12 months later" versus "AGI in the US, and in China 3-6 months later".

To add to the discussion, my impression is that many people in the US believe they have some moral superiority or know what is good for other people. The whole "we need a manhattan project for AI" discourse is reminiscent of calling for global domination. Also, doing things for the public good is controversial in the US as it can infringe on individual freedom.

This makes me really uncertain as to which AGI would be better (assuming somebody controls it).

I think there's a deep question here as to whether Trump is "America's true self finally being revealed" or just the insane but half predictable accident of a known-retarded "first past the post" voting system and an aging electorate that isn't super great at tracking reality.

I tend to think that Trump is aberrant relative to two important standards:

(1) No one like Trump would win an election with Ranked Ballots that were properly counted either via the Schulze method (which I tend to like) or the Borda method (which might have virtues I don't understand (yet! (growth mindset))). Someone that the vast majority of America thinks is reasonable and decent and wise would be selected by either method.

I grant that if you're looking at America from the outside as a black box, we're unlikely to change our voting method to something that isn't insanely broken any time soon, and so you could hold it against the overall polity that we are dangerously bad at selecting leaders... and unlikely to fix this fast enough to matter... but in terms of the basic decency of our median voter I think that Trump isn't strong evidence that we are morally degenerate sociopaths.

In fact, Americans tend to smile a lot, and donate to charity, and are generally quite reasonable, and don't want an empire, and quite like the idea of being a fair, tolerant, prosperous, just, non-racist shining city on a hill.

Americans created Wikipedia, give it away for free, and it runs on donations. If that impulse runs from the people directly into the AGI, then that's better rather than worse. (Assuming alignment is even real. If it isn't possible/easy/whatever then it doesn't matter which country builds "the alien monster that will inevitably kill us all without remorse given that it is very powerful and doesn't love us and doesn't even understand the concept of love".)

The CCP blocks access to Wikipedia by default: you have to use a VPN, which is illegal, but also >30% of the population uses these illegal VPNs, and also some VPNs are tolerated if they install backdoors for the CCP to spy on them. Fuck that noise.

(2) The broad material intellectual history of Rights Respecting Democractic Republicanism is real, and being shat upon by Trump, but it still exists to drop into an LLM and give positive reinforcement for feeling good about that stuff and endorsing it.

America and Americans have often failed to live up to the ideals, but we also articulated those ideals, and also articulated the idea of "approximating them more and more successfully in real life over the course of history".

The White House was built by slaves, and then eventually slavery was outlawed, and black cultural integration proceeded decade by decade, in fits and starts, and eventually a descendant to slaves (though (to be clear) Michelle and Barry also had ancestors who owned slaves) moved in as the President and the First Lady. And everyone who was willing to talk about it in public was proud of this. Because the formal written ideals of our country are about god-given inalienable rights, including the right of everyone to own property and pursue happiness. The government can take your shit... but they have to do it via eminent  domain and pay you fair value for it. (I grant that, if Obama is part of the evidence about America then so is Trump. Both, in my opinion, are in some deep senses "accidents of a terribly designed voting system" but I think Obama was a happy accident, and people clapped and wrote happy things about progress and fairness and justice afterwards. That writing is part of what goes into an American LLM, by default.)

By contrast, the CCP runs Uyghur Gulags right now and basically doesn't even apologize. They want to conquer Tibet, and Taiwan, and are proud of it. They violated the treaty with the UK, whereby the UK gave up Hong Kong fair and square (according to the letter of a treaty signed long ago) after the CCP promised to grant them rights to vote for their own city government in the way they were used to under UK guidance... 

...and then there were brutal crackdowns and something like 10k people were thrown in secret prisons for trying to insist on those rights. At least Hitler was elected. The CCP weren't even elected. They seized power irregularly, through violence, authorized by the slogan political power comes our of the barrel of a gun. They still formally oppose the concept of elections. The entire idea of "consent ethics" is foreign to the logic of their system.

And if the intelligence of the governing class is of any relevance to the likelihood of a positive outcome, um, CCP seems to have USG beat hands down.

Intelligence is only a positive sign when the agent that is intelligent cares about you.

If you are certain that they would murder you and take your shit if they could get away with that somehow, then intelligence is a worrying sign, because it gives them a better chance to realize their preference of murdering you and taking your shit.

Personally, I'm in favor of establishing a world government, with a proportionally representative parliament that elects a Condorcet prime minister.

From my perspective, unboxed ASI might very well be like first contact with aliens (from platospace rather than outerspace), and the outerspace aliens generally say "take me to your leaders" when they meet humans in stories, and... currently Earth has no such people to take them to! It'd be nice to fix this error in my opinion.

But the CCP will never endorse this, whereas quite a few Americans will notice that this is consistent with our founding ideals, that many of us still cherish, and be on board with offering such influence to the people of Earth in a fair and reasonable way.

I don't think it's possible to align AGI with democracy. AGI, or at least ASI, is an inherently political technology. The power structures that ASI creates within a democratic system would likely destroy the system from within. Whichever group would end up controlling an ASI would get decisive strategic advantage over everyone else within the country, which would undermine the checks and balances that make democracy a democracy.

I've also noticed this assumption. I myself don't have it, at all. My first thought has always been something like "If we actually get AGI then preventing terrible outcomes will probably require drastic actions and if anything I have less faith in the US government to take those". Which is a pretty different approach from just assuming that AGI being developed by government $x$ will automatically lead to a world with values of government $x$. But this a very uncertain take and it wouldn't surprise me if someone smart could change my mind pretty quickly.

There's more variance within countries than between countries.  Where did the disruptive upstart that cares about Free Software^[1] come from?  China.  Is that because China is more libertarian than the US?  No, it's because there's a wide variance in both the US and China and by chance the most software-libertarian company was Chinese.  Don't treat countries like point estimates.

1. ^{^}

   Free as in freedom, not as in beer

Based on previous data, it's plausible like CCP AGI will perform worse on safety benchmarks than US AGI. Take Cisco Harmbench evaluation results:

• DeepSeek R1: Demonstrated a 100% failure rate in blocking harmful prompts  according to Anthropic's safety tests.
• OpenAI GPT-4o: Showed an 86% failure rate in the same tests, indicating better but still concerning gaps in safety measures.
• Meta Llama-3.1-405B: Had a 96% failure rate, performing slightly better than DeepSeek but worse than OpenAI.

Though, if it was just CCP making AGI or just US making AGI it might be better because it'd reduce competitive pressures. 

But, due to competitive pressures and investments like Stargate, the AGI timeline is accelerated, and the first AGI model may not perform well on safety benchmarks.

↑ comment by Stephen Fowler (LosPolloFowler) · 2025-04-24T01:53:59.536Z · LW(p) · GW(p)

You have conflated two separate evaluations, both mentioned in the TechCrunch article. 

The percentages you quoted come from Cisco’s HarmBench evaluation of multiple frontier models, not from Anthropic and were not specific to bioweapons.

Dario Amondei stated that an unnamed DeepSeek variant performed worst on bioweapons prompts, but offered no quantitative data. Separately, Cisco reported that DeepSeek-R1 failed to block 100% of harmful prompts, while Meta’s Llama 3.1 405B and OpenAI’s GPT-4o failed at 96 % and 86 %, respectively.

When we look at performance breakdown by Cisco, we see that all 3 models performed equally badly on chemical/biological safety.

Replies from: ram-potham

↑ comment by Ram Potham (ram-potham) · 2025-04-25T23:01:45.990Z · LW(p) · GW(p)

Thanks, updated the comment to be more accurate

It seems like it would depend pretty strongly on which side you view as having a closer alignment with human values generally. That probably depends a lot on your worldview and it would be very hard to be unbiased about this.

There was actually a post about almost this exact question [EA · GW] on the EA Forums a while back. You may want to peruse some of the comments there.

I'm very happy this post is getting traction, because I think spotlighting and questioning these invisible assumptions should become standard practice if we want to raise the epistemic quality of AI safety discourse. Especially since these assumptions tangibly translate into agendas and real-world policy.

I must say that I find it troubling how often I see people accept the implicit narrative that “CCP AGI < USG AGI” as an obvious truth. Such a high-stakes assumption should first be made explicit, and then justified on the basis of sound epistemics. The burden of justifying these assumptions should lie on people who invoke them, and I think AI Safety discourse's epistemic quality would benefit greatly if we called out those who fail to state + justify their underlying assumptions (a virtuous cycle, I hope)

Similarly, I think its very detrimental for terms like “AGI with Western values” or “aligned with democracy” (implied positive-valences) to circulate without their authors providing operational clarity. On this note, I think it quite important that the AI Safety community isn't co-opted by their respective governments' halo terms or applause lights [LW · GW]; let's leave it to politicians/AI company leaders to be rhetorically potent but epistemically hollow, and bar their memetic trojan horses from entering our gates. As such, I staunchly advocate for these phrases to be tabooed until defined precisely, or else they function as proxies for geopolitical affiliation (useful for political agendas) rather than as technical descriptions of value alignment architecture (useful for AI Safety's epistemic quality, ideally upstream from political agenda setting)


Perhaps more foundationally, we should take a step back and interrogate the notion of "civilizational-value binaries" that sneak into such thinking/discourse. The “Western vs Asian values” framing often relies on cached stereotypes [LW · GW]. E.g “Western = liberal/democratic” and “Asian = authoritarian/hierarchical.” 

This is where the actual empirical literature becomes useful:

A direct empirical test of this idea was conducted by Christian Welzel (2011) using World Values Survey data from 87 countries, including 15 in Asia. His study, “The Asian Values Thesis Revisited”, asked whether people in Asia show a cultural immunity to “emancipative values” like personal autonomy, gender equality, freedom of expression, and liberal democracy — even under modernization.

His findings decisively refute the claim that Asian cultures are categorically resistant to these values:

• Japan ranks above the US and UK in support for emancipative values.
• East Asians with higher education levels support liberal-democratic values just as strongly as Westerners — sometimes more so.
• The apparent East–West divide disappears once you control for knowledge development (education, access to information, scientific output).
• In multilevel regression models, the “Asia vs West” distinction becomes statistically insignificant once development indicators are accounted for.
   

TLDR: The civilizational-value binary dissolves under empirical scrutiny. What predicts support for so-called “emancipative values” is not cultural origin, but development indicators like education and access to knowledge infrastructure.

Regardless of whether one endorses these values as desirable, I think the key point is this: those “emancipative values”  are not exclusive to the West, and their presence is developmental, not civilizational. Accordingly, this breaks the frame that AGI systems trained in ‘the West’ are inherently more likely to reflect what are assumed to be morally superior goals.

If the belief that “CCP AGI < USG AGI” is riding on a fuzzy vibes logic that “Western AGI = liberal democracy values = better for... all humans on the planet(?)”, we should spotlight and scrutinize it.

Useful truth-seeking, scrutinizing questions might be:

• What values are you actually gesturing to here?
• How are these values defined, implemented during development, enforced institutionally, and ultimately reflected in AGI behavior and the governance systems surrounding it?
  • initially carried out by the actors + institutions responsible for creating and deploying the AGI, and later, by both the AGI itself and the governance structures designed to guide or constrain it
• Compared to a USG AGI, what do we actually know about the CCP’s institutional architecture, AGI-specific value-setting processes, and incentive structures etc — such that we can confidently say a CCP-developed AGI would lead to worse outcomes for... all humans across the planet(?)^[1] ?
   

Until we can answer that last question, the claim that “CCP AGI is worse” remains not an argument, but an assumption.

1. ^{^}

   When people say CCP AGI is worse, they don't often specify "worse for who". 
   I'm left to guess. At the end of the AGI day, who really wins and who really loses?

We don’t want an ASI to be ”democratic”. We want it to be ”moral”. Many people in the West conflate the two words thinking that democratic and moral is the same thing but it is not. Democracy is a certain system of organizing a state. Morality is how people and (in the future) an ASI behave towards one another.

There are no obvious reasons why an authocratic state would care more or less about a future ASI being immoral, but an argument can be made that autocratic states will be more cautious and put more restrictions on the development of an ASI because autocrats usually fear any kind of opposition and an ASI could be a powerful adversary of itself or in the hands of powerful competitors.

One very important consideration is whether they hold values that they believe are universalist, or merely locally appropriate.

For example, a Chinese AI might believe the following: "Confucianist thought is very good for Chinese people living in China. People in other countries can have their own worse philosophies, and that is fine so long as they aren't doing any harm to China, its people or its interests. Those idiots could probably do better by copying China, but frankly it might be better if they stick to their barbarian ways so that they remain too weak to pose a threat."

Now, the USA AI thinks: "Democracy is good. Not just for Americans living in America, but also for everyone living anywhere. Even if they never interact ever again with America or its allies the Taliban are still a problem that needs solving. Their ideology needs to be confronted, not just ignored and left to fester."

The sort of thing America says it stands for is much more appealing to me than a lot of what the Chinese government does. (I like the government being accountable to the people it serves - which of course entails democracy,  a free press and so on). But, my impression is that American values are held to be Universal Truths, not uniquely American idiosyncratic features, which makes the possibility of the maximally bad outcome (worldwide domination by a single power) higher.

What would it mean for an AGI to be aligned with "Democracy," or "Confucianism," or "Marxism with Chinese characteristics," or "the American Constitution"? Contingent on a world where such an entity exists and is compatible with my existence, what would my life be like in a weird transhuman future as a non-citizen in each system?

None of these philosophies or ideologies was created with an interplanetary transhuman order in mind, so to some extent a superintelligent AI guided by them, will find itself "out of distribution" when deciding what to do. And how that turns out, should depend on underlying features of the AGI's thought - how it reasons and how it deals with ontological crisis. We could in fact do some experiments along these lines - tell an existing frontier AI to suppose that it is guided by historic human systems like these, and ask how it might reinterpret the central concepts, in order to deal with being in a situation of relative omnipotence. 

Supposing that the human culture of America and China is also a clue to the world that their AIs would build when unleashed, one could look to their science fiction for paradigms of life under cosmic circumstances. The West has lots of science fiction, but the one we keep returning to in the context of AI, is the Culture universe of Iain Banks. As for China, we know about Liu Cixin ("Three-Body Problem" series), and I also dwell on the xianxia novels of Er Gen, which are fantasy but do depict a kind of politics of omnipotence. 

The state of the geopolitical board will influence how the pre-ASI chaos unfolds, and how the pre-ASI AGIs behave. Less plausibly intentions of the humans in charge might influence something about the path-dependent characteristics of ASI (by the time it takes control). But given the state of the "science" and lack of the will to be appropriately cautious and wait a few centuries before taking the leap, it seems more likely that the outcome will be randomly sampled from approximately the same distribution regardless of who sets off the intelligence explosion.

There's also the possibility that a CCP AGI can only happen through being trained on Western data to some extent (i.e., the English language internet) because otherwise they can't scale data enough. This implies that it would probably be a "Marxism with Chinese characteristics [with American characteristics]" AI since it seems like that just raises the "alignment to CCP values" technical challenge difficulty a lot.

I'm relieved not to be the only one wondering about this.

I know this particular thread is granting that "AGI will be aligned with the national interest of a great power", but that assumption also seems very questionable to me. Is there another discussion somewhere of whether it's likely that AGI values cleave on the level of national interest, rather than narrower (whichever half-dozen guys are in the room during a FOOM) or broader (international internet-using public opinion) levels?

From an individual person perspective, less authoritarian ASI is better. "Authoritarian" measure here means the degree it allows itself to restrict your freedoms. 

The implicit assumption here as I understand it is that Chinese ASI would be more authoritarian than US. It may not be a correct assumption, as US has proven to commit fairly heinous things to domestic (spying on) and foreign (mass murdering) citizens.

I'm guessing you live in a country with a US military base? Are you more free than the average Chinese citizen?

As things stand today, if AGI is created (aligned or not) in the US, it won't be by the USG or agents of the USG. I'll be by a private or public company. Depending on the path to get there, there will be more or less USG influence of some sort. But if we're going to assume the AGI is aligned to something deliberate, I wouldn't assume AGI built in the US is aligned to the current administration, or at least significantly less so than the degree to which I'd assume AGI built in China by a Chinese company would be aligned to the current CCP. 

For more concrete reasons regarding national ideals, the US has a stronger tradition of self-determination and shifting values over time, plausibly reducing risk of lock-in. It has a stronger tradition (modern conservative politics notwithstanding) of immigration and openness.

In other words, it matters a lot whether the aligned US-built AGI is aligned to the Trump administration, the Constitution, the combined writings of the US founding fathers and renowned leaders and thinkers, the current consensus of the leadership at Google or OpenAI, the overall gestalt opinions of the English-language internet, or something else. I don't have enough understanding to make a similar list of possibilities for China, but some of the things I'd expect it would include don't seem terrible. For example, I don't think a genuinely-aligned Confucian sovereign AGI is anywhere near the worst outcome we could get.

Chinese culture is just less sympathetic in general. China practically has no concept of philanthropy, animal welfare. They are also pretty explicitly ethnonationalist. You don’t hear about these things because the Chinese government has banned dissent and walled off its inhabitants.

However, I think the Hong Kong reunification is going better than I'd expect given the 2019 protests. You'd expect mass social upheaval, but people are just either satisfied or moderately dissatisfied.

I am neither an American citizen nor a Chinese citizen.

does not describe most people who make that argument.

Most of these people are US citizens, or could be. under liberalism/democracy those sorts of people get a say in the future, so  think AGI will be better if it gives those sorts of people a say. 

Most people talking about the USG AGI have structural investments in the US, which are better and give them more chances to bid on not destroying the world. (many are citizens or are in the US block). Since the US government is expected to treat other stakeholders in its previous block better than China treats members of it's block, it is better for people who are only US aligned if the US gets more powerful, since it will probably support its traditional allies even when it is vastly more powerful, as it did during the early cold war. (This was obvious last year and no longer obvious).

In short, the USG was commited to international liberalism, which is a great thing for AGI to have for various reasons which are hard to say, but basically of the form that liberals are commited to not doing crazy stuff. 

People who can't reason well about the CCP's internal ideologies /political conflicts(like me), and predict ideological alignemnt for AGI, think that USG AGI will use the frames of international liberalism (which don't let you get away with terrible things even if you are powerful), and worry about frames of international realism (which they assign to China, since they cannot tell, and argue that if you have the power you must/should/can use it to do anything, including ruining everybody else). 

As a summary, if you are not an american citizen, do not trust the US natsec framing. A lot of this is carryover from times when the US liberal international block (global international order), was stronger, and so as a block framing it is better iff the US block is somehow bigger, which at the time it was. 

USA wins on the merits of historically preferring to pretend it isn't ruling the world and mostly letting other countries do their thing, even when it has extreme military dominance (nukes)

China seems to be better at governance

On values USA is more adapted to wealth, while China has the communistic underpinnings which may be very good in a fully-automated economy.

Comes down to whether you want the easygoing less competent (and slightly psychotic) overlords or the more competent higher-strung control freaks I suppose.

I think the the assumption is that this is the USG of the last 50 years - which has flaws, but also has human rights goals and an ability to eventually change and accommodate the public’s beliefs. 

So in the scenario where AI is controlled by a strongly democratic USG, you have a much more robust “alignment” to enlightenment values and no one person with too much power. 

That said, that’s probably a flawed assumption for how the US government operates now/ over the next decade. 

Western AI is much more likely to be democratic and have humanity's values a bit higher up. Chinese one is much more likely to put CCP values and control higher up. 

But yes, if it's the current US administration specifically, neither option is that optimistic.

I think people focus too much on "would US AGI be safer than China" and not as much on "how much safer"

In the sense that US has 15% pdoom and China has 22%, this notion that everyone needs to get onboard and help US win with full effort could be bad

Could be used (and arguably is currently being used) to be even LESS safe, and empower an authoritarian mercantilist behemoth state, and possibly invade other countries for resources 

And in general massively increase and accelerate pdoom simply on the idea that our pdoom is lower than theirs 

I mostly put this question through the same filter I do the question of Chinese vs. US hegemony/empire. China has a long history of empire and knows how to do it well. The political bureaucracy in China is well developed for preserving both itself and the empire (even within changes at the top/changes of dynasty). Culturally and socially the population seems to be well acclimated to being ruled rather than seeing government as the servant of the people (which I not quite the same as saying they are resigned to abusive totalitarianism, the empire has to be providing something that amounts to public good and peace but seem more tolerant of the means applied than western cultures).

The US on the other hand pretty much sucks at empire and lacks a well functioning and well developed bureaucracy for supporting empire.

So I think perhaps one might need to ask what the specific risk one is most concerned about. China probably produces an AI that serves the Party and Empire and so perhaps is a bit more corrigible and won't decide to kill everyone. But if you're concerned about what those in control of an AGI/ASI might do with it China might be a bigger risk than the US. With the US you probably have greater risks from the AI wanting to kill everyone or simply doing that without caring but likely have more controls on what the AI will allow its "owners" to do with it.

Id guess its more likely to be good. The logic of "post scarcity utopia" is pretty far from market capitalism. Also China has been leading in open source models. Open source is a lot more aligned with humanity as a whole. 

I really like that I see more discussion of "ok even if we managed to avoid xrisk  what then?", e.g. recent papers on AI-enabled coups and so on. To the point however, I think the problem runs deeper. What I fear the most is that by "Western values imbued in AGI" people mean "we create an everlasting upperclass with no class mobility because capital is everything that matters and we freeze the capital structure, you will get UBI so you should be grateful."

It probably makes sense to keep the capitalist structure between ASIs but between humans? Seems like a very bad outcome for me (You will live in a pod and you will be happy type of endgame for the masses).

I feel the question misstates the natsec framing by jumping to the later stages of AGI and ASI. This is important because it leads to a misunderstanding of the rhetoric that convinces normal non-futurists, who aren't spending their days thinking about superintelligence.

The American natsec framing is about an effort to preserve the status quo in which the US is the hegemon. It is a conservative appeal with global reach, which works because Pax Americana has been relatively peaceful and prosperous. Anything that threatens American dominance, including giving ground in the AI race, appears dangerously destabilizing. Any risks from AI acceleration are literally after-thoughts (a problem for tomorrow, not today).

Absurd as it is, the Trumpist effort to burn the American-led system of global cooperation to the ground is still branded as a conservative return to an imagined glorious past.

The challenge in defeating this conservative natsec framing lies in communicating that radical change is all that is on the menu, but with some options far worse than others. I, for one, currently believe the fatal effect pre-AGI AI will have on democracy and other liberal values, regardless of who wields it, to be a promising rhetorical avenue that should be amplified.

In a post-scarcity world you probably want a lot of personal freedom. 

Anglo armies have been extremely unusual historically speaking for their low rates of atrocity.

(I don't think this is super relevant for AI, but I think this is where intuitions about the superiority of the west bottoms out)

I think history is a good teacher when it comes to AI in general, especially AI we did not (at least at the time of deployment, and perhaps now, do not) fully understand. 

I too feel a temptation to imagine that a USG AGI would hypothetically have alignment with US ideals, and likewise a CCP AGI would align with CCP ideals.

That said, I struggle with, given our lack of robust knowledge of what alignment with any set of ideals would look like in an AGI system, and how we could assure them, having any certainty that these systems would align with anything the USG or CCP would find desirable at all. Progress is being made in this area by, Anthropic, but I'd need to see that move forward significantly. 

One can look at current gen LLMs like DeepSeek and see that it is censored to align with CCP concepts during fine tuning, and perhaps see that as predictive. I find it doubtful that some fine tuning would be sufficient to serve as the moral backbone of an AI system that is capable of AGI. 

Which speaks to history. AI systems tend to be very aligned with what their output task is. The largest and most mature networks we have are Deep Learning Recommendation Models deployed by social media entities to keep us glued to our phones. 

The intention was to serve engaging content to people, the impact was to flood people with content that is emotionally resonant, but not necessarily accurate. That has arguably led to increased polarization, radicalization, and increased suicide rates, primarily in young women. 

While it would be tempting to say that social media companies don't care, the reality is that these DLRMs are very difficult to align. They are trained using RL and the corpus of their interactions with billions of daily users. They reward hack incessantly, and in very unpredictable ways. This leads to most of the mitigating actions being taken downstream of the recommendations (content warnings, etc.) Not out of design, but out of the simple fact that the models that are the best at getting users to keep scrolling are seldom the best at serving accurate content. 

Currently, I think both flavors of AGI present the same fundamental risks. No matter the architecture, one cannot expect human like values to emerge in AI systems inherently, and we don't understand the drivers of those values within humans particularly well and how they lead to party lines/party divisions. 

Without that understanding, we're shooting in the dark. It would be awfully embarrassing if the systems both, instead of flag waving aligned on dolphin species propagation. 

Are you genuinely unfamiliar with what is happening to the uyghurs, or is this a rhetorical question?

Judging from historical figures, the entire West represented by the United States and Europe is much worse. The things that the United States accuses China of as a whole but has no actual evidence are all things that the United States has done before. The United States' systematic genocide of Indians, the United States' large-scale network surveillance and wiretapping of leaders including European allies, and the United States' direct use of force to suppress veterans' protests. The corresponding events accused of China are the genocide of Uyghurs (although there is no complete sentence to prove that China is slaughtering Uyghurs), the security issues of China's Huawei (there is no evidence that Huawei is wiretapping), and China's Tiananmen Square incident (in fact, there was no bloodshed in Tiananmen Square).

So I also find it ridiculous that the content produced by this so-called rationalist community defaults to portraying China as the bad guy. You think you are rational, but you treat China and other ethnic groups in exactly the same way as a monotheistic religion treats infidels. You have a conclusion first and then look for arguments. Is your thinking really Less wrong?