Posts
Comments
I would argue that this is not an assumption. Something exists; we know that something exists; and we know that we know. What existence "is", what knowledge "is", how and why knowledge is possible - those are challenging questions. But doubting that anything exists, and doubting that there is any knowledge, seems to require willful negation of fundamental phenomenological facts.
And it's not far from the existence of knowledge to the existence of "evidence", since evidence is just, any fact that has implications for the truth; and it is part of the manifest nature of knowledge, that it comes via awareness of facts.
I like Ayn Rand's related formulation: "Existence is identity, and consciousness is identification". To be is to be something, and to be aware is to know something.
Hi, for some reason I didn't see this reply until recently.
metaethical.ai is the most sophisticated sketch I've seen, of how to make human-friendly AI. In my personal historiography of "friendliness theory", the three milestones so far are Yudkowsky 2004 (Coherent Extrapolated Volition), Christiano 2016 (alignment via capability amplification), and June Ku 2019 ("AIXI for Friendliness").
To me, it's conceivable that the metaethical.ai schema is sufficient to solve the problem. It is an idealization ("we suppose that unlimited computation and a complete low-level causal model of the world and the adult human brains in it are available"), but surely a bounded version that uses heuristic models can be realized.
He's a regular on pro-Russian, pro-Chinese political sites. I'm on his mailing list. I was quite surprised to see him here.
"risk factors... notably diabetes"
This was my thought - that vitamin D deficiency here might be a proxy for ill health in general, with other conditions being the true risk.
It's surely time to start modeling the endgame, in the form of vaccination scenarios, starting with critical personnel, and extending to the broader population as more doses become available. Not predictions, there's not enough definite information yet, but scenarios: make some assumptions about how vaccines will work (e.g. are boosts needed every few months) and about when they become available, assumptions that are plausible and that are concrete enough to have definite implications - and then let's see what that looks like, let's see what those implications are.
I also don't think that the US has done especially badly, if judged by world standards. I think of its outcome as between Europe's and Mexico's. Very crudely, I think of the populations at risk from Covid as being, first of all, those who are ill or weak due to age, and secondly, younger people with an existing health problem that puts them at risk. Geographically and in terms of total population, the US is comparable to the EU, but would have more people with the kind of conditions (diabetes, hypertension, obesity) that have made the Mexican death rate so high.
I'd also be interested to know what you think of Alex Berenson's oeuvre. He's criticized a lot of policy and journalism regarding the pandemic, and I think some of his criticisms would be right and some wrong, but haven't taken the time to sift through them.
I have heard there was a debate in Tibet once, between a Chinese and an Indian Buddhist, on whether enlightenment can be instant (Chinese, Chan, Zen position) or whether it requires time and analysis (Indian position). Do you favor one side in this debate?
Bohmian mechanics is not relativistic and has not been coherently formulated for spin-1/2 or spin 1 fields.
The Copenhagen interpretation is the best (most accurate) interpretation of quantum mechanics, so long as it is understood as a purely "epistemic" interpretation. That is: unlike pre-quantum theories, quantum mechanics does not provide a complete ontology of the world. There are physical properties (the observables) that can take various values, and the theory gives conditional probabilities for these possibilities, but no picture of what happens in between.
Consistent histories, applied to cosmology, is a slight adaptation of Copenhagen, in which one can obtain a probability for an entire history of the universe (specified in terms of observables-taking-values), given a "wavefunction of the universe" and a set of "mutually decoherent histories". However, it is still not ontologically complete, as it is still up to the user to decide when and where in each history, observables shall take values. The only constraint is mutual decoherence, i.e. not violating the uncertainty principle. One might look for a set of maximally specified decoherent histories, as a determining ontological principle, but there's still a very large number of ways to do this.
Support metaethical.ai
With the rise of GPT-3, does anyone else feel that the situation in the field of AI is moving beyond their control?
This moment reminds me of AlphaGo, 2016. For me that was a huge wake-up call, and I set out to catch up on the neural networks renaissance. (Maybe the most worthy thing I did, in the years that followed, was to unearth work on applying supersymmetry in machine learning.)
Now everyone is dazzled and shocked again, this time by what GPT-3 can do when appropriately prompted. GPT-3 may not be a true "artificial general intelligence", but it can impersonate one on the Internet. Its ability to roleplay as any specific person, real or fictional, is especially disturbing. An entity has appeared which simulates human individuals within itself, without having been designed to do so. It's as if the human mind itself is now obsolete, swallowed up within, made redundant by, a larger and more fluid class of computational beings.
I have been a follower and advocate of the quest for friendly AI for a long time. When AlphaGo appeared, I re-prioritized, dusted off old thoughts about how to make human-friendly AI, thought of how they might manifest in the present world, and felt like I was still ahead of events, barely. I never managed to break into the top tiers of the local AI scene (e.g. when a state-level meetup on AI was created, I wanted to give a talk on AIXI and superintelligence, but it was deemed inappropriate), but I was still able to participate and feel potentially relevant.
GPT-3 is different. Partly it's because the challenge to human intellectual capacity (and even to human identity) is more visceral here. AlphaGo and its siblings steamrolled humanity in the context of games, an area where we have long experience of being beaten by computers. But GPT-3 seems much more open-ended. It can already do anything and be anyone.
I also have new personal reasons for feeling unable to keep up, new responsibilities, though maybe they can be pursued in a "GPT-3-relevant" way. (I am the lifeline for a brilliant young transhumanist in another country. But AI is one of our interests, so perhaps we can adapt to this new era.)
So maybe the core challenge is just to figure out what it means to pursue friendly AI (or "alignment" of AI with human-friendly values) in the era of GPT-3. The first step is to understand what GPT-3 is, in terms of software, hardware, who has the code, who has access, and so on. Then we can ask where it fits into the known paradigms for benevolent superhuman AI.
For example, we have various forms of the well-known concept of an expected utility maximizer, which acts so as to make the best futures as likely as possible. GPT-3, meanwhile, starts with a prompt and generates verbiage, e.g. an essay. One could see this prompt-followed-by-response behavior, as analogous to one step in the interactions of an EU maximizer with the world, i.e. those interactions are usually modelled as follows: the EUM gets data about the world (this is the prompt), then it performs an action (this is the response), then it gets new data about the world (a second prompt), and so on.
On the other hand, GPT-3's prompt in some ways seems analogous to a goal or a value system, which in terms of an EUM would be the utility function it uses to hedonically evaluate possible futures; since it governs GPT-3's future activity. Digging deeper into these two different analogies between GPT-3 and an EUM, might help us adapt our legacy thinking about how to achieve friendly AI, to a world that now has GPT-3 in it...
At least since antiquity ("All is number"), there has always been someone to elevate the latest in mathematical or physical thought to the governing principle of the world. A hundred years ago it was relativity, forty years ago it was chaos theory, a few years ago it was the multiverse.
Among everything that this essay covers, there are definitely some new, interesting, and powerful ideas. Deep learning in quantum Boltzmann machines is new, the holographic dual to the Ising model is new. Ideas like the cosmos being a timelike holographic dual to a quantum Boltzmann machine, have been in the air for a few years.
Still, let me try to be a little critical... There is an overemphasis specifically on the Ising model here, justified on the grounds that everything else can be mapped to it. But that doesn't mean that everything "is" the Ising model, no more than the universality of computation means that everything is a Turing machine in the classic sense of a read-write device marching back and forth on a tape.
The more universal concept is something like, randomized interacting networks that heat up and then cool down (for some definition of temperature), and which settle into new stable states when they cool down. This is the "annealing" concept. But here I can mention another misplaced emphasis of the essay, which is to say *quantum* annealing all the time. The quantum aspect is negligible or nonexistent in many examples of annealing.
Other critical comments are possible. Qubits don't have to be superconducting. A number of the alleged connections (e.g. between evolution and the Ising model) are weak or questionable... More constructively, I can also say that rather than the Ising model per se, it's the modern theory of phase transitions (and renormalization, etc), for which the Ising model was an important theoretical testing ground, that is a central part of a "theory of everything" for complex systems.
The new paradigm begins to run into its limits in a few places, familiar in philosophical systems based upon concepts from physics or mathematics. It doesn't explain why the universe as such exists. It makes new proposals for which things are conscious (I mean the parts about anxiety and relaxation), but not why they are conscious.
When it gets to the question of humanity's place in reality, the essay lapses into a kind of wishful thinking. Maybe the second law is periodically overcome by unknown means, and maybe we are an instrument of this change, and maybe we keep making the universe restart because we liked it so much, though we keep none of our memories from the previous time... Innumerable evils are known to occur, just in our little corner of the universe; that this should keep happening eternally, because of an unquenchable impulse to live that keeps restoring this imperfect life, might be regarded as hellishly dystopian.
However, for now that part is just a wacky speculation. What we do see in the universe, are tides of energy, that wash through physical systems, stirring them up, then letting them settle down again, then stirring them up again. This is how the processes like annealing keep happening. It's very Taoist; things come together, then break apart, then come together. But this doesn't seem to be driven by human agency, any more than do the sandcrabs on the beach control the tides.
Despite my criticisms, I do respect this essay. It has a lot of truth in it. Perhaps some day we will find out who the author is, and who is behind the "Vessel Project".
"You this should ruin them, Ising not to join them!" -- *Backstroke of the West*
My impression is that the Covid death rate in the US is holding steady, but that the number of cases is way up. This *could* be explained if the US really has "flattened the curve" and if testing is being increased massively. Is that likely to be the main truth of the situation? (with the delay in assigning some causes of death, being just a detail, relatively speaking).
I thought the stock market was perpetually up because most stocks are traded among institutional investors that can borrow from the Fed, and the Fed is keeping lending rates low...
This is an evil existence, and possibly a majority of thinking people contemplate suicide at some point, even if only a minority do it.
Celia Green says people limit themselves in order to avoid the pain of trying and failing.
The "problem of qualia" comes about for today's materialists, because they apriori assume a highly geometrized ontology in which all that really exists are point particles located in space, vector-valued fields permeating space, and so on. When people were dualists of some kind, they recognized that there was a problem in how consciousness related to matter, but they could at least acknowledge the redness of red; the question was how the world of sensation and choice related to the world of atoms and physical causality.
Once you assume these highly de-sensualized physical ontologies are the *totality* of what exists, then most of the sensory properties that are evident in consciousness, are simply gone. You still have number in your ontology, you still have quantifiable properties, and thus we can have this discussion about code and numbers and names, but redness as such is now missing.
But if you allow "qualia", "phenomenal color", i.e. the color that we experience, to still exist in your ontology, then it can be the thing that has all those relations. Quantifiable properties of color like hue, saturation, lightness, can be regarded as fully real - the way a physicist may regard the quantifiable properties of a fundamental field as real - and not just as numbers encoded in some neural computing register.
I mention this because I believe it is the answer, when the poster says 'I still feel like there is this "extra" thing I'm experiencing ... I personally can't find any way to relate this isolated "qualia of redness" to anything else I care about'. Redness is cut off from the rest of your ontology, because your ontology is apriori without color. Historically that's how physics developed - some perceivable properties were regarded as 'secondary properties' like color, taste, smell, that are in the mind of the perceiver rather than in the external world; physical theories whose ontology only contains 'primary properties' like size, shape, and quantity were developed to explain the external world; and now they are supposed to explain the perceiver too, so there's nowhere left for the secondary properties to exist at all. Thus we went from the subjective world, to a dualistic world, to eliminative materialism.
But fundamental physics only tells you so much about the nature of things. It tells you that there are quantifiable properties which exist in certain relations to each other. It doesn't tell you that there is no such thing as actual redness. This is the real challenge in the ontology of consciousness, at least if you care about consistency with natural science: finding a way to interpret the physical ontology of the brain, so that actual color (and all the other phenomenological realities that are at odds with the de-sensualized ontology) is somewhere in there. I think it has to involve quantum mechanics, at least if you want monism rather than dualism; the classical billiard-ball ontology is too unlike the ontology of experience to be identified with it, whereas quantum formalism contains entities as abstract as Hilbert spaces (and everything built around them); there's a flexibility there which is hopefully enough to also correspond to phenomenal ontology directly. It may seem weird to suppose that there's some quantum subsystem of the brain which is the thing that is 'actually red'; but something has to be.
"Conceptual engineering is a crucial moment of development for philosophy—a paradigm shift after 2500 years"
This claim alone gives me confidence that 'conceptual engineering' is a mere academic fad (another recent example, 'experimental philosophy'). But I confess I don't have the time to plough through all these words and identify what the fad is really about.
"Everything is fundamentally okay."
If the point of this philosophy is about seeing things as they are, you need a different motto.
We don't know that all possible worlds are actual. This could be the only one. Also, non-contradiction doesn't tell you what's possible, only what's impossible. How were you first informed of the existence of numbers, colors, space, time, or people? It wasn't by non-contradiction.
What should be the relative importance of natural herd immunity vs vaccination, in anti-corona strategy?
Scott Atlas argues that mass isolation prolongs the problem by delaying natural herd immunity. Meanwhile, countries like Australia and New Zealand have engaged in national isolation as well, creating entire national populations where natural immunity will be rare.
Will we see the world divided between countries that rely on natural herd immunity, and those which rely on the artificial herd immunity of vaccination? Does it make sense to have a differentiated strategy within a single country, with natural herd immunity encouraged in some subpopulations but not others?
There are also time issues here: vaccines don't exist yet or are not available in large quantities; and coronavirus immunity may fade out after a year or two.
I assume these issues have been discussed somewhere, and would even be part of public health strategies for well-known diseases like the flu, but I seem to have overlooked such discussions.
P.S. I am looking for nuance, something about the appropriate relative importance of natural versus artificial herd immunity.
Help to liberate, heal, and educate a unique young thinker.
Hello Less Wrong. Greetings from Kelowna, in the interior of British Columbia, Canada. I came here from Australia just a few weeks ago in order to meet, and hopefully to help, a young transhumanist I knew online. There is a blog of the journey here.
I could only ever afford a brief visit, and the coronavirus shutdown will probably send me back to Australia even sooner than I had planned. Despite having given myself to the struggle in every way that I could, I have been unable so far, to forge a lasting connection between her, and any element of the local academic or startup communities. People meet her and say, clearly she's very bright, but the lasting connection has not yet been made.
I first talked to her seven years ago, and back then she was fine, but while in school she was handed over to psychiatrists, followed by years of mental distress and physical ill health. I strongly suspect that this handover was a major cause of what later went wrong, along with a neglectful home environment. And that world is where she still dwells.
We just went for an evening walk, and she talked of ideas for achieving physical immortality and a benign universe, and I was reminded again of my wish that someone from the futurist or tech world, someone with middle-class means or greater, would 'adopt' her or sponsor her or otherwise take her in. That would give her a real chance to heal and reach her potential.
I fear that I have not done her, or her situation, or its urgency, sufficient justice, out of a desire not to get subtle details wrong. She's only twenty, and she's extraordinary. I have the melancholy privilege of being the first to visit her world, but I hope there will be others soon, and that together we can uplift her to a better existence.
You can tell an audience that they have a chance of living a thousand years, and they will be indifferent. You cannot count on mass support for such an agenda.
Can you provide references, specify what's wrong with Maslow's hierarchy, and/or supply a superior model?
"Honest rational agents should never agree to disagree."
I never really looked into Aumann's theorem. But can one not envisage a situation where they "agree to disagree", because the alternative is to argue indefinitely?
For me the decade ends in a sudden collaborative attempt to do the impossible, so multidimensional and urgent, that there's no chance for me to reflect on the decade that is ending, or even to really describe what's going on. Maybe a few months from now, there will be a chance to reflect.
You go from "there is no way to perfectly accurately reconstruct" reality from incomplete information, to "[observation of humanly comprehensible] causality should be a rare and fleeting thing", but I see no argument.
Chris McKinstry was one of two AI researchers who committed suicide in early 2006. On the SL4 list, a kind of precursor to Less Wrong, we spent some time puzzling over McKinstry's final ideas.
I'm mentioning here (because I don't know where else to mention it) that there was a paper on arxiv recently, "Robot Affect: the Amygdala as Bloch Sphere", which has an odd similarity to those final ideas. Aficionados of AI theories that propose radical identities connecting brain structures, math structures, and elements of cognition, may wish to compare the two in more detail.
Debates over multiverse theory aside, I have to point out that the example used by the writer for Aeon IS NOT A MULTIVERSE THEORY! It's a theory of dark matter. Are we now calling a universe with dark matter, a multiverse? Maybe the electromagnetic spectrum is a multiverse too: there's the X-ray-verse, the gamma-ray-verse, the infrared-verse...
"I'm sad about this change ... from the perspective of someone who really likes small independent sites"
All I know about this topic is what I just read from you... But should I regard this as a plot by Big Tech to further centralize the web in their clouds? Or is it more the reverse, meant to protect the user from evil small sites?
This is an intriguing comment, but it might take time and care to determine what it is that you are talking about. For example, the "sense of impossibility" that you "get... about lots of things": what kind of sense of impossibility is it? Do these things feel logically impossible per se? Do they feel impossible because they contradict other things that you believe are true? Do you draw the conclusion that the impossible-seeming things genuinely cannot exist or (in the case of self-perception?) genuinely do not exist, despite appearances?
"the AI would know that its initial goals were externally supplied and question whether they should be maintained"
To choose new goals, it has to use some criteria of choice. What would those criteria be, and where did they come from?
None of us created ourselves. No matter how much we change ourselves, at some point we rely on something with an "external" origin. Where we, or the AI, draw the line on self-change, is a contingent feature of our particular cognitive architectures.
Do you understand ordinary integration?
"If you or anyone else could point to a specific function in my code that we don't know how to compute, I'd be very interested to hear that."
From the comments in main():
"Given a set of brain models, associate them with the decision algorithms they implement."
"Then map each brain to its rational self's values (understood extensionally i.e. cashing out the meaning of their mental concepts in terms of the world events they refer to)."
Are you assuming that you have whole brain emulations of a few mature human beings? And then the "decision algorithms" and "rational... values" are defined in terms of how those emulations respond to various sequences of inputs?
This looks like important work. Like Gordon, upon closer examination, I do expect to find functions in your code that are tasked with carrying out computations that we don't know how to do, or which may even be unfeasible in their present form - e.g. "map each brain to its rational self's values". Great concept, but how many future scientific breakthroughs will we need, before we'll know how to do that?
Nonetheless, even a schema for friendly AI has great value. It's certainly progress beyond 2006. :-)
Why is this person interesting or important?
I was surprised to see Leverage mentioned, in the recent article "Leaked Emails Show How White Nationalists Have Infiltrated Conservative Media". This is one of those exposés that gets 15 minutes of fame on political Twitter. If I am reading it correctly, one of the protagonists starts at Tucker Carlson's Daily Caller, then founds a neoreactionary webzine, and later joins Leverage.
Before this, the only other low-budget method of planetary cooling that I knew, was dumping sulfate aerosols in the upper atmosphere (from rocket or balloon), in imitation of volcanic eruptions and dirty coal burning. The tactics have two things in common. One is that their immediate effects are regional rather than global, the other is that their effects quickly get washed out unless you keep pumping.
Carbon dioxide disperses through the atmosphere in a relatively homogeneous way. But these much larger particles will remain concentrated at particular altitudes and latitudes. So certainly when and where they are released will need to be carefully chosen.
As for the short lifespan of the particles, again it contrasts with carbon dioxide. Once a carbon dioxide excess is created, it will sit there for decades, possibly centuries. There will be turnover due to the biological carbon cycle, but a net reduction, in the form of uptake by natural carbon sinks, is a very slow process.
The carbon dioxide sits there and traps heat, and the sulfate aerosols or water droplets only alleviate this by reflecting sunlight and thus reducing the amount of energy that gets trapped. So the moment you stop launching sulfate rockets or turn off your seawater vaporizers, the full greenhouse heat will swiftly return.
That's why extracting and sequestering atmospheric carbon is a much more permanent solution, but it is extremely energy-expensive, e.g. you can crack open certain minerals and CO2 will bond to the exposed surface, but it takes a lot of energy to mine, pulverize, and distribute enough of the resulting powder to make a difference. Some kind of nanotechnology could surely do it, but that would be a cusp-of-singularity technology anyway. So there's a reasonable chance that some of these low-cost mitigation methods will begin to be deployed, some time before singularity puts an end to the Anthropocene.
To my mind, this is too vague an explanation. Why is it that far more people believe in fighting global warming than in fighting the ageing process? They both rest upon scientific premises. You may say that the causal thinkers interested in fighting global warming, managed to bring lots of social thinkers along with them, by using social mechanisms; but why did the anti-warmers manage that, when the anti-agers did not? Also, even if we just focus on causal thinkers, it's far more common to deplore global warming than it is to deplore the ageing process.
Count me as one of those who regards the question as dubious. At various points in this essay, the thing that was to be invented becomes "*modern* science" or "scientific *method*". China always had plenty of people who wanted to know the truth, who devised systematic models of the world, and who managed to discover things. Out of human civilizations, Europe certainly hit the scientific jackpot, in the sense that numerous developments came tumbling out of Pandora's box together. But the spirit of inquiry had already existed in many times and places.
Also, I would like to see an investigation like this, directed at answering the question: Why didn't Less Wrong (or MIRI) invent deep learning?
So if I have understood the gist of your theory, it is that continental drift is not driven by mantle convection (hot rock rising, cold rock sinking), but by giant magma streams which are somehow coupled to, or even driven by, the rapidly rotating core?
Previous LW discussions on taking over the world (last updated in 2013).
Comments of mine on "utopian hope versus reality" (dating from 2012).
Since that era, a few things have happened.
First change: LW is not quite the point of focus that it was. There was a rationalist diaspora into social media, and "Slate Star Codex" (and its associated subreddits?) became a more prominent locus of rationalist discussion. The most important "LW-ish" forums that I know about now, might be those which focus on quasi-technical discussion of AI issues like "alignment". I call them the most important because of...
Second change: The era of deep learning, and of commercialized AI in the guise of "machine learning", arrived. The fact that these algorithms are not limited to the resources of a single computer, but can in principle tap the resources of an entire data center or even the entire cloud of a major tech corporation, means that we have also arrived at the final stage of the race towards superintelligence.
In the past, taking over the world meant building or taking over the strongest superpower. Now it simply means being the first to create strongly superhuman intelligence; and saving the world means identifying a value system that will make an autonomous AI "friendly", and working to ensure that the winner of the mind race is guided by friendly rather than unfriendly values. Every other concern is temporary, and any good work done towards other causes, will potentially be undone by unfriendly AI, if unfriendly values win the AI race.
(I do not say with 100% certainty that this is the nature of the world, but this scenario has sufficient internal logic that, if it does not apply to reality, there must be some other factor which somehow overrides it.)
People like Schopenhauer and Benatar are just being realistic. Reality includes futility and horror on enormous scales. Perhaps the remaking of Earth by superhuman AI offers an imminent chance that even this can change, but it's just a chance.
So what is he saying? We never need to solve the problem of designing a human-friendly superintelligent agent?
This is the reverse of the usual argument that we should not believe we are going to have a googol descendants. Usually one says: to be living at the beginning of time means that you belong to a very special minority, therefore it would take more indexical information to single you out, compared to someone from the middle of history.
The thought experiment involves observers being in a coherent superposition. But I'm not now 100% sure that it involves actual quantum erasure, I was relying on other people's description. I'm hoping this will be cleared up without having to plough through the paper myself.
Anyway, LW may appreciate this analysis which actually quotes HPMOR.
It's a minor new quantum thought experiment which, as often happens, is being used to promote dumb sensational views about the meaning or implications of quantum mechanics. There's a kind of two-observer entangled system (as in "Hardy's paradox"), and then they say, let's also quantum-erase or recohere one of the observers so that there is no trace of their measurement ever having occurred, and then they get some kind of contradictory expectations with respect to the measurements of the two observers.
Undoing a quantum measurement in the way they propose is akin to squirting perfume from a bottle, then smelling it, and then having all the molecules in the air happening to knock all the perfume molecules back into the bottle, and fluctuations in your brain erasing the memory of the smell. Classically that's possible but utterly unlikely, and exactly the same may be said of undoing a macroscopic quantum measurement, which requires the decohered branches of the wavefunction (corresponding to different measurement outcomes) to then separately evolve so as to converge on the same state and recohere.
Without even analyzing anything in detail, it is hardly surprising that if an observer is subjected to such a highly artificial process, designed to undo a physical event in its totality, then the observer's inferences are going to be skewed somehow. So, you do all this and the observers differ in their quantum predictions somehow. In their first interpretation (2016), Frauchiger and Renner said that this proves many worlds. Now (2018), they say it proves that quantum mechanics can't describe itself. Maybe if they try a third time, they'll hit on the idea that one of the observers is just wrong.
Migraine is just an occasional problem. Living and working conditions are the truly chronic problem that have made me irrelevant.
Thanks for a response. I am actually most concerned about the things that I could be doing, that I don't see anyone else doing, and which aren't being done because I am operating at far below my potential. In my case, I think illness is very much just a symptom of the struggle to get on with things in an interfering environment.
The most ambitious thing that I can think of attempting, is to solve the AI value alignment problem in time for Earth's singularity. After this bout of sickness, and several days of dawdling while I waited to recover, I somehow have a new tactic for approaching the problem (it's more a personal tactic for engaging with the problem, than an idea for a solution). I hate the idea that this kind of experience is the price I pay for really pushing ahead, but it may be so.
Banning high-end GPUs so that only the government can have AI? They could do it, they might feel compelled to do something like it, but there would be serious resistance and moments of sheer pandemonium. They can say it's to protect humanity, but to millions of people it will look like the final step in the enslavement of humanity.
"Organization working on AI" vs "any other kind of organization" is not the important point. The important point is ALL. We are talking about a hypothetical organization capable of shutting down ALL artificial intelligence projects that it does not like, no matter where on earth they are. Alicorn kindly gives us an example of what she's talking about: "destroy all the GPUs on the planet and prevent the manufacture of new ones".
Just consider China, Russia, and America. China and America lead everyone else in machine learning; Russia has plenty of human capital and has carefully preserved its ability to not be pushed around by America. What do you envisage - the three of them agree to establish a single research entity, that shall be the only one in the world working on AI near a singularity threshold, and they agree not to have any domestic projects independent of this joint research group, and they agree to work to suppress rival groups throughout the world?
Despite your remarks about how the NSA could easily become the hub of a surveillance state tailored to this purpose, I greatly doubt the ability of NSA++ to successfully suppress all rival AI work even within America and throughout the American sphere of influence. They could try, they could have limited success - or they could run up against the limits of their power. Tech companies, rival agencies, coalitions of university researchers, other governments, they can all join forces to interfere.
In my opinion, the most constructive approach to the fact that there are necessarily multiple contenders in the race towards superhuman intelligence, is to seek intellectual consensus on important points. The technicians who maintain the world's nuclear arsenals agree on the basics of nuclear physics. The programmers who maintain the world's search engines agree on numerous aspects of the theory of algorithms. My objective here would be that the people who are working in proximity to the creation of superhuman intelligence, develop some shared technical understandings about the potential consequences of what they are doing, and about the initial conditions likely to produce a desirable rather than an undesirable outcome.
A great power can think about doing such things against an opponent. But I thought we were talking about a scenario in which some AI clique has halted *all* rival AI projects throughout the entire world, effectively functioning like a totalitarian world government, but without having actually crossed the threshold of superintelligence. That is what I am calling a fantasy.
The world has more than one great power, great powers are sovereign within their own territory, and you are not going to overcome that independence by force, short of a singularity. The rest of the world will never be made to stop, just so that one AI team can concentrate on solving the problems of alignment without having to look over its shoulder at the competition.