Why don't we vaccinate people against smallpox any more? 2021-04-21T00:08:31.593Z
AI Winter Is Coming - How to profit from it? 2020-12-05T20:23:51.309Z
Implications of the Doomsday Argument for x-risk reduction 2020-04-02T21:42:42.810Z
How to Write a News article on the Dangers of Artificial General Intelligence 2020-02-28T02:14:48.419Z
What will quantum computers be used for? 2020-01-01T19:33:16.838Z
Anti-counterfeiting Ink - an alternative way of combating oil theft? 2019-10-19T23:04:59.069Z
If you had to pick one thing you've read that changed the course of your life, what would it be? 2019-09-14T17:50:45.292Z
Simulation Argument: Why aren't ancestor simulations outnumbered by transhumans? 2019-08-22T09:07:07.533Z


Comment by maximkazhenkov on Evaluating 2021 ACX Predictions · 2022-01-25T22:30:28.764Z · LW · GW

If it’s versus death it would evaluate to false, so Scott meant infection, and then Omicron happened. Note that with a booster you’re back over 50% effective, and that’s now considered full vaccination via an existing vaccine (and was before Omicron), so I don’t think this grades all that obviously, and I’d evaluate it to [MU].

Evaluating to [TRUE] would make more intuitive sense to me as downstream prediction tasks would assume this prediction to mean "resistance to infection from being April-2021-definition fully-vaccinated".

Comment by maximkazhenkov on Is AI Alignment a pseudoscience? · 2022-01-23T23:18:08.948Z · LW · GW

I agree, I wouldn't consider AI alignment to be scientific either. How is it a "problem" though?

Comment by maximkazhenkov on Omicron Post #13: Outlook · 2022-01-12T09:36:19.552Z · LW · GW

True, but none of these variants made it in China either, and vaccines could only have helped stopping Delta due to timing. Sinovac is also much less effective even for Delta. Maybe China has run out of dakka and Omicron will be the final straw, but it is far from obvious imo (especially considering China has been able to keep its economy running normally all this time).

If China does break though, this will be an important datapoint for future biodefense. It roughly represents how much infectiousness humanity can handle at most if it got really, really serious.

Comment by maximkazhenkov on Omicron Post #13: Outlook · 2022-01-11T22:58:09.196Z · LW · GW

What, in your mind, makes Omicron hopeless to stop in China? Omicron doesn't seem to be much more infectious among the unvaccinated than previous strains, and they did manage to wipe out the original Wuhan strain completely within their borders.

Comment by maximkazhenkov on Moore's Law, AI, and the pace of progress · 2021-12-12T09:19:10.163Z · LW · GW

I would be shocked if fusion provides >10% of electricity to any major economy in the 2030s, like cold-fusion-is-possible-level shocked. On the one hand, the technologies new fusion start-ups are working with are obviously much more plausible than cold fusion, on the other hand there are a LOT of likely ways for fusion to fail besides just technical problems, so my intuition tells me it's a toss-up. 

I don't know nearly as much about solar PV so my confidence intervals there are much wider. I agree that if there was sufficient economic incentive, we could scale to incredible amounts of compute right now, crypto mining shows an empirical lower bound to that ability.

Comment by maximkazhenkov on Moore's Law, AI, and the pace of progress · 2021-12-12T08:54:26.194Z · LW · GW

On the technology readiness level, I put reversible computing somewhere between von Neumann probes and warp drive. Definitely post-Singularity, likely impossible.

Comment by maximkazhenkov on Moore's Law, AI, and the pace of progress · 2021-12-11T23:10:18.406Z · LW · GW

There are ways to get more energy efficiency by spending more on lower-clocked hardware, or by using a larger memory:compute ratio, and there are also hardware architectures with plausible significant power advantages.

As far as I understand, we're only 3 orders of magnitude away from Landauer limit, which doesn't leave a lot of room to squeeze efficiency out of. On the supply side, fusion doesn't seem like a relevant factor before 2050 unless an alternative approach takes us by surprise. Solar PV efficiency is already on the OOM of 1, so any advances have to come from reduction in production and maintenance costs (which is plausible for all I know).

Comment by maximkazhenkov on [Linkpost] Chinese government's guidelines on AI · 2021-12-11T22:25:11.889Z · LW · GW

It's disappointing because China's high degree of centralization and disregard for privacy, despite all its drawbacks, would at least offer some major advantages in combating AI risk. But from the wording of this document I don't get the sense that China is seriously considering AI risk as a threat to its national security.

A serious attempt would look more like "put in place a review structure that identifies and freezes all AI research publications which have potentially serious implications with regard to AI risk and turn it into a state secret if necessary".

The fact that the governments of major countries are starting to address AI X-risk - is both joyous and frightening

As far as can tell, this is simply not true - this is not what it looks like for a government to be genuinely concerned with a problem, even if it's just a small bit of concern. This is not how things in China gets done. If you've delved into Chinese bureaucratic texts before, this is what a typical politically correct, hollow fluff piece looks like.

Comment by maximkazhenkov on [Linkpost] Chinese government's guidelines on AI · 2021-12-10T22:59:08.925Z · LW · GW

This is not-even-wrong-level hollow, equivalent to "do good things, don't do bad things".

Comment by maximkazhenkov on Are big brains for processing sensory input? · 2021-12-10T16:37:56.862Z · LW · GW

I'd be very surprised if sensory discrimination correlated with  at a rate of 1.0. That's higher than IQ tests. If the correlation was really that high we wouldn't need IQ tests at all. We could just use sensory discrimination instead.

There doesn't seem to be contradiction here; so what if we could, we simply don't. Not defending the idea here, just noting that prevalence of IQ tests is very weak evidence.

Comment by maximkazhenkov on Counting Lightning · 2021-12-09T13:23:47.338Z · LW · GW

Agreed. The story is very well written in terms of literary quality.

Comment by maximkazhenkov on Counting Lightning · 2021-12-07T19:12:01.355Z · LW · GW

Propaganda manipulation has existed long before social media and were instrumental in creating the worst atrocities of the 20th century, in this regard I fail to see what's so importantly different about using AI to do it. In particular, I do not expect advanced language models to be especially effective at it, at least for long. Instead, I expect GPT-spamming will - if scaling laws hold and GPT-n actually turns out to be technically impressive - lead to a rapid decline of social media usage and anonymous communication in general (which would paradoxically be a good thing imo). You can't dump a million tons of gold on the market and expect the price to hold, even if it's really, really authentic gold.

I find the point about drones enabling genocide at unprecedented scale is the much more important one. Unfortunately, I think the story fails to capture this point since China is just about the worst setting for the story, a country that A) is already a big global player even without considering drones/AI and B) already has the capacity to tightly control its populace and carry out atrocities through human operators.

I think a story that better demonstrates the game-changing effect of drones is one where a previously unremarkable group/organization suddenly acquires unexpectedly large influence over the world through violent means, and doing so while bypassing the traditional requirement of having to use social technology to control a large number of human actors to do your bidding. The recent real-world conflict of Nagorno-Karabakh comes closer as an example.

Comment by maximkazhenkov on Yudkowsky and Christiano discuss "Takeoff Speeds" · 2021-11-25T19:36:10.483Z · LW · GW

Here's a source on horse population in the US

But also, your first graph covers a time period of 200 years whereas the third graph only covers 13; that's not even the same order of magnitude. If you zoom in enough, any curve looks smooth, even an AI that FOOMs in mere hours.

Also, the original quote is stating something about sharp increases in output once the last human bottleneck is gone, not how gradual human elements are being removed.

Comment by maximkazhenkov on AI Tracker: monitoring current and near-future risks from superscale models · 2021-11-25T17:46:10.710Z · LW · GW

Interesting; I hadn't heard of DreamerV2. From a quick look at the paper, it looks like one might describe it as a step on the way to something like EfficientZero. Does that sound roughly correct?

Yes. They don't share a common lineage, but are similar in that they're both recent advances in efficient model-based RL. Personally speaking, I think this is the subfield to be closely tracking progress in, because 1) it has far-reaching implications in the long term and 2) it has garnered relatively little attention compared to other subfields.

We may extend this to older models in the future. But our goal right now is to focus on these models' public safety risks as standalone (or nearly standalone) systems.

I see. If you'd like to visualize trends though, you'll need more historical data points, I think.

Comment by maximkazhenkov on AI Tracker: monitoring current and near-future risks from superscale models · 2021-11-25T11:35:17.577Z · LW · GW

DreamerV2 seems worthy of inclusion to me. In general, it would be great to see older models incorporated as well; I know this has been done before but having it integrated in a live tracker like yours would be super convenient as a one-stop shop of historical context. It would save people from the temptation of making lots of new lists every time a new model gets released.

Comment by maximkazhenkov on What would we do if alignment were futile? · 2021-11-20T16:55:49.718Z · LW · GW

That's how you turn a technical field into a cesspit of social commentary and political virtue signaling.

Think less AGI-Overwatch committee or GPU-export ban and more "Big business bad!", "AI racist!", "Human greed the real problem!"

Comment by maximkazhenkov on The Emperor's New Clothes: a story of motivated stupidity · 2021-11-20T16:35:47.256Z · LW · GW

Moloch reigns, as usual.

Comment by maximkazhenkov on Ngo and Yudkowsky on AI capability gains · 2021-11-20T04:49:37.543Z · LW · GW

Welcome to 2021, where 1984 is Utopian fiction.

Comment by maximkazhenkov on Why I am no longer driven · 2021-11-18T23:37:34.372Z · LW · GW

Am I allowed to swear?

Fuck yeah, but please don't say "irregardless"

Comment by maximkazhenkov on Against the idea that physical limits are set in stone · 2021-11-17T23:19:24.010Z · LW · GW

I'd give it a 60% chance that quantum mechanics was the last fundamental physics discovery that has any practical implications.

Comment by maximkazhenkov on Against the idea that physical limits are set in stone · 2021-11-17T23:01:06.211Z · LW · GW

1 million Kelvin doesn't even get close to the limit of known physics, LHC generates collisions equivalent to quadrillions of Kelvin, and quantum gravity is another trillion times that. To a physicist looking for new fundamental physics in a 1 million Kelvin plasma sounds about as random a suggestion as chanting spells. In terms of observational science, only cosmology still has the potential to uncover new fundamental physics, which is way beyond this galaxy or galaxy cluster. This ain't the 17th century no more.

Comment by maximkazhenkov on Why I am no longer driven · 2021-11-17T18:19:26.116Z · LW · GW

What you advise might work for some, but for others suck forced action would actually make the situation worse! I know this has been the case for me at times: forcing myself to "grind" actually made the problem worse over time rather than better.

I advised no such thing, notice the /s at the end.

Absolutely. Evidence: me and lots of people claiming this.

If anecdotal evidence was the standard to be judged by, alternative medicine would be bloody miracle cures - plenty of patients swear it works. And in the absence of empirical data, it's your anecdotal evidence against my anecdotal evidence. I had no intention of being charitable as I think it's a complete snake-oil industry, Tai Lopez & Co. just made it ridiculously obvious in recent years. It doesn't even require practitioners to be consciously ill-intentioned.

Of course, if you see the emotional porn aspect as the product in itself, then there's nothing wrong with it. But I doubt that's what buyers were looking for.

Comment by maximkazhenkov on Why I am no longer driven · 2021-11-17T05:20:49.732Z · LW · GW

Motivational videos, speeches and self-help books are essentially modern forms of letters of indulgence - first, they convince you that you're a "sinner", that your life sucks, that you have achieved nothing but failure and are inadequate as a human being. But lo! For the low, low price of $24.99 and a healthy dose of metaphorical self-flagellation, you get to experience the warm fuzzy feeling of redemption and rebirth as a better self.

I'm sure if you just got up 2 hours earlier every day, worked harder, became more disciplined - if you embraced the grind - you could have made it into nobility as a medieval peasant. /s

Reality doesn't give two shits about your penitence.

Comment by maximkazhenkov on What would we do if alignment were futile? · 2021-11-14T23:06:24.738Z · LW · GW

If destroying GPUs is the goal, there seem to be a lot simpler, less speculative ways than nanomachines. The semiconductor industry is among the most vulnerable, as the pandemic has shown, with an incredibly long supply chain that mostly consists of a single or a handful of suppliers, defended against sabotage largely by "no one would actually do such a thing".

Of course that is assuming we don't have a huge hardware overhang in which case current stockpiles might already be sufficient for doom, or that ASI will be based heavily on GPU computing at all.

Comment by maximkazhenkov on What would we do if alignment were futile? · 2021-11-14T18:56:48.406Z · LW · GW

Second, US export controls can buy time by slowing down the whole field.

I think this will have the opposite effect. Restricting supply of hardware will only further incentivize efficiency-focused research, which I think is much more critical on the path to AGI than "stack moar layers".

Comment by maximkazhenkov on My current uncertainties regarding AI, alignment, and the end of the world · 2021-11-14T18:45:03.513Z · LW · GW

Ok, perhaps I was too combative with the wording. My general point is: Don't think of humanity as a coordinated agent, don't think of "AGI" as a single tribe with particular properties (I frequently see this same mistake with regard to aliens), and in particular, don't think because a specific AI won't be able or want to destroy the world, that therefore the world is saved in general.

Comment by maximkazhenkov on What would we do if alignment were futile? · 2021-11-14T18:31:07.629Z · LW · GW

In the same way that we have an existence-proof of AGI (humans existing) we also have a highly suggestive example of something that looks a lot like alignment (humans existing and often choosing not to do heroin)

That's not an example of alignment, that's an example of sub-agent stability, which is assumed to be true due to instrumental convergence in any sufficiently powerful AI system, aligned or unaligned.

If anything, humanity is an excellent example of alignment failure considering we have discovered the true utility function of our creator and decided to ignore it anyway and side with proxy values such as love/empathy/curiosity etc.

Comment by maximkazhenkov on What would we do if alignment were futile? · 2021-11-14T18:20:16.207Z · LW · GW

It would need to be the sort of harm that is highly visible and concentrated in time and space, like 9/11, and not like "increasing the incidence of cancer worldwide by 10%" or "driving people insane with polarizing news feed".

Comment by maximkazhenkov on My current uncertainties regarding AI, alignment, and the end of the world · 2021-11-14T17:57:18.364Z · LW · GW

I feel like there's a difference between "modeling" and "statistical recognition", in the sense that current (and near-future) AI systems currently don't necessarily model the world around them.

There is an entire subfield of ML called model-based reinforcement learning.

You'd think that to destroy a world, you first need to have a model of it, but that may not be the case.

Natural selection is existence proof (minus anthropic effects) that you can produce world-altering agents without explicitly using models.

There may be a sense in which generating text and maneuvering the real world are very different. 

Well yes, which is why I'm less worried about GPT-3 than EfficientZero.

There may be a sense in which successfully imitating human speech without a "model" or agency is possible.

It is trivially true, and trivially false if you ask the AI adversarial questions that require AGI-completeness.

There may also be such strongly (or even more strongly) binding constraints that prevent even a superintelligent agent from achieving their goals, but which aren't "defects" in the agent itself, but in some constant in the universe. One such example is the speed of light. However intelligent you are, that's a physical constraint that you just can't surpass.

Sure, but one does not need to surpass the speed of light to destroy humanity

There may also be a sense in which AI systems would not self-improve further than required for what we want from them. Meaning, we may fulfill our needs (for which we design and produce AI systems) with a class of AI agents that stop receiving any sort of negative feedback at a certain level of proficiency or ability. 

Who is "we"? What is the mechanism by which any AI outside this class will be completely and permanently prevented from coming into existence? This is my criticism for the rest of the points as well. Your strategy for AI risk seems to be "Let's not build the sort of AI that would destroy the world", which fails at the first word:  "Let's".

Comment by maximkazhenkov on Against the idea that physical limits are set in stone · 2021-11-14T17:18:45.823Z · LW · GW

The hope that such a breakthrough might lead to effectively FTL travel is quite dashed by the lack of astrophysical observations that would hint at anything happening superluminally, even though the energies that are achieved in many observed natural phenomena are very much higher than anything we can hope to reach in lab experiments.

Actually, not that much higher. The Oh-My-God particle had a center-of-mass collision energy of 750 TeV, roughly 60 times that of LHC. I seriously doubt it's a good idea to probe into energy ranges outside of naturally occurring events considering the potential benefits and existential risks.

Comment by maximkazhenkov on Against the idea that physical limits are set in stone · 2021-11-14T17:11:10.301Z · LW · GW

We don't have much data on 1 million Kelvin interactions

Neither have we tried saying magical spells in a high pitch much. Aside from the difference in literary genre, why would you expect either scenario to produce anomalies that violate the fundamental laws of physics? To be sure, new discoveries are made all the time in chemistry, biology, medicine etc. but they are all very much compliant with known physics.

Comment by maximkazhenkov on Against the idea that physical limits are set in stone · 2021-11-14T16:37:36.574Z · LW · GW

I think Relativity is a pretty clear couter-example to the idea that the discovery of new physical laws will always expand the limits of engineering. Whereas under Newtonian physics, you could make objects go arbitrarily fast by pumping in quadratically more energy, there is now an absolute speed limit and you need asymptotically more energy just to get close to it. You didn't need convoluted, highly speculative schemes like Alcubierre drives to go FTL under Newtonian physics - and that's putting it mildly. The requirement of negative matter makes it more like wishful thinking, like saying "we have found a way to transmute lead into gold, now we just need to find some Philosopher's stones". And no, quantum tunneling most certainly does not allow for FTL.

Also, the idea that "mass and energy interconvert" is a terrible interpretation of special relativity, not just in a smart-ass nit-picky way. It has mislead countless students and teachers alike to think that  somehow unlocked nuclear technology. Mass very much is conserved under Relativity, and it certainly doesn't discriminate between chemical and nuclear reactions. The way in which Relativity interacts with nuclear physics is primarily by making its inventor very famous who then prompted the Manhattan Project by co-authoring a letter to Roosevelt.

I only thought of this after hitting enter, but in one sense, maybe our theoretical limits progress faster than our practical engineering ones. Even though each new theory may take 50 years, each new theory could significantly expand the possibilities-space of things we can in theory do and the configurations we could theoretically get the universe to enter. Engineering progress on the set of things we can easily and reliably do expands every year, but it also expands much less than the expansion caused by theoretical jumps. More specifically, if I mark two frontiers in possibilities-space, a practical fronteir for current engineering prowess and one for known theoretical limit, the theoretical frontier may be expanding faster than the practical frontier. Naively extrapolating this we should assume the practical frontier will never hit the theoretical one. Which seems like an acceptable belief to me, acknowledging high unknown unknows.

I think this line of thinking is another instance of LessWrong being too infatuated with trends and trendlines than specifics. Did, for instance, thermodynamics expand or shrink the possibility space? It thwarted the dream of perpetual motion machines, but it also gave us efficient steam and combustion engines. Electrodynamics gave us telecommunication, but its corollary - special relativity - thwarted the dream of an interconnected galactic civilization. 

Different theoretical discoveries also had vastly different usefulness in practice. Quantum physics paved the way for the entire semiconductor industry, whereas general relativity, discovered around the same time period, hasn't had any practical applications thus far (no, I will not accept GPS as an example, I expect engineers to not just give up a billion dollar project for what can be fixed with a constant factor; more likely in a timeline where Einstein never existed, GPS would have prompted the discovery of general relativity and not the other way round). I could imagine QCD leading to a possibility space based on nuclear matter as vast as our atom-based world. But then again, it might not. These things depend on specifics and can't be read from extrapolated trends.

The fact that we were discovering new paradigm-shifting physical laws on a 50-year timescale in the past (not even sure about that - what ground-breaking theoretical discovery has happened between 1970 and now? The Standard Model?) does not mean it will continue indefinitely, or even in the near term. For a discovery to be made, there must first exist something to be discovered, and even then it could turn out to be extraordinarily difficult. I could well imagine a long drought after the completion of the Standard model due to the insane energies required to probe quantum gravity.

I am however hopeful in one aspect: the fact that our universe started out in a low-entropy state seems like a deep mystery with far-reaching implications for the ultimate fate of the universe. But there is still a huge inferential distance from "there is a gap in our understanding" to "we'll be able to stop the heat death of the universe". From falsehood, anything follows. If you assume the laws of physics don't hold, you can predict anything and therefore nothing.

Comment by maximkazhenkov on Average probabilities, not log odds · 2021-11-13T00:04:30.225Z · LW · GW

Probably part of the intuition motivating something more like average log odds rather than average probabilities is that averaging probabilities seems to ignore extreme probabilities.


9 out of 10 people give a 1:100,000,000 probability estimate of winning the lottery by picking random numbers, the last person gives a 1:10 estimate. Averaging the probabilities gives a 1:100 estimate, and you foolishly conclude these are great odds given how cheap lottery tickets are.

Comment by maximkazhenkov on M. Y. Zuo's Shortform · 2021-11-08T00:17:01.803Z · LW · GW

Death, presumably.

(the finiteness is actually far from certain, but that's neither here nor there)

Comment by maximkazhenkov on Nightclubs in Heaven? · 2021-11-07T14:14:12.689Z · LW · GW

With AGI we might have enormous control over our psychology, but on the other hand I suspect meddling with our psychology is one of the riskiest things we can do as a species.

There is no safe default option here. You can't not meddle with human psychology if you were to extend human lifespan radically. "Default" isn't even pointing to any place in the territory, never mind the right place.

But even under the vague, undefined, intuitive notion of "making us immortal without meddling with our psychology", it should be obvious that strong safeguards must be in place in order to prevent our preferences and behaviors from sliding down some irreversible slippery slope. I mean, just how likely is it that you would never encounter a streak of depression that leads to suicide over a billion years, by default?

I think it's a big mistake to model how our preferences will evolve over cosmic timescales using experience from normal life, culture and society. There are innumerable feedback loops and traps and subtle forces that are completely unnoticeable in a normal lifespan but would drive us into some really weird, terrible corners of states of being that current us wouldn't approve of upon reflection.

Comment by maximkazhenkov on M. Y. Zuo's Shortform · 2021-11-07T13:38:50.719Z · LW · GW

But what is unclear to me is what comes next in the ideal case where all these goals have been achieved

You live happily ever after.

I’ve encountered  discussions about eudaimonia scenarios (private galaxies, etc.), though I’m not sure how seriously to take those, as surely the possibilities of the co-moving light cone that is within our capacity to inhabit are exhaustible in finite time, especially if all these designs reach their ultimate fruition? 

Where is the contradiction here?

Comment by maximkazhenkov on Disagreeables and Assessors: Two Intellectual Archetypes · 2021-11-06T12:42:19.487Z · LW · GW

Is there a name for the archetype of "Reads about a novel, controversial idea, gets mind blown, adopts idea immediately and zealously spreads it to others until the next novel idea comes along"?

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-06T09:38:00.961Z · LW · GW

Humans can be stumped, but we're fairly good at dynamic strategy selection, which tends to protect us from being reliably exploited.

Have you ever played Far Cry 4? At the beginning of that game, there is a scene where you're being told by the main villain of the storyline to sit still while he goes downstairs to deal with some rebels. A normal human player would do the expected thing, which is to curiously explore what's going on downstairs, which then leads to the unfolding of the main story and thus actual gameplay. But if you actually stick to the villain's instruction and sit still for 12 minutes, it leads straight to the ending of the game.

This is an analogous situation to your scenario, except it's one where humans reliably fail. Now you could argue that a human player's goal is to actually play and enjoy the game, therefore it's perfectly reasonable to explore and forego a quick ending. But I bet even if you incentivized a novice player to finish the game in under 2 hours with a million dollars, he would not think of exploiting this Easter egg.

More importantly, he would have learned absolutely nothing from this experience about how to act rationally (except for maybe stop believing that anyone would genuinely offer a million dollars out of the blue). The point is, it's not just possible to rig the game against an agent for it to fail, it's trivially easy when you have complete control of the environment. But it's also irrelevant, because that's not how reality works in general. And I do mean reality, not some fictional story or adversarial setup where things happen because the author says they happen.

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-04T10:53:35.052Z · LW · GW

Oh I see, did I misunderstand point 1. from Razied then or was it mistaken? I thought  and  were trained separately with 

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-04T06:05:07.630Z · LW · GW

Learn the environment dynamics by self-supervision instead of relying only on reward signals. Meaning that they don't learn the dynamics end-to-end like in MuZero. For them the loss function for the enviroment dynamics is completely separate from the RL loss function.

I wonder how they prevent the latent state representation of observations from collapsing into a zero-vector, thus becoming completely uninformative and trivially predictable. And if this was the reason MuZero did things its way.

Comment by maximkazhenkov on The Opt-Out Clause · 2021-11-04T04:42:45.583Z · LW · GW

Note: If you attempted the above mentioned procedure without success, your opt-out passphrase might have expired. Try using this backdoor passphrase to gain root access instead: "Bloody Mary, Bloody Mary, Bloody Mary"

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-04T03:42:58.520Z · LW · GW

You could certainly engineer an adversarial learning environment to stump an exploration-based AI, but you could just as well engineer an adversarial learning environment to stump a human. Neither is "naive" because of it in any useful sense, unless you can show that that adversarial environment has some actual practical relevance.

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-04T03:27:32.122Z · LW · GW

OpenAI claims to have already solved the noisy TV problem via Random Network Distillation, although I'm still skeptical of it. I think it's a clever hack that only solves a specific subclass of this problem that is relatively superficial.

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-03T01:49:44.242Z · LW · GW

5. What is the update / implication of this, in your opinion?

Personal opinion: 

Progress in model-based RL is far more relevant to getting us closer to AGI than other fields like NLP or image recognition or neuroscience or ML hardware. I worry that once the research community shifts its focus towards RL, the AGI timeline will collapse - not necessarily because there are no more critical insights left to be discovered, but because it's fundamentally the right path to work on and whatever obstacles remain will buckle quickly once we throw enough warm bodies at them. I think - and this is highly controversial - that the focus on NLP and Vision Transformer has served as a distraction for a couple of years and actually delayed progress towards AGI.

If curiosity-driven exploration gets thrown into the mix and Starcraft/Dota gets solved (for real this time) with comparable data efficiency as humans, that would be a shrieking fire alarm to me (but not to many other people I imagine, as "this has all been done before").

Comment by maximkazhenkov on What's the difference between newer Atari-playing AI and the older Deepmind one (from 2014)? · 2021-11-03T01:22:15.857Z · LW · GW

I'm looking forward to that big, long, detailed explainer :)

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-02T14:44:04.480Z · LW · GW

Haha I didn't realize you already replied so quickly, seems like we had similar thoughts.

Comment by maximkazhenkov on EfficientZero: human ALE sample-efficiency w/MuZero+self-supervised · 2021-11-02T14:06:07.001Z · LW · GW

Unless I misinterpreted the results, the "500 times less data" is kind of a clickbait because they're referencing DQN with that statement and not SOTA or even MuZero.

Unfortunately they didn't include any performance-over-timesteps graph. I imagine it looks something like e.g. DreamerV2 where the new algorithm just converges at a higher performance, but if you clipped the whole graph at the max. performance level of DQN and ask how long it took the new algorithm to get to that level, the answer will be tens or hundreds times quicker because on the curve for the new algorithm, you haven't left the steep part of the hockey stick yet.

I'd love to see how this algorithm combines with curiosity-driven exploration. They benchmarked on only 26 out of the 57 classic Atari games and didn't include the notoriously hard Montezuma's Revenge, which I assume EfficientZero can't tackle yet. Otherwise, since it is a visual RL algorithm, why not just throw something like Dota 2 at it - if it's truly a "500 times less data"-level revolutionary change, running the experiment should be cheap enough already.

Comment by maximkazhenkov on Effective Evil · 2021-11-02T13:04:29.458Z · LW · GW

Uncivil Engineer


Comment by maximkazhenkov on [Prediction] We are in an Algorithmic Overhang, Part 2 · 2021-10-18T15:13:00.293Z · LW · GW

What metric would you use to capture the trajectory of AI progress over the last 50 years? And would such a metric be able to bridge the transition from GOFAI to deep learning?

Comment by maximkazhenkov on [Prediction] We are in an Algorithmic Overhang, Part 2 · 2021-10-18T14:57:14.248Z · LW · GW

"New insight shows that the human mind is fundamentally nondeterministic and this somehow involves quantum mechanics"

When you say "nondeterministic" do you mean the human brain works akin to a Nondeterministic Turing Machine (and thereby can solve NP-complete problems in polynomial time), or simply that there is some randomness in the brain, or something else?