Posts

Pausing AI is Positive Expected Value 2024-03-10T17:10:58.008Z
Goal-Completeness is like Turing-Completeness for AGI 2023-12-19T18:12:29.947Z
Contra Hanson on AI Risk 2023-03-04T08:02:02.375Z
Robin Hanson’s latest AI risk position statement 2023-03-03T14:25:04.545Z
Best resource to go from "typical smart tech-savvy person" to "person who gets AGI risk urgency"? 2022-10-15T22:26:52.377Z
Scott Aaronson and Steven Pinker Debate AI Scaling 2022-06-28T16:04:58.515Z
Naval Ravikant and Chris Dixon Didn't Explain Any Web3 Use Cases 2021-10-29T21:54:50.184Z
Chris Dixon's Crypto Claims are Logically Flimsy 2021-09-26T21:50:47.132Z
Seeing Status Quo Bias 2021-03-08T00:24:19.666Z
Dissolving the Problem of Induction 2020-12-27T17:58:27.536Z
Are aircraft carriers super vulnerable in a modern war? 2020-09-20T18:52:29.270Z
Titan (the Wealthfront of active stock picking) - What's the catch? 2020-08-06T01:06:04.599Z
Asset Prices Consistently Violate Efficient Market Hypothesis 2020-07-28T14:21:15.220Z
Half-Baked Products and Idea Kernels 2020-06-24T01:00:20.466Z
Liron's Shortform 2020-06-09T12:27:51.078Z
How does publishing a paper work? 2020-05-21T12:14:17.589Z
Isn't Tesla stock highly undervalued? 2020-05-18T01:56:58.415Z
How About a Remote Variolation Study? 2020-04-03T12:04:04.439Z
How to Frame Negative Feedback as Forward-Facing Guidance 2020-02-09T02:47:37.230Z
The Power to Draw Better 2019-11-18T03:06:02.832Z
The Thinking Ladder - Wait But Why 2019-09-29T18:51:00.409Z
Is Specificity a Mental Model? 2019-09-28T22:53:56.886Z
The Power to Teach Concepts Better 2019-09-23T00:21:55.849Z
The Power to Be Emotionally Mature 2019-09-16T02:41:37.604Z
The Power to Understand "God" 2019-09-12T18:38:00.438Z
The Power to Solve Climate Change 2019-09-12T18:37:32.672Z
The Power to Make Scientific Breakthroughs 2019-09-08T04:14:14.402Z
Examples of Examples 2019-09-06T14:04:07.511Z
The Power to Judge Startup Ideas 2019-09-04T15:07:25.486Z
How Specificity Works 2019-09-03T12:11:36.216Z
The Power to Demolish Bad Arguments 2019-09-02T12:57:23.341Z
Specificity: Your Brain's Superpower 2019-09-02T12:53:55.022Z
What are the biggest "moonshots" currently in progress? 2019-09-01T19:41:22.556Z
Is there a simple parameter that controls human working memory capacity, which has been set tragically low? 2019-08-23T22:10:40.154Z
Is the "business cycle" an actual economic principle? 2019-06-18T14:52:00.348Z
Is "physical nondeterminism" a meaningful concept? 2019-06-16T15:55:58.198Z
What's the most annoying part of your life/job? 2016-10-23T03:37:55.440Z
Quick puzzle about utility functions under affine transformations 2016-07-16T17:11:25.988Z
You Are A Brain - Intro to LW/Rationality Concepts [Video & Slides] 2015-08-16T05:51:51.459Z
Wisdom for Smart Teens - my talk at SPARC 2014 2015-02-09T18:58:17.449Z
A proposed inefficiency in the Bitcoin markets 2013-12-27T03:48:56.031Z
Atkins Diet - How Should I Update? 2012-06-11T21:40:14.138Z
Quixey Challenge - Fix a bug in 1 minute, win $100. Refer a winner, win $50. 2012-01-19T19:39:58.264Z
Quixey is hiring a writer 2012-01-05T06:22:06.326Z
Quixey - startup applying LW-style rationality - hiring engineers 2011-09-28T04:50:45.130Z
Quixey Engineering Screening Questions 2010-10-09T10:33:23.188Z
Bloggingheads: Robert Wright and Eliezer Yudkowsky 2010-08-07T06:09:32.684Z
Selfishness Signals Status 2010-03-07T03:38:30.190Z
Med Patient Social Networks Are Better Scientific Institutions 2010-02-19T08:11:21.500Z
What is the Singularity Summit? 2009-09-16T07:18:06.675Z

Comments

Comment by Liron on Pausing AI is Positive Expected Value · 2024-03-10T18:24:27.508Z · LW · GW

Your baseline scenario (0 value) thus assumes away the possibility that civilization permanently collapses (in some sense) in the absence of some path to greater intelligence (whether via AI or whatever else), which would also wipe out any future value. This is a non-negligible possibility. 

Yes, my mainline no-superintelligence-by-2100 scenario is that the trend toward a better world continues to 2100.

You're welcome to set the baseline number to a negative, or tweak the numbers however you want to reflect any probability of a non-ASI existential disaster happening before 2100. I doubt it'll affect the conclusion.

To be honest the only thing preventing me from granting paperclippers as much or more value than humans is uncertainty/conservatism about my metaethics

Ah ok, the crux of our disagreement is how much you value the paperclipper type scenario that I'd consider a very bad outcome. If you think that outcome is good then yeah, that licenses you in this formula to conclude that rushing toward AI is good.

Comment by Liron on Open Thread – Winter 2023/2024 · 2024-03-06T23:43:26.289Z · LW · GW

Founder here :) I'm biased now, but FWIW I was also saying the same thing before I started this company in 2017: a good dating/relationship coach is super helpful. At this point we've coached over 100,000 clients and racked up many good reviews.

I've personally used a dating coach and a couples counselor. IMO it helps twofold:

  1. Relevant insights and advice that the coach has that most people don't, e.g. in the domain of communication skills, common tactics that best improve a situation, pitfalls to avoid.
  2. A neutral party who's good at letting you (and potentially a partner) objectively review and analyze the situation.

Relationship Hero hires, measures and curates the best coaches, and streamlines matching you to the best coach based on your scenario. Here's a discount link for LW users to get $50 off.

Comment by Liron on The Power to Demolish Bad Arguments · 2024-02-29T03:50:11.777Z · LW · GW

Personally I just have the habit of reaching for specifics to begin my communication to help make things clear. This post may help.

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2024-02-20T20:10:20.253Z · LW · GW

Unlike the other animals, humans can represent any goal in a large domain like the physical universe, and then in a large fraction of cases, they can think of useful things to steer the universe toward that goal to an appreciable degree.

Some goals are more difficult than others / require giving the human control over more resources than others, and measurements of optimization power are hard to define, but this definition is taking a step toward formalizing the claim that humans are more of a "general intelligence" than animals. Presumably you agree with this claim?

It seems the crux of our disagreement factors down to a disagreement about whether this Optimization Power post by Eliezer is pointing at a sufficiently coherent concept.

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2024-02-19T03:26:00.758Z · LW · GW

I don’t get what point you’re trying to make about the takeaway of my analogy by bringing up the halting problem. There might not even be something analogous to the halting problem in my analogy of goal-completeness, but so what?

I also don’t get why you’re bringing up the detail that “single correct output” is not 100% the same thing as “single goal-specification with variable degrees of success measured on a utility function”. It’s in the nature of analogies that details are different yet we’re still able to infer an analogous conclusion on some dimension.

Humans are goal-complete, or equivalently “humans are general intelligences”, in the sense that many of us in the smartest quartile can output plans with the expectation of a much better than random score on a very broad range of utility functions over arbitrary domains.

Comment by Liron on Against Nonlinear (Thing Of Things) · 2024-01-19T05:41:22.122Z · LW · GW

These 4 beefs are different and less serious than the original accusations, or at least feel that way to me. Retconning a motte after the bailey is lost? That said, they're reasonable beefs for someone to have.

Comment by Liron on The Power to Demolish Bad Arguments · 2024-01-19T03:46:45.010Z · LW · GW

I’m not saying “mapping a big category to a single example is what it’s all about”. I’m saying that it’s a sanity check. Like why wouldn’t you be able to do that? Yet sometimes you can’t, and it’s cause for alarm.

Comment by Liron on The Power to Demolish Bad Arguments · 2024-01-17T05:20:10.210Z · LW · GW

Meaningful claims don't have to be specific; they just have to be able to be substantiated by a nonzero number of specific examples. Here's how I imagine this conversation:

Chris: Love your neighbor!

Liron: Can you give me an example of a time in your life where that exhortation was relevant?

Chris: Sure. People in my apartment complex like to smoke cigarettes in the courtyard and the smoke wafts up to my window. It's actually a nonsmoking complex, so I could complain to management and get them to stop, but I understand the relaxing feeling of a good smoke, so I let them be.

Liron: Ah I see, that was pretty accommodating of you.

Chris: Yeah, and I felt love in my heart for my fellow man when I did that.

Liron: Cool beans. Thanks for helping me understand what kind of scenarios you mean for your exhortation to "love your neighbor" to map to.

Comment by Liron on The Power to Demolish Bad Arguments · 2024-01-17T05:07:06.937Z · LW · GW

Sweet thanks

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-21T01:01:37.080Z · LW · GW

I agree that if a goal-complete AI steers the future very slowly, or very weakly - as by just trying every possible action one at a time - then at some point it becomes a degenerate case of the concept.

(Applying the same level of pedantry to Turing-completeness, you could similarly ask if the simple Turing machine that enumerates all possible output-tape configurations one-by-one is a UTM.)

The reason "goal-complete" (or "AGI") is a useful coinage, is that there's a large cluster in plausible-reality-space of goal-complete agents with a reasonable amount of goal-complete optimization power (e.g. humans, natural selection, and probably AI starting in a few years), and another large distinguishable cluster of non-goal-complete agents (e.g. the other animals, narrow AI).

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-21T00:53:27.126Z · LW · GW

Yeah, no doubt there are cases where people save money by having a narrower AI, just like the scenario you describe, or using ASICs for Bitcoin mining. The goal-complete AI itself would be expected to often solve problems by creating optimized problem-specific hardware.

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T22:51:54.865Z · LW · GW

Hmm it seems to me that you're just being pedantic about goal-completeness in a way that you aren't symmetrically being for Turing-completeness.

You could point out that "most" Turing machines output tapes full of 10^100 1s and 0s in a near-random configuration, and every computing device on earth is equally hopeless at doing that.

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T22:48:18.136Z · LW · GW

That's getting into details of the scenario that are hard to predict. Like I said, I think most scenarios where goal-complete AI exists are just ones where humans get disempowered and then a single AI fooms (or a small number make a deal to split up the universe and foom together).

As to whether humans will prevent goal-complete AI: some of us are yelling "Pause AI!"

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T22:43:42.586Z · LW · GW

Humans will trust human brain capable AI models to say, drive a bus, despite the poor reliability, as long as it crashes less than humans?

Yes, because the goal-complete AI won't just perform better than humans, it'll also perform better than narrower AIs.

(Well, I think we'll actually be dead if the premise of the hypothetical is that goal-complete AI exists, but let's assume we aren't.)

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T22:42:06.671Z · LW · GW

A goal is essentially a specification of a function to optimise, and all optimisation algorithms perform equally well (or rather poorly) when averaged across all functions.

Well, I've never met a monkey that has an "optimization algorithm" by your definition. I've only met humans who have such optimization algorithms. And that distinction is what I'm pointing at.

Goal-completeness points to the same thing as what most people mean by "AGI".

E.g. I claim humans are goal-complete General Intelligences because you can give us any goal-specification and we'll very often be able to steer the future closer toward it.

Currently, no other known organism or software program has this property to the degree that humans do. GPT-4 has it for an unprecedentedly large domain, by virtue of giving satisfying answers to a large fraction of arbitrary natural-language prompts.

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T22:36:07.368Z · LW · GW

Fine, I agree that if computation-specific electronics, like logic gates, weren't reliable, then it would introduce reliability as an important factor in the equation. Or in the case of AGI, that you can break the analogy to Turing-complete convergence by considering what happens if a component specific to goal-complete AI is unreliable.

I currently see no reason to expect such an unreliable component in AGI, so I expect that the reliability part of the analogy to Turing-completeness will hold.

In scenario (1) and (2), you're giving descriptions at a level of detail that I don't think is necessarily an accurate characterization of goal-complete AI. E.g. in my predicted future, a goal-complete AI will eventually have the form of a compact program that can run on a laptop. (After all, the human brain is only 12W and 20Hz, and full of known reasoning "bugs".)

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T22:04:08.408Z · LW · GW

But microcontrollers are reliable for the same reason that video-game circuit boards are reliable: They both derive their reliability from the reliability of electronic components in the same manner, a manner which doesn't change during the convergence from application-specific circuits to Turing-complete chips.

The engineer who designed it didn't trust the microcontroller not to fail in a way that left the heating element on all the time. So it had a thermal fuse to prevent this failure mode.

If the microcontroller fails to turn off the heating element, that may be a result of the extra complexity/brittleness of the Turing-complete architecture, but the risk there isn't that much higher than the risk of using a simpler design involving an electronic circuit. I'm pretty sure that safety fuse would have been judged worthwile even if the heating element was controlled by a simpler circuit.


I think we can model the convergence to a Turing-complete architecture as having a negligible decrease in reliability. In many cases it even increases reliability, since:

  1. Due to the higher expressive power that the developers have, creating a piece of software is often easier to do correctly, with fewer unforeseen error conditions, than creating a complex circuit to do the same thing.
  2. Software systems make it easier to implement a powerful range of self-monitoring and self-correcting behaviors. E.g. If every Google employee took a 1-week vacation and natural disasters shut down multiple data centers, Google search would probably stay up and running.

Similarly, to the extent that any "narrow AI" application is reliable (e.g. Go players, self-driving cars), I'd expect that a goal-complete AI implementation would be equally reliable, or more so.

Comment by Liron on Goal-Completeness is like Turing-Completeness for AGI · 2023-12-20T07:09:08.691Z · LW · GW

A great post that helped inspire me to write this up is Steering Systems. The "goal engine + steering code" architecture that we're anticipating for AIs is analogous to the "computer + software" architecture whose convergence I got to witness in my lifetime.

I'm surprised this post isn't getting any engagement (yet), because for me the analogy to Turing-complete convergence is a deep source of my intuition about powerful broad-domain goal-optimizing AIs being on the horizon.

Comment by Liron on Why Yudkowsky is wrong about "covalently bonded equivalents of biology" · 2023-12-06T18:33:22.770Z · LW · GW

Titotal, do you agree with Eliezer’s larger point that a superintellience engineering physical actuators from the ground up can probably do much better than what our evolutionary search process produced? If so, how would you steel man the argument?

Comment by Liron on Shane Legg interview on alignment · 2023-10-29T19:22:29.118Z · LW · GW

I made a short clip highlighting how Legg seems to miss an opportunity to acknowledge the inner alignment problem, since his proposed alignment solution seems to be a fundamentally training / black box approach.

Comment by Liron on AI #35: Responsible Scaling Policies · 2023-10-26T18:30:26.186Z · LW · GW

Full video of the SF protest

Comment by Liron on TOMORROW: the largest AI Safety protest ever! · 2023-10-26T05:03:43.425Z · LW · GW

Here’s a 2-min edited video of the protest.

Most people who hear our message do so well after the protest, via sharing of this kind of media.

Comment by Liron on TOMORROW: the largest AI Safety protest ever! · 2023-10-22T14:51:07.330Z · LW · GW

The SF one went great! Here’s a first batch of pics. A lot of the impact will come from sharing the pics and videos.

Comment by Liron on TOMORROW: the largest AI Safety protest ever! · 2023-10-20T23:15:15.577Z · LW · GW

I think the impact will be pretty significant:

  1. It's one of those things where a lot of people - the majority according to some polls - already agree with it, so it's building mutual knowledge and unlocking some tailwinds
  2. It's interesting and polarizing. People who think the movement is crazy are having fun with it on social media, which also keeps it top of mind.
Comment by Liron on TOMORROW: the largest AI Safety protest ever! · 2023-10-20T23:13:35.007Z · LW · GW

I'll be at the San Francisco protest!

We have Pause AI T-shirts, costumes, signs and other fun stuff. In addition to being a historic event, it's a great day to make sane friends and we'll grab some food/drinks after.

Comment by Liron on Holly Elmore and Rob Miles dialogue on AI Safety Advocacy · 2023-10-20T21:28:45.731Z · LW · GW

Just in case you missed that link at the top:

The global Pause AI protest is TOMORROW (Saturday Oct 21)!

This is a historic event, the first time hundreds of people are coming out in 8 countries to protest AI.

I'm helping with logistics for the San Francisco one which you can join here. Feel free to contact me or Holly on DM/email for any reason.

Comment by Liron on Evolution provides no evidence for the sharp left turn · 2023-10-15T22:39:47.436Z · LW · GW

Hey Quintin thanks for the diagram.

Have you tried comparing the cumulative amount of genetic info over 3.5B years?

Isn't it a big coincidence that the time of brains that process info quickly / increase information rapidly, is also the time where those brains are much more powerful than all other products of evolution?

(The obvious explanation in my view is that brains are vastly better optimizers/searchers per computation step, but I'm trying to make sure I understand your view.)

Comment by Liron on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-10-06T03:36:38.980Z · LW · GW

Appreciate the detailed analysis.

I don’t think this was a good debate, but I felt I was in a position where I would have had to invest a lot of time to do better by the other side’s standards.

Quintin and I have agreed to do a X Space debate, and I’m optimistic that format can be more productive. While I don’t necesarily expect to update my view much, I am interested to at least understand what the crux is, which I’m not super clear on atm.

Here’s a meta-level opinion:

I don’t think it was the best choice of Quintin to keep writing replies that were disproportionally long compared to mine.

There’s such a thing as zooming claims and arguments out. When I write short tweets, that’s what I’m doing. If he wants to zoom in on something, I think it would be a better conversation if he made an effort to do it less at a time, or do it for fewer parts at a time, for a more productive back & forth.

Comment by Liron on Sharing Information About Nonlinear · 2023-09-07T14:38:48.686Z · LW · GW

FWIW I’ve never known a character of high integrity who I could imagine writing the phrase “your career in EA would be over with a few DMs”.

Comment by Liron on Lightcone Infrastructure/LessWrong is looking for funding · 2023-06-14T21:47:56.329Z · LW · GW

Donated!

Comment by Liron on The Dial of Progress · 2023-06-13T20:42:55.648Z · LW · GW

Actually, the only time I know they cashed in early was selling half their Coinbase shares at the direct listing after holding for 7 years.

Their racket was to be the #1 crypto fund with the most assets under management ($7.6B total) so that they can collect the most management fees (probably about $1B total). It's great business for a16z to be in the sector-leader AUM game even when the sector makes no logical sense.

I'm just saying Marc's reputation for publicly making logically-flimsy arguments and not updating on evidence should be considered when he enters a new area of discourse.

Comment by Liron on The Dial of Progress · 2023-06-13T18:54:33.857Z · LW · GW

I encourage you to look into his firm's Web3 claims and the reasoning behind them. My sibling comment has one link that is particularly egregious and recent. Here's another badly-reasoned Web3 argument made by his partner, which implies Marc's endorsement, and the time his firm invested over $100M in an obvious Ponzi scheme.

Comment by Liron on The Dial of Progress · 2023-06-13T18:49:42.147Z · LW · GW

My #1 and #2 are in a separate video Marc made after the post Zvi referred to, but ya, could fall under the "bizarrely poor arguments" Zvi is trying to explain.

My #3 and his firm's various statements about Web3 in the last couple years, like this recent gaslighting, are additional examples of bizarrely poor arguments in an unrelated field.

If we don't come in with an a-priori belief that Marc is an honest or capable reasoner, there's less confusion for Zvi to explain.

Comment by Liron on The Dial of Progress · 2023-06-13T16:43:51.351Z · LW · GW

My model is that Marc Andreessen just consistently makes badly-reasoned statements:

  1. Comparing AI doomerism to love of killing Nazis
  2. Endorsing the claim that arbitrarily powerful technologies don't change the equilibrum of good and bad forces
  3. Last year being unable to coherently explain a single Web3 use case despite his firm investing $7.6B in the space
Comment by Liron on "notkilleveryoneism" sounds dumb · 2023-04-29T07:16:17.224Z · LW · GW

I’ve personally been saying “AI Doom” as the topic identifier since it’s clear and catchy and won’t be confused with smaller issues.

Comment by Liron on Steering systems · 2023-04-10T05:14:23.371Z · LW · GW

Great post! Agree with everything. You came at some points from a unique angle. I especially appreciate the insight of "most of the useful steering work of a system comes from the very last bits of glue code".

Comment by Liron on Conceding a short timelines bet early · 2023-03-16T23:02:11.506Z · LW · GW

Bravo.

Which 2+ outcomes from the list do you think are most likely to lead to your loss?

Comment by Liron on Abuse in LessWrong and rationalist communities in Bloomberg News · 2023-03-08T01:58:51.813Z · LW · GW

It seems from your link like CFAR has taken responsibility, taken corrective action, and states how they’ll do everything in their power to avoid a similar abuse incident in the future.

I think in general the way to deal with abuse situations within an organization is to identify which authority should be taking appropriate disciplinary action regarding the abuser’s role and privileges. A failure to act there, like CFAR’s admitted process failure that they later corrected, would be concerning if we thought it was still happening.

If every abuse is being properly disciplined by the relevant organization, and the rate of abuse isn’t high compared to the base rate in the non-rationalist population, then the current situation isn’t a crisis - even if some instances of abuse unfortunately involve the perpetrator referencing rationality or EA concepts.

Comment by Liron on Startups are like firewood · 2023-03-06T02:03:06.021Z · LW · GW

Great post! I agree with this analogy.

I think the fire stands for value creation. My Lean MVP Flowchart post advises to always orient your strategy about what it'll take to double the size of your current value creation. Paul Graham's Do Things That Don't Scale is a coarse-grained version of this advice, pointing out that doubling a small fire is qualitatively different from doubling a large fire.

Comment by Liron on Contra Hanson on AI Risk · 2023-03-05T00:52:33.500Z · LW · GW

I guess that’s plausible, but then my main doom scenario would involve them getting leapfrogged by a different AI that has hit a rapid positive feedback loop of how to keep amplifying its consequentialist planning abilities.

Comment by Liron on Contra Hanson on AI Risk · 2023-03-05T00:46:56.465Z · LW · GW

My reasoning stems from believing that AI-space contains designs that can easily plan effective strategies to get the universe into virtually any configuration.

And they’re going to be low-complexity designs. Because engineering stuff in the universe isn’t a hard problem from a complexity theory perspective.

Why should the path from today to the first instantiation of such an algorithm be long?

So I think we can state properties of an unprecedented future that first-principles computer science can constrain, and historical trends can’t.

Comment by Liron on Contra Hanson on AI Risk · 2023-03-04T18:10:11.476Z · LW · GW

I think the mental model of needing “advances in chemistry” isn’t accurate about superintelligence. I think a ton of understanding of how to precisely engineer anything you want out of atoms just clicks from a tiny amount of observational data when you’re really good at reasoning.

Comment by Liron on Contra Hanson on AI Risk · 2023-03-04T17:42:42.149Z · LW · GW

I don’t know if LLM Ems can really be a significant factorizable part of the AI tech tree. If they have anything like today’s LLM limitations, they’re not as powerful as humans and ems. If they’re much more powerful than today’s LLMs, they’re likely to have powerful submodules that are qualitatively different from what we think of as LLMs.

Comment by Liron on Contra Hanson on AI Risk · 2023-03-04T17:16:07.089Z · LW · GW

I agree that rapid capability gain is a key part of the AI doom scenario.

During the Manhattan project, Feynman prevented an accident by pointing out that labs were storing too much uranium too close together. We’re not just lucky that the accident was prevented; we’re also lucky that if the accident had happened, the nuclear chain reaction wouldn’t have fed on the atmosphere.

We similarly depend on luck whenever a new AI capability gain such as LLM general-topic chatting emerges. We’re lucky that it’s not a capability that can feed on itself rapidly. Maybe we’ll keep being lucky when new AI advances happen, and each time it’ll keep being more like past human economic progress or like past human software development. But there’s also a significant chance that it could instead be more like a slightly-worse-than-nuclear-weapon scenario.

We just keep taking next steps of unknown magnitude into an attractor of superintelligent AI. At some point our steps will trigger a rapid positive-feedback slide where each step is dealing with very powerful and complex things that we’re far from being able to understand. I just don’t see why there’s more than 90% chance that this will proceed at a survivable pace.

Comment by Liron on Robin Hanson’s latest AI risk position statement · 2023-03-04T08:08:32.002Z · LW · GW

My commentary on this grew into a separate post: Contra Hanson on AI Risk

Comment by Liron on Bankless Podcast: 159 - We’re All Gonna Die with Eliezer Yudkowsky · 2023-02-22T02:58:49.286Z · LW · GW

I tweeted my notes of Eliezer's points with abridged clips.

Comment by Liron on I Am Scared of Posting Negative Takes About Bing's AI · 2023-02-18T02:58:40.383Z · LW · GW

Do not taunt Bing Chat

Comment by Liron on Why you should learn sign language · 2023-01-20T02:49:33.689Z · LW · GW

Thanks. Love this kind of post, like rationalist community show&tell.

Comment by Liron on Looking Back on Posts From 2022 · 2022-12-27T02:54:51.894Z · LW · GW

I continue to be unsatisfied with the availability of a simple, quick explanation for why AGI likely destroys all value in the universe and AI Dontkilleveryoneism is an important cause.

In an ideal world, the author of that post is most definitely absolutely not me. That doesn’t rule out that I should take a crack at writing it anyway at some point.

I’m working on this atm! I also find it surprising that it doesn’t seem to exist yet.

Comment by Liron on Let’s think about slowing down AI · 2022-12-24T16:18:58.160Z · LW · GW

I arrogantly think I could write a broadly compelling and accessible case for AI risk

I recently asked whether such a resource exists and didn't find one that meets what I'm looking for. Currently trying to write my own version.