Posts

Facebook is Simulacra Level 3, Andreessen is Level 4 2021-04-28T17:38:03.981Z
Against "Context-Free Integrity" 2021-04-14T08:20:44.368Z
"Taking your environment as object" vs "Being subject to your environment" 2021-04-11T22:47:04.978Z
I'm from a parallel Earth with much higher coordination: AMA 2021-04-05T22:09:24.033Z
Why We Launched LessWrong.SubStack 2021-04-01T06:34:00.907Z
"Infra-Bayesianism with Vanessa Kosoy" – Watch/Discuss Party 2021-03-22T23:44:19.795Z
"You and Your Research" – Hamming Watch/Discuss Party 2021-03-19T00:16:13.605Z
Review Voting Thread 2020-12-30T03:23:06.075Z
Final Day to Order LW Books by Christmas for US 2020-12-09T23:30:36.877Z
The LessWrong 2018 Book is Available for Pre-order 2020-12-02T11:21:00.221Z
AGI Predictions 2020-11-21T03:46:28.357Z
Rationalist Town Hall: Pandemic Edition 2020-10-21T23:54:03.528Z
Sunday October 25, 12:00PM (PT) — Scott Garrabrant on "Cartesian Frames" 2020-10-21T03:27:12.739Z
Sunday October 18, 12:00PM (PT) — Garden Party 2020-10-17T19:36:52.829Z
Have the lockdowns been worth it? 2020-10-12T23:35:14.835Z
Fermi Challenge: Trains and Air Cargo 2020-10-05T21:51:45.281Z
Postmortem to Petrov Day, 2020 2020-10-03T21:30:56.491Z
Open & Welcome Thread – October 2020 2020-10-01T19:06:45.928Z
What are good rationality exercises? 2020-09-27T21:25:24.574Z
Honoring Petrov Day on LessWrong, in 2020 2020-09-26T08:01:36.838Z
Sunday August 23rd, 12pm (PDT) – Double Crux with Buck Shlegeris and Oliver Habryka on Slow vs. Fast AI Takeoff 2020-08-22T06:37:07.173Z
Forecasting Thread: AI Timelines 2020-08-22T02:33:09.431Z
[Oops, there is actually an event] Notice: No LW event this weekend 2020-08-22T01:26:31.820Z
Highlights from the Blackmail Debate (Robin Hanson vs Zvi Mowshowitz) 2020-08-20T00:49:49.639Z
Survey Results: 10 Fun Questions for LWers 2020-08-19T06:10:55.386Z
10 Fun Questions for LessWrongers 2020-08-18T03:28:05.276Z
Sunday August 16, 12pm (PDT) — talks by Ozzie Gooen, habryka, Ben Pace 2020-08-14T18:32:35.378Z
Is Wirecutter still good? 2020-08-07T21:54:06.141Z
Sunday August 9, 1pm (PDT) — talks by elityre, jacobjacob, Ruby 2020-08-06T22:50:21.550Z
Sunday August 2, 12pm (PDT) — talks by jimrandomh, johnswenthworth, Daniel Filan, Jacobian 2020-07-30T23:55:44.712Z
What Failure Looks Like: Distilling the Discussion 2020-07-29T21:49:17.255Z
"Should Blackmail Be Legal" Hanson/Zvi Debate (Sun July 26th, 3pm PDT) 2020-07-20T04:06:26.275Z
Sunday July 19, 1pm (PDT) — talks by Raemon, ricraz, mr-hire, Jameson Quinn 2020-07-16T20:04:37.974Z
Sunday July 12 — talks by Scott Garrabrant, Alexflint, alexei, Stuart_Armstrong 2020-07-08T00:27:57.876Z
The silence is deafening – Devon Zuegel 2020-07-04T02:30:59.409Z
Inviting Curated Authors to Give 5-Min Online Talks 2020-07-01T01:05:39.794Z
Radical Probabilism [Transcript] 2020-06-26T22:14:13.523Z
Sunday June 28 – talks by johnswentworth, Daniel kokotajlo, Charlie Steiner, TurnTrout 2020-06-26T19:13:23.754Z
DontDoxScottAlexander.com - A Petition 2020-06-25T05:44:50.050Z
Prediction = Compression [Transcript] 2020-06-22T23:54:22.170Z
Online Curated LessWrong Talks 2020-06-19T02:16:14.824Z
Sunday June 21st – talks by Abram Demski, alkjash, orthonormal, eukaryote, Vaniver 2020-06-18T20:10:38.978Z
Superexponential Historic Growth, by David Roodman 2020-06-15T21:49:00.188Z
The one where Quirrell is an egg 2020-04-15T06:02:36.337Z
Coronavirus: Justified Key Insights Thread 2020-04-13T22:40:03.104Z
Hanson & Mowshowitz Debate: COVID-19 Variolation 2020-04-08T00:07:28.315Z
April Fools: Announcing LessWrong 3.0 – Now in VR! 2020-04-01T08:00:15.199Z
Small Comment on Organisational Disclaimers 2020-03-29T17:07:48.339Z
[Update: New URL] Today's Online Meetup: We're Using Mozilla Hubs 2020-03-29T04:00:18.228Z
March 25: Daily Coronavirus Updates 2020-03-27T04:32:18.530Z

Comments

Comment by Ben Pace (Benito) on What's the status of third vaccine doses? · 2021-08-04T06:42:05.478Z · LW · GW

Good point Alex. I've moved my and the other Ben's answers to comments.

Comment by Ben Pace (Benito) on What's the status of third vaccine doses? · 2021-08-04T04:37:00.763Z · LW · GW

Not an answer, but just my epistemic status: I heard that they're good to get 6 months after the 2nd dose, so I'm planning to get another dose in ~2 months.

Comment by Ben Pace (Benito) on Delta Strain: Fact Dump and Some Policy Takeaways · 2021-08-01T23:20:51.431Z · LW · GW

So Adam is saying microcovids are cheaper than the OP does? Connor writes

1 hour of your life lost every 1k-5k uCOVIDs

(Pardon me commenting while not getting into the details on this important topic, I am busy but am trying to track at least whether there's disagreement and in what direction.)

Comment by Ben Pace (Benito) on Delta Strain: Fact Dump and Some Policy Takeaways · 2021-08-01T18:10:06.093Z · LW · GW

(I'd personally appreciate you saying how many microcovids you think is equivalent to an hour's time; that's the main number I've been using to figure out whether various costs are worth it.)

Comment by Ben Pace (Benito) on Delta Strain: Fact Dump and Some Policy Takeaways · 2021-07-28T04:06:25.505Z · LW · GW

Thanks for sharing this stuff. I like the principles, and I like the bottom line being a rate of exchange: 1 hour of your life lost every 1k-5k uCOVIDs. Will go do my own calculations to figure out what that implies for me...

Comment by Ben Pace (Benito) on Problems facing a correspondence theory of knowledge · 2021-07-27T18:04:49.873Z · LW · GW

This was a great summary, thx.

Comment by Ben Pace (Benito) on "If and Only If" Should Be Spelled "Ifeff" · 2021-07-18T01:57:55.749Z · LW · GW

Alt: I think ifif makes much more sense than ifeff. It's super unclear where the 'e' comes from or why there's two 'f's, but "if and only if" is "ifif" if you take the start and the end.

Comment by Ben Pace (Benito) on "If and Only If" Should Be Spelled "Ifeff" · 2021-07-16T22:26:11.988Z · LW · GW

Counter: the reason "iff" works is because people who don't get it can still read the sentence and not worry they missed something key. If you use "ifeff" then massive audiences will infer that the sentence is unreadable.

Comment by Ben Pace (Benito) on Jimrandomh's Shortform · 2021-07-16T04:36:35.077Z · LW · GW

Yep. Seems you have broadly rediscovered conflict vs mistake.

Comment by Ben Pace (Benito) on What will the twenties look like if AGI is 30 years away? · 2021-07-15T01:17:27.480Z · LW · GW

I'd actually be fine with a solution where we all agree to stop using the terms "long timelines" and "short timelines" and just use numbers instead. How does that sound?

I think this could be excellent.

Comment by Ben Pace (Benito) on Agency and the unreliable autonomous car · 2021-07-09T04:19:03.798Z · LW · GW

What is this, "A Series of Unfortunate Logical Events"? I laughed quite a bit, and enjoyed walking through the issues in self-knowledge that the löbstacle poses.

Comment by Ben Pace (Benito) on Benito's Shortform Feed · 2021-07-07T20:55:32.900Z · LW · GW

You fix it! If you think it's such a good idea :)

I am relatively hesitant to start doing opinionated fixes on Wikipedia, I think that's not the culture of page setup that they want. My understanding is that the best Wikipedia editors write masses of pages that they're relatively disinterested in, and that being overly interested in a specific page mostly leads you to violating all of their rules and getting banned. This sort of actively political editing is precisely the sort of thing that they're trying to avoid.

Comment by Ben Pace (Benito) on AXRP Episode 9 - Finite Factored Sets with Scott Garrabrant · 2021-07-05T21:41:27.759Z · LW · GW

Curated, in part for this episode, and also as a celebration of the whole series. I've listened to 6 out of the 9, and I've learned a great deal about people's work and their motivations for it. This episode in particular was excellent because I finally learned what a finite factored set was – your example of the Cartesian plane was really helpful! Which is a credit to your communication skills.

Basically every episode has been worthwhile and valuable for me, it's been easy to sit down with a researcher and hear them explain their research, and Daniel always brings thoughtful and on-point questions. I would personally be very gratified to see a future where AXRP has 10x the number of episodes, where for any key piece of AI x-risk research I can listen to the author talk it through for 1-2 hours. Please keep going!

Added: it's also great that you make transcripts, that's really valuable for a lot of people and searchability.

Comment by Ben Pace (Benito) on Benito's Shortform Feed · 2021-07-01T22:44:18.008Z · LW · GW

Er, Wikipedia has a page on misinformation about Covid, and the first example is Wuhan lab origin. Kinda shocked that Wikipedia is calling this misinformation. Seems like their authoritative sources are abusing their positions. I am scared that I'm going to stop trusting Wikipedia soon enough, which is leaving me feeling pretty shook.

Comment by Ben Pace (Benito) on Musings on general systems alignment · 2021-06-30T18:35:40.745Z · LW · GW

That’s an inspiring narrative that rings true to me, I’m sure I will think on that framing more. Thank you.

Comment by Ben Pace (Benito) on rohinmshah's Shortform · 2021-06-29T17:51:06.656Z · LW · GW

Or just logical convergence. Two calculators get the same answer to 2 + 2 = 4, and it's not because they're both power-seeking.

Comment by Ben Pace (Benito) on rohinmshah's Shortform · 2021-06-28T19:14:55.480Z · LW · GW

Mark Xu’s house is not EH.

Comment by Ben Pace (Benito) on rohinmshah's Shortform · 2021-06-28T19:13:05.632Z · LW · GW

You could certainly hire a good software engineer at that salary, but I don’t think you could give them a vision and network and trust them to be autonomous. Money isn’t the bottleneck there. Just because you have the funding to hire someone for a role doesn’t mean you can. Hiring is incredibly difficult. Go see YC on hiring, or PG.

Most founding startup people are worth way more than their salary.

Comment by Ben Pace (Benito) on rohinmshah's Shortform · 2021-06-28T07:05:30.344Z · LW · GW

If it's 8 hour workdays and 5 days a week, at $100/hour that's 8 * 10 * 100 = $8k. No, you could not pay me $8k to stop working on the LW team for 2 weeks.

I think $30k-$40k might make sense.

Comment by Ben Pace (Benito) on rohinmshah's Shortform · 2021-06-28T06:58:23.968Z · LW · GW

List of changes that stand out to me:

  • I ended up saying that long-covid costs were roughly the same as death, so it was a factor of 2x.
  • Price of a life at $10 million is a bit low, I put mine at $50 million, so a factor of 5x difference.

I didn't follow all of your calculations about being out for 2 weeks and isolated, I basically just did those two (death and long covid) and it came to ~$200k for me. Roughly say that's the average among 5 people and then you get to $1 per microcovid to the house.

Comment by Ben Pace (Benito) on rohinmshah's Shortform · 2021-06-28T01:56:21.220Z · LW · GW

At Event Horizon we had a policy for around 6-9 months where if you got a microcovid, you paid $1 to the house, and it was split between everyone else. Do whatever you like, we don't mind, as long as you bring a microcovid estimate and pay the house.

Comment by Ben Pace (Benito) on What is the most effective way to donate to AGI XRisk mitigation? · 2021-06-11T20:37:42.228Z · LW · GW

Gonna +1 the other comments that name the LTFF and Larks' annual reviews. Though if I were to donate myself I'd probably go with a donor lottery. (The CEA donor lottery is not currently up alas.)

Comment by Ben Pace (Benito) on What are the best ways to improve resiliency? · 2021-06-11T06:34:46.617Z · LW · GW

Meaningful work and strong close relationships (romantic, family, friends, colleagues).

Comment by Ben Pace (Benito) on ML is now automating parts of chip R&D. How big a deal is this? · 2021-06-11T05:18:41.514Z · LW · GW

Mod here, I put a table in your comment.

(Tables aren't in comment editors right now, I made it in the post editor and copied it in.)

Comment by Ben Pace (Benito) on Reply to Nate Soares on Dolphins · 2021-06-11T05:09:50.893Z · LW · GW

Wow, am surprised that the dolphins example has 180'd in Nate's recent thread.

I do endorse shitposting as a form of posting, it's great and I'd like Nate to do more.

Comment by Ben Pace (Benito) on Power dynamics as a blind spot or blurry spot in our collective world-modeling, especially around AI · 2021-06-08T02:13:54.619Z · LW · GW

This felt like a pretty helpful clarification of your thoughts Paul, thx.

Comment by Ben Pace (Benito) on Five Whys · 2021-06-07T08:08:45.454Z · LW · GW

I did my first 5-whys analysis this week. The LW team has just run a big in-person event, and I got all the team together to go through about 4 of these.

One main issue I ran into is that at every step of the 5-whys, there were a bunch of different things that could've caused the problem. Like, a task didn't get done. Why not? Well, nobody knew it was their responsibility for one. For two, the person who saw the problem didn't know who to tell. For three, we hadn't noticed in advance that the problem would arise, and could've fixed it at an earlier point. Etc. Sometimes we got a bunch of branches.

The take we had was that we should pick the cause that we think we "should" have fixed. Another way of putting it is: the one we'd most like to do better on in the future. This was generally good.

(The other thing that was not clear to me at the time but was when we re-read the Lean book, and that you get right in the post, is that you're supposed to make an effort to solve each of the 5 problems, not just the most fundamental one. The idea being that if you regularly do 5-whys, then the fundamental problems will get solved by just lots of problems stemming from them.)

Comment by Ben Pace (Benito) on Rogue AGI Embodies Valuable Intellectual Property · 2021-06-06T00:04:39.715Z · LW · GW

Assuming that the discounted value of a monopoly in this IP is reasonably close to Alice’s cost of training, e.g. 1x-3x, competition between Alpha and Beta only shrinks the available profits by half, and Beta expects to acquire between 10%-50% of the market,

Basic econ q here: I think that 2 competitors can often cut the profits by much more than half, because they can always undercut each other until they hit the cost of production. Especially if you're going from 1 seller to 2, I think that can shift a market from monopoly to not-a-monopoly, so I think it might be a lot less valuable.

Still, obviously likely to be worth it to the second company, so I totally expect the competition to happen.

Comment by Ben Pace (Benito) on Predict responses to the "existential risk from AI" survey · 2021-06-03T08:00:07.945Z · LW · GW

1. How likely do you think it is that the overall value of the future will be drastically less than it could have been, as a result of humanity not doing enough technical AI safety research?
 

The clarification note was helpful, because this is an odd question to me. There's lots of things that could prevent x-risk from AI, including e.g. better world governance. It's not as a result of not doing technical research, even if technical research is a great way to prevent it.

Comment by Ben Pace (Benito) on Finite Factored Sets · 2021-05-27T07:55:23.699Z · LW · GW

Curated. This is a fascinating framework that (to the best of my understanding) makes substantive improvements on the Pearlian paradigm. It's also really exciting that you found a new simple sequence. 

Re: the writeup, it's explained very clearly, the Q&A interspersed is a very nice touch. I like that the talk factorizes.

I really appreciate the research exploration you do around ideas of agency and am very happy to celebrate the writeups like this when you produce them.

Comment by Ben Pace (Benito) on Knowledge is not just map/territory resemblance · 2021-05-26T04:34:13.428Z · LW · GW

The original lesswrong 1.0 had the following header at the top of each page, pointing at a certain concept of map/territory resemblance:

I don't remember the image you show. I looked it up, I don't see this header on the wayback machine. I see a map atop this post in 2009 and then not too long after it becomes the grey texture that stayed until LW 2.0. Where did you get your image from?

Comment by Ben Pace (Benito) on Abstraction Talk · 2021-05-25T18:14:10.129Z · LW · GW

Yeah, we can have a try and see whether it ends up being worth publishing.

Comment by Ben Pace (Benito) on Abstraction Talk · 2021-05-25T17:36:40.755Z · LW · GW

Nice. I'll get a transcript made on Rev.com and share it with you for edits.

Comment by Ben Pace (Benito) on Testing The Natural Abstraction Hypothesis: Project Intro · 2021-05-24T02:39:02.791Z · LW · GW

Curated. This is a pretty compelling research line and seems to me like it has the potential to help us a great deal in understanding how to interface and understand and align machine intelligence systems. It's also the compilation of a bunch of good writing+work from you that I'd like to celebrate, and it's something of a mission statement for the ongoing work.

I generally love all the images and like the way it adds a bunch of prior ideas together.

Comment by Ben Pace (Benito) on My Journey to the Dark Side · 2021-05-19T16:26:33.663Z · LW · GW

I do mean to imply that animals that can pass the mirror test are much more morally meaningful than the others. Computer game characters also exhibit ”intentions” and such, but there’s nobody home a lot of the time, unless you’re playing against another person.

I am fairly interested in knowing which animals are being factory farmed that pass the mirror test. If it’s anything like cows then I will be pretty upset.

Comment by Ben Pace (Benito) on My Journey to the Dark Side · 2021-05-19T02:03:31.628Z · LW · GW

can you really make rational arguments that it's not at least as bad as a quantitatively reduced version of doing the same things to human beings?

Yep. The animals aren't meaningfully conscious. That's such an argument. (And one that I believe.)

Comment by Ben Pace (Benito) on Formal Inner Alignment, Prospectus · 2021-05-17T06:34:49.237Z · LW · GW

Curated. Solid attempt to formalize the core problem, and solid comment section from lots of people.

Comment by Ben Pace (Benito) on Concerning not getting lost · 2021-05-15T02:33:45.336Z · LW · GW

Thank you for writing this post. I wish you both strength and wisdom, in not getting lost.

Comment by Ben Pace (Benito) on Agency in Conway’s Game of Life · 2021-05-13T01:26:19.033Z · LW · GW

I recall once seeing someone say with 99.9% probability that the sun would still rise 100 million years from now, citing information about the life-cycle of stars like our sun. Someone else pointed out that this was clearly wrong, that by default that sun would be taken apart for fuel on that time scale, by us or some AI, and that this was a lesson in people's predictions about the future being highly inaccurate. 

But also, "the thing that means there won't be a sun sometime soon" is one of the things I'm pointing to when talking about "general intelligence". This post reminded me of that.

Comment by Ben Pace (Benito) on My Journey to the Dark Side · 2021-05-12T01:26:12.305Z · LW · GW

Thx for the comment.

There's a scientific-consensus answer to how many calories people typically need, the Harris-Benedict formula.

Out of interest, I checked wikipedia, which gives three different equations (Harris-Benedict, Mifflin St Jeor, and Katch-McArdle), saying "Historically, the most notable formula was the Harris–Benedict equation".

I just put in my details on them, and my daily caloric burn varied over 1000 calories (one was 500 less than the other which was 500 less than the other). One of them was pretty close to 2000. /shrug

Comment by Ben Pace (Benito) on Open and Welcome Thread - May 2021 · 2021-05-11T18:11:18.485Z · LW · GW

We have a list of bans and warnings here, though I'm not sure it's up to date. It certainly doesn't include all the bans of spam accounts and new user accounts.

Comment by Ben Pace (Benito) on There’s no such thing as a tree (phylogenetically) · 2021-05-10T04:03:01.654Z · LW · GW

Curated. This is a dope post, I really enjoyed boggling at the evolution of trees with you for five minutes. I'd love to read more posts like this.

Comment by Ben Pace (Benito) on MIRI location optimization (and related topics) discussion · 2021-05-09T20:40:19.075Z · LW · GW

Yeah. I think I'd currently also be willing to 2x my current rent to be able to sustain this level of weather relative to e.g. Manchester (where I grew up).

Comment by Ben Pace (Benito) on MIRI location optimization (and related topics) discussion · 2021-05-09T19:51:22.281Z · LW · GW

Sunlight changes my mood pretty significantly, and I’d definitely be sad to move to Seattle for this reason. Though it wouldn’t be a dealbreaker for me.

Comment by Ben Pace (Benito) on MIRI location optimization (and related topics) discussion · 2021-05-09T18:39:31.114Z · LW · GW

I’m curious for examples of properties that were ruled out and why.

Comment by Ben Pace (Benito) on MIRI location optimization (and related topics) discussion · 2021-05-09T10:07:52.261Z · LW · GW

This is a great comment, thanks. (Other people doing comments in this style would also be pretty great.)

Comment by Ben Pace (Benito) on My Journey to the Dark Side · 2021-05-07T01:18:51.763Z · LW · GW

A bunch of parts of civilization have nice barriers around them, and if you step outside of them you're at sea in a storm with no idea what's coming next. They tried to be disruptive+scary by chaining their car to an entrance way, and then suddenly there were kids involved and police with guns and helicopters. 

Step outside of the barriers at your own peril. I'm not saying that the barriers are just, but your life is more stable and easier-to-predict when you stay inside of them. And don't go outside of them if you're incompetent.

Comment by Ben Pace (Benito) on Your Dog is Even Smarter Than You Think · 2021-05-06T21:08:41.245Z · LW · GW

At what betting odds?

Comment by Ben Pace (Benito) on [Lecture Club] Awakening from the Meaning Crisis · 2021-05-05T20:38:09.447Z · LW · GW

I think laying out your thoughts on this would make a great top-level post. Starting from your comments here and then adding a bit more detail.

Comment by Ben Pace (Benito) on Your Dog is Even Smarter Than You Think · 2021-05-04T04:37:09.685Z · LW · GW

I watched the first two videos and was kinda shocked. I could see the dog thinking. Like, it stopped, was slow, then made a deliberate choice to click certain words. It was not something I believed a dog could do. I feel way more like I could chat with a dog. I have a bunch more empathy for dogs too now.

I think there’s a fair chance that somehow this is a mistake / a fluke, but have now visualized the world where it’s not, and it’s really cool.

(Thanks for the post! You may be interested in this other post on dogs that also increased my respect for them.)