Open & Welcome Thread — February 2023

To give an update about what the Lightcone Infrastructure team has been working on: we recently decided to close down a big project we'd been running for the last 1.5 years, an office space in Berkeley for people working on x-risk/EA/rationalist things that we opened August 2021.

At some point I hope we publish a postmortem, but for now here's a copy of the announcement I wrote in the office slack announcing its closure 2-3 week ago.

Hello there everyone,

Sadly, I'm here to write that we've decided to close down the Lightcone Offices by the end of March. While we initially intended to transplant the office to the Rose Garden Inn, Oliver has decided (and I am on the same page about this decision) to make a clean break going forward to allow us to step back and renegotiate our relationship to the entire EA/longtermist ecosystem, as well as change what products and services we build.

Below I'll give context on the decision and other details, but the main practical information is that the office will no longer be open after Friday March 24th. (There will be a goodbye party on that day.)

I asked Oli to briefly state his reasoning for this decision, here's what he says:

An explicit part of my impact model for the Lightcone Offices has been that its value was substantially dependent on the existing EA/AI Alignment/Rationality ecosystem being roughly on track to solve the world's most important problems, and that while there are issues, pouring gas into this existing engine, and ironing out its bugs and problems, is one of the most valuable things to do in the world.

I had been doubting this assumption of our strategy for a while, even before FTX. Over the past year (with a substantial boost by the FTX collapse) my actual trust in this ecosystem and interest in pouring gas into this existing engine has greatly declined, and I now stand before what I have helped built with great doubts about whether it all will be or has been good for the world.

I respect many of the people working here, and I am glad about the overall effect of Lightcone on this ecosystem we have built, and am excited about many of the individuals in the space, and probably in many, maybe even most, future worlds I will come back with new conviction to invest and build out this community that I have been building infrastructure for for almost a full decade. But right now, I think both me and the rest of Lightcone need some space to reconsider our relationship to this whole ecosystem, and I currently assign enough probability that building things in the space is harmful for the world that I can't really justify the level of effort and energy and money that Lightcone has been investing into doing things that pretty indiscriminately grow and accelerate the things around us.

(To Oli's points I'll add to this that it's also an ongoing cost in terms of time, effort, stress, and in terms of a lack of organizational focus on the other ideas and projects we'd like to pursue.)

Oli, myself, and the rest of the Lightcone team will be available to discuss more about this in the channel #closing-office-reasoning where I invite any and all of you who wish to to discuss this with me, the rest of the lightcone team, and each other.

In the last few weeks I sat down and interviewed people leading the 3 orgs whose primary office is here (FAR, AI Impacts, and Encultured) and 13 other individual contributors. I asked about how this would affect them, how we could ease the change, and generally get their feelings about how the ecosystem is working out. These conversations lasted on average 45 mins each, and it was very interesting to hear people's thoughts about this, and also their suggestions about other things Lightcone could work on.

These conversations also left me feeling more hopeful about building related community-infrastructure in the future, as I learned of a number of positive effects that I wasn't aware of. These conversations all felt pretty real, I respect all the people involved more, and I hope to talk to many more of you at length before we close.

From the check-ins I've done with people, this seems to me to be enough time to not disrupt any SERI MATS mentorships, and to give the orgs here a comfortable enough amount of time to make new plans, but if this does put you in a tight spot, please talk to us and we'll see how we can help.

The campus team (me, Oli, Jacob, Rafe) will be in the office for lunch tomorrow (Friday at 1pm) to discuss any and all of this with you. We'd like to know how this is affecting you, and I'd really like to know about costs this has for you that I'm not aware of. Please feel free (and encouraged) to just chat with us in your lightcone channels (or in any of the public office channels too).

Otherwise, a few notes:

• The Lighthouse system is going away when the leases end. Lighthouse 1 has closed, and Lighthouse 2 will continue to be open for a few more months.
• If you would like to start renting your room yourself from WeWork, I can introduce you to our point of contact, who I think would be glad to continue to rent the offices. Offices cost between $1k and $6k a month depending on how many desks are in them.
• Here's a form to give the Lightcone team anonymous feedback about this decision (or anything).
• To talk with people about future plans starting now and after the offices close, whether to propose plans or just to let others know what you'll be doing, I've made the #future-plans channel and added you all to it.

It's been a thrilling experience to work alongside and get to know so many people dedicated to preventing an existential catastrophe, and I've made many new friends working here, thank you, but I think me and the Lightcone Team need space to reflect and to build something better if Earth is going to have a shot at aligning the AGIs we build.

https://www.youtube.com/watch?v=8Fow61Zsn2s

Hi all, 

nice to meet you! I've been silently reading LW for the past 4ish years (really enjoyed it), even went to CFAR workshops (really enjoyed it), I regard myself as a rationalist overall (usually enjoy it, sometimes, as expected, it's a bit hard) but so far felt too shy to comment. But now I have a question, so well, it leaves me with a very little choice ;). 

Is it possible to export LW series to PDF? I don't like reading on my computer, accessing forum from my e-reader each time is a pain, and I really would like to read, for example, the whole "2022 MIRI alignment discussion". Any advice here? 

LessWrong open thread, I love it. I hope they become as lively as the ACX Open Threads (if they have the same intention).

I'm reading the sequences this year (1/day, motivated by this post [EA · GW]) and am enjoying it so far. Lmk if I'm wasting my time by not "just" reading the highlights.

PS: In case you or someone you know is looking for a software engineer, here's my profile: https://cv.martinmilbradt.de/. Preferably freelance, but I'm open to employment if the project is impactful or innovative.

I remember seeing a short fiction story about a EURISKO-type AI system taking over the world in the 1980s (?) here on LessWrong, but I can't find it via search engines, to the point that I wonder whether I hallucinated this. Does anyone have an idea where to find this story?

I've been on lesswrong every day for almost a year now, and I'm really interested in intelligence amplification/heavy rationality boosting. 

I have a complicated but solid plan to read the sequences and implement the CFAR handbook over the next few months (important since you can only read them the first time once).

I need a third thing to do simultaneously with the sequences and the CFAR handbook. It's gotta be three. What is the best thing I can do for heavy intelligence/rationality amplification? Is it possible to ask a CFAR employee/alumni without being a bother? (I do AI policy, not technical alignment)

Feature suggestion: Allow one to sort a user's comments by the number of votes.

Context: I saw a comment [LW(p) · GW(p)] by Paul Christiano, and realized that probably a significant portion of the views expressed by a person lie in comments, not top-level posts. However, many people (such as Christiano) have written a lot of comments, so sorting them would allow one to find more valuable comments more easily.

Is there anyone who'd be willing to intensively (that is, over a period of possibly months, at least one hour a day) help someone with terminal spaghetti brain organize a mountain of ideas about how to design a human hive mind (that is: system for maximizing collective human intelligence and coordination ability) operating without BCIs, using only factored cognition, prediction markets, behaviorist psychology, and LLMs?

I cannot pay you except with the amusement inherent to my companionship - I literally have no money and live with my parents lol - but one of the things for which I've been using my prodigious free time the past few years is inventing-in-my-head a system which at this point is too complicated for me to be able to figure out how to explain clearly in one go, although it feels to me as if it's rooted ultimately in simple principles.

I've never been able to organize my thoughts, in general, on any topic, better than a totally unordered ramble that I cannot afterward figure out any way to revise - and I mean, I am pathologically bad at this, as a personality trait which has remained immune to practice, which is why I rarely write top level posts here - so I actually need someone who is as good at it as I am bad and can afford to put a lot of time and effort into helping me translate my rambles into a coherent, actionable outline or better yet, wiki full of pseudocode and diagrams.

Obviously I will find some way to reciprocate, but probably in another way; I doubt I can do much good towards organizing your stuff if I can't organize my own lol. But who knows, maybe it's my closeness to the topic that makes me unable to do so? Anyway, thanks in advance if you decide to take this on.

Hello, I'm new here and still working on reading the Sequences and the other amazing content on here; hopefully then I'll feel more able to join in some of the discussions and things. For now, I have what I'm sure is an embarrassingly basic question, but I can't find an answer on here anywhere and it keeps distracting me from focusing on the content: would someone please tell me what's the deal with the little degree symbols after some links but not others?

Thank you in advance, and warm regards to you all.

LW/AF did something very recently that broke/hindered the text to speech software I use for a lot of my LW reading.

This is a considerable inconvenience.

Feature request: An option to sort comments by agreement-karma.

I have a vague preference to switch from my nickname here to my real name. Are there any unexpected upsides or downsides? (To the extent that matters, I'm no-one important, probably nobody knows me, and in any case it's probably already easy to find out my real name.)

Plus in case I go through with it, are there any recommendations for how to ease the transition? (I've seen some users temporarily use nicknames like <Real Name, formerly MondSemmel> or something, though preferably as a shorter version.)

Question for people working in AI Safety: Why are researchers generally dismissive of the notion that a subhuman level AI could pose an existential risk? I see a lot of attention paid to the risks a superintelligence would pose, but what prevents, say, an AI model capable of producing biological weapons from also being an existential threat, particularly if the model is operated by a person with malicious or misguided intentions?

"Games we play": civilians helping troops from different sides of conflict. As in, Army I entered the village; N sold his neighbors, the neighbors died horribly. Then Army II chased away Army I. Would N be reported as a collaborationist? Commonly, no. But everybody knows that everybody knows. And everybody knows who knows what everybody knows, which means N is probably going to sell a lot of people if another opportunity arises.