Lightcone Infrastructure/LessWrong is looking for funding

With that in mind, I was surprised by the lack of information in this funding request. I feel mixed about this: high-status AIS orgs often (accurately) recognize that they don't really need to spend time justifying their funding requests, but I think this often harms community epistemics (e.g., by leading to situations where everyone is like "oh X org is great-- I totally support them" without actually knowing much about what work they're planning to do, what models they have, etc.)

Sorry about that! I've drafted like 3-4 different fundraising posts over the last few months, most of which were much longer and had more detail, but I also repeatedly ran into problems that when I showed them to people, they ended up misunderstanding how Lightcone relates to our future work and thinking about our efforts in a very narrow way by overfitting to all the details I put into the post, while missing something important about the way we expect to approach things over the coming years. 

I ended up deciding to instead publish a short post, expecting that people will write a lot of questions in the comments, and then to engage straightforwardly and transparently there, which felt like a way that was more likely to end up with shared understanding. Not sure whether this was the right call, and definitely a good chunk of my decisions here are also driven by finding fundraising to be the single most stressful aspect of running Lightcone, and I just find navigating the stress of that easier if I respond to things in a more reactive way.

What are Lightcone's plans for the next 3-6 months? (is it just going to involve continuing the projects that were listed?)

We are going to be wrapping up renovations and construction for the next month, and will then host a bunch of programs over the summer (like SERI MATS and a bunch of workshops and conferences). During that time I hope to reconnect a bit more with the surrounding AI-Alignment/X-Risk/Rationality/EA/Longtermist-adjacent diaspora, which I intentionally took a pretty big step away from after the collapse of FTX. 

I will also be putting a bunch of efforts into Lightspeed Grants. We will see how much traction we get here, but I definitely think there is a chance that blows up into the primary project we'll be working on for a while, since I think there is a lot of value in diversifying and improving the funding ecosystem, which currently seems to drive a lot of crazy status-dynamics and crazy epistemics within people working on AI risk stuff.

After that, I expect to focus a lot more on online things. I probably want to do a major revamp of the AI Alignment Forum, as well as focus a lot of my attention more on LessWrong again. I am particularly excited about finally properly launching the dialogues feature and driving adoption of that, probably in parts by me and other Lightcone team members participating in a lot of dialogues while we also continue developing the technology on the backend.

How is Lightcone orienting to the recent rise in interest in AI policy? Which policy/governance plans (if any) does Lightcone support?

I've been thinking a lot about this, and I don't yet have a clear answer. My tentative guess is something like "with a lot of the best people that I know going into AI policy stuff and the hype/excitement around that increasing, the comparative advantage of the Lightcone team is actually even more strongly pointing in the direction of focusing on research forums that ground the epistemic health of the people jumping headfirst into policy stuff". This means I currently expect to not get super deeply involved, but to interface a lot with people who are jumping into the policy fray, moving to DC, etc. and to figure out what infrastructure can allow those people to stay sane and grounded, since I do really expect that as we get more involved in advocacy, politics and policy-making that thinking clearly will become a lot harder. 

But again, I don't know yet, and at the meta-level I might just organize a bunch of events to help people orient to the shifting policy landscape while I am orienting myself to it as well.

What is Lightcone's general worldview/vibe these days? (Is it pretty much covered in this post? [LW · GW]) Where does Lightcone disagree with other folks who work on reducing existential risk?

This sure seems hard to answer concisely. Hopefully you can figure out our vibe from my comments and posts. I still endorse a lot of the post you linked, though also changed my mind on a bunch of stuff. I might write more here later. I think this is a valid question, this comment is just already getting very long and I don't have an immediate good cached answer. 

What are Lightcone's biggest "wins" and "losses" over the past ~3-6 months?

In my books I think by far the biggest win is that I think we relatively successfully handled a really major shift in relationship to the rest of our surrounding community, as an organization whose lifeblood is building infrastructure. My sense is that in all previous organization I've worked in, I would have rather left or shut the organization down when FTX collapsed, because I wouldn't have been able to think clearly and see things with fresh eyes, and orient with my organization to the changing landscape. I think Lightcone successfully handled reorienting together, and I think this is really hard (and probably the result of us staying consistently very small in our permanent staff headcount, which is currently just 8 people).

We also built an IMO really amazing campus that I expect to utilize and get a lot of value out of for the next few years. I also think I am proud of me writing a lot of things publicly on the EA Forum during the time of the FTX collapse and afterwards. I think it also helped the rest of the ecosystem orient better, and a lot of the things were things that nobody else was saying and seemed quite important.

We are just wrapping up renovations so not much yet (though we are done very soon). This summer we are likely hosting a good chunk of the SERI MATS scholars, as well as providing space for various other retreats and events (like the Singular Value Learning Theory workshop [LW · GW] and we are talking to Manifold about maybe running a 100+ person forecasting conference here). 

In-parallel we are also providing office space to a small number of people that I expect to slowly grow over time, trying to build a tight-knit community of people working to reduce existential risk and develop an art of rationality. Currently this includes John Wentworth, Adam Scholl, Sydney von Arx and a few other people. 

I am currently holding this part of our plan kind of lightly, and we might end up mostly just running events here, since I think a bunch of things didn't work amazingly well with the way we ran the Lightcone Offices, and I don't want the Rose Garden to become a place with similar dynamics where working here could end up feeling like some kind of status marker or like some necessary component to helping with existential risk when living in the Bay Area. 

Thank you!

I've been a casual LW fan off and on for a while, probably over a decade (friend introduced me to The Sequences either shortly after or shortly before completion) and this is the first time I've heard of Lightcone. A link to your main page might have been useful in addition to (not instead of; the summary is good) the summary. Yes we all know where Google is, but it's like not stamping a return envelope

We used to just be called the LessWrong team, but then we expanded our operations into more than just LW and changed our name. 

Agree that a link to our website would have still been useful. Just for posterity, here is one: https://www.lightconeinfrastructure.com/ 

Are you USA deductable?

Yep! We are a fiscally sponsored project by the Center for Applied Rationality, which is a 501c3 organization in the US.

How much are you asking per individual? I know it's a broadcast but I still strongly advise organizations on whose boards I sit to include a number anyway. "Whatever you want" is a bad answer. Also if someone asks for $1k "no" sometimes looks like $100, but if I'm willing to give $5k to my local fire department and their top ask is $50 I'm not writing a check for more than $50. (actual situation. After a few years I asked and it turns out they just don't need more money. Good on them.)

I think $1k seems like a reasonable ask for people who are heavy users of LessWrong and have benefitted a lot from it over the years. Though to keep existing I think some individuals/foundations would definitely have to give a bunch more than that.

Are there any investor incentives? Naming opportunities are fun, raffles maybe aren't a fit (though I was just talking with a friend about what an hour of EY answering a stupid question would auction for). MIRI's leaderboard always struck me as cool.

Not currently, though I also like the leaderboard! We do have a physical campus now, and it somehow never occurred to me that we could name rooms or buildings or other structures after people. 

My guess is if someone donates $10k+ there is probably an option of having a room or some structure on the campus be named after you or some other name of your choosing.

Up and down. What would you do if your current operating budget doubled? How much of a cut/shortfall is probably game over?

If our operating budget doubled we would probably hire slightly faster, though not enormously so (since I think hiring too fast is a very frequent way for organizations to fail). There are also a lot of projects that I would love to take on that would come with having access to more money. I've been wanting to run more AI Alignment events and conferences. 

But I think mostly it would lead to us having more runway, which would open us more generically to taking risks, and what exact risks we would take is really very uncertain right now. I didn't expect to be working on a hotel renovation for the last few months, and I didn't expect I would be doing a bunch of mechanism design for grantmaking a few years ago.

Are your financials public, and if not could you generate some manner of redacted-as-you-like p&l and balance sheet (and a lot of folks care more than me about cash flow statements. I don't, in most cases, but that's a whole other conversation)?

Our expenses for the last year are really very heavily loaded towards renovation and construction (most of which ends up capitalized in the property value on which we have a mortgage on), and looks roughly like this: 

However, this table is pretty misleading, in that it doesn't amortize our renovation expenses (which I think you want to amortize over something like 5-7 years to get an accurate sense of costs), does not take into account property appreciation, and also is really not very representative of future spending (this was a year of spending heavily supported by FTX, and we've made massive cuts to reduce that spending).

See also this comment on my thinking on the economics of a large real estate project like this: https://www.lesswrong.com/posts/9iDw6ugMPk7pmXuyW/lightcone-infrastructure-lesswrong-is-looking-for-funding?commentId=ZijNLGtN9KAD2i8hi [LW(p) · GW(p)] 

Here is a rough breakdown of how I expect us to spend money over the coming year: 

And then I expect those expenses to further go down next year, since we won't have any renovation expenses left, and revenue will have gone up, since we'll have more of a track record.

Your bullet points are great. Better than half I work with easily

Thank you! 

Are there any investor incentives? Naming opportunities are fun

If we set any new concrete, I think it could be neat to write donors' names into the concrete. Or maybe we can simply carve them into a wall somewhere.

Here are some things I like about owning this space:

• You don't have to ask anyone's permission to make modifications to it or to use it in unusual ways. For instance, if we want to cover every wall with whiteboards (as we've done in many rooms), we don't have to ask anyone's permission, we just order the boards, hire a few laborers, and have them cut them to size and nail them all along the walls. If I want to turn a 4-person team office into a 4-person bedroom, I just get a few people, carry the furniture into storage, carry some beds in, and we're done. This means that in the span of just a few days whole buildings can be turned from one use case into another (e.g. from housing into a conference venue), as demand comes up. This is not true of rented office spaces (where you certainly cannot reliably sleep many people or do surgery to the walls) or chain hotels (I cannot ask a hotel to use a room as a team office for 3 montsh), and most residential spaces are far too small. As another datapoint about the limitations of rented spaces, the last office building I was using wouldn't even allow us to have a fully-dark nap room due to regulations about lighting visibility in the event of a fire.
• This space is much more aesthetically beautiful and harmonious with our intentions than most alternatives we could find in Berkeley. It is relatively central in Berkeley, yet has a large, private outdoor area that you can work from in the sun all day. The rooms are not inhuman exact squares with sterile lighting, they're all unique shapes, often with nice old woods and we can install our own lighting. We've had the space to build a number of wooden outdoor huts to work from with electricity and soon-to-be-heated cushions. In my model of the world the aesthetics of a space really affect the sorts of affordances and feelings and problem-solving-approaches that people will consider. For example I think this sort of natural beauty helps people be more willing to go deep on an idea or hypothesis for a longer amount of time.
• The space is made of 5 separate buildings, which gives a little village-like feel, and (we're hoping) can allow some fairly different things to be happening on-site at the same time. In one building you could have a self-contained retreat (CFAR workshop, AI alignment workshop, etc), in another building a few x-risk focused teams could be getting work done in their offices as they have ever day for the last 2 months, another building could be having a weekly communal lunch open to 100+ people as well as hosting a few visitors in rooms upstairs who have flown in from out of town for a few weeks, etc. My hope is that the separate buildings can allow a lot of different things to be happening concurrently, and have natural boundaries between them.

Perhaps of interest, when we were considering alternative locations, the main other places that had the properties of my first two bullets were educational religious spaces. The School of Religion, the School of Theology, and a strange surprise-Buddhist-temple that Habryka and I unexpectedly found ourselves in one evening (as the woman was showing us around the school-like building, she fully walked past the temple doors, until I politely asked to look inside, and she unlocked them to show us a ~7k square foot room with a 40-foot high ceiling, filled with golden statues and colorful ribbons hanging from the ceiling and ancient texts inscribed on rotating pillars and 300 folding chairs and a big stage). These places had a lot of beauty. But one of them basically wasn't on sale, and the other two were only partially on sale (we couldn't have owned the whole property and would have to share with some religious groups, which is not a total dealbreaker but I strongly prefer having full ownership). 

We also considered renting solely office spaces, which would have been much faster to get started with, and were on the verge of going through on a deal last year. But then at the last minute they explained the elevator needed replacing and would be out of use for the first 2 months of us living there (which is a pretty big obstruction for moving in all of our heavy furniture up ~3 floors). They wouldn't negotiate at all on this and we walked away. I actually heard (epistemic status: I assign 75% to this being true, I have pinged the person who said this to me to double-check) that the elevator only actually got fixed around a month or two ago. To me not having to deal with this sort of thing is part of the advantage of having full-ownership I describe in the first bullet above.

Will much of that $3-6M go into renovating and managing the Rose Garden Inn, or to cover work that could have been covered by existing funding if the Inn wasn't purchased?

Thinking about the exact financing of the Inn is a bit messy, especially if we compare it to doing something like running the Lightcone Offices, because of stuff like property appreciation, rental income from people hosting events here, and the hard-to-quantify costs of tying up capital in real estate as opposed to more liquid assets like stocks.

If you assume something like 5% property appreciation per year going forward, and you amortize the part of our construction costs that didn't increase the property value over the next 7 years (which seems like a reasonable estimate for how long the venue is going to get used), I get an annual cost of running the Rose Garden of about $1.15M (property value of ~$19MM, ~$3.5M in uncapitalized construction costs amortized over 7 years, plus $700k in annual upkeep and maintenance costs). 

This is pretty cost-effective compared to other options of even just providing office space, and I think is my preferred way of thinking about the cost of doing this construction project (most of which came from a loan that we took out specifically to finance the purchase and renovation of the place, which is insured against the Rose Garden property itself, so it didn't straightforwardly funge against our other funding). 

To compare this to other costs, renting two floors of the WeWork, which we did for most of the summer last year, cost around $1.2M/yr for 14,000 sq. ft. of office space. The Rose Garden has 20,000 sq. ft. of floor space and 20,000 additional sq. ft. of usable outdoor space for less implied annual cost than that.

In order to fund this initial capital investment, we did definitely cut into the funding we received in 2022, and we would have a bunch of money in our bank accounts right now if we didn't do this construction project. On the other hand, our spending over the next few years would also be a bunch larger, since we would have to pay more in rent if we wanted to do anything in-person community shaped, for both events and office space, than we will have to pay in interest and upkeep minus appreciation for the Rose Garden Inn. 

This doesn't take into account the cost of having Jaan Tallinn's capital tied up in real estate with 5% annual interest from us. In one sense it seems good for Jaan to diversify his funds away from crypto. On the other hand he has probably been making closer to 10%-20% annual returns over the last 10-15 years with his investments. My current guess is the diversification benefits here outweigh the higher potential returns (because man, x-risk efforts sure still are quite tech-stock and crypto loaded and I am very glad about more diversification), but I am really not confident, and this could easily dominate the relevant cost-calculations. 

It's also a bit unclear how to think about rental income from people working on existential risk stuff that we wouldn't otherwise support, which I expect to also make us back some of these funds.

I could write more about how to think about financing the Rose Garden, but I'll leave it at this. I think it it mostly makes sense for people to think that roughly $1.5M of our annual budget going forward will be used for our Rose Garden Inn, with ~$350k of that going into very illiquid long-term savings for the organization.

=====

By far the biggest source of uncertainty about our budget comes from the potential for FTX clawbacks, which explains most of the wide range of our budget. I currently think there are some good arguments that we owe back about $1.5M to FTX creditors, though the exact game theory here is a bit messy (and they might request up to $4M back from us), and I continue to feel quite confused about what the right thing to do here is. We don't know if and when they will ask for that money.

If so, I'm curious to hear more about the strategy behind buying and renovating the space, since it seems like a substantial capital investment, and a divergence from LightCone Infrastructure's previous work and areas of expertise.

The purchase of the Rose Garden Inn and really setting the whole renovation project in-motion was made before November last year, and as such before the collapse of FTX, and as is probably apparent from my comments and posts over the last few months (as well as our reasoning for closing down the Lightcone Offices), a lot of our strategy and perspective has shifted since then. 

That said, I am actually still quite excited about our plans with the Rose Garden, but I do think it's an important disclaimer to add that yeah, a lot of our plans for the Rose Garden were substantially changed when FTX collapsed, and our plans for it have become a bunch less straightforward (the original reasoning for it was something much closer to "this makes sense from a finance perspective even if we just compare it to renting office space at our current location"). 

That said I am still quite excited about our plans. Some concrete things that continue to drive my excitement for our Rose Garden Inn investment: 

• I continue to think that in-person collaboration is really essential, especially for mentoring relationships. I think there continues to be a pretty substantial bottleneck in people getting oriented to working on AI Alignment that goes a lot better with in-person peers and via in-person mentoring from people who have made real contributions to the field, and investing in infrastructure for that is one of the best ways to drive progress on reducing existential risk from AI.
• I really want to create a more distinct and intentionally separate culture both on LessWrong and at the Rose Garden Inn, and I think owning a physical space hugely helps with that. FTX, various experiences I've had in the EA space over the past few years, as well as a lot of safetywashing in AI Alignment in more recent years, have made me much more hesitant to build a community that can as easily get swept up in respectability cascades and get exploited as easily by bad actors, and I really want to develop a more intentional culture in what we are building here. Hopefully this will enable the people I am supporting to work on things like AI Alignment without making the world overall worse, or displaying low-integrity behavior, or get taken advantage of. 
• I am very excited about the setup where we get to have both long-term office space and short-term events in one location. I am hopeful that this will enable us to have a lot of people flowing through the space, and will allow us to interface and experiment with a lot of different approaches to AI alignment and rationality, and then slowly grow by picking a small number of the people coming through to work more long-term from our office space.

For the record, our relationship to supporting events for this ecosystem is changing from something like "all of our resources are the same, here have my venue for free if you need it" to "markets and pricing are a great way for large masses of people to coordinate on the value of a good or service, let's coordinate substantially via trade". 

For instance, during a previous cohort of SERI MATS scholars at the Lightcone Offices, I spent a couple of weeks of work adding a second floor and getting it furnished and doing interior design, hiring another support person to the office team, and then later on dealing with closing it down and downsizing when the demand went away. I did all of that for free, and was not paid salary or anything by MATS, it was part of my Lightcone work, because we wanted to support mentorship happening in the AI alignment ecosystem. It's different this time around. They're paying us a substantial amount of money (well over $100k) for the use of 2.5 of our nicely furnished and designed buildings for 2 months, an amount that makes the trade pretty good for Lightcone (and I hope+expect to work hard and make it worthwhile for SERI MATS too!). The other workshops Habryka has mentioned elsethread will also mostly be paying trade partners (general pricing TBD as we get a better sense of the demand).

I bring this up because the extent to which funds for Lightcone are spent supporting SERI MATS in particular (and other teams/orgs/events) is (I suspect) much less than you are thinking.

At least for the coming year, our expenses are pretty entangled between all the different projects in a way that makes differentially funding things hard. I do take preferences of our donors on how to focus our efforts into account, so donating and just telling us that you would prefer us to work more on one kind of thing vs. another will have some effect. 

My guess is you will mostly just have to average our impact across different areas and decide whether the whole portfolio is above your bar.

Ehh... feels like your base rate of 10% for LW users who are willing to pay for a subscription is too high, especially seeing how the 'free' version would still offer everything I (and presumably others) care about. Generalizing to other platforms, this feels closest to Twitter's situation with Twitter Blue, whose rates appear is far, far lower: if we be generous and say they have one million subscribers, then out of the 41.5 million monetizable daily active users they currently have, this would suggest a base rate of less than 3%.

There is a Coinbase donate button on this page: https://rationality.org/donate 

Let me know if that doesn't work for you, and we can also figure out a direct ETH address.