@Ruby @RobertM 

I mistyped a bit with the use of "relationships". Yes, names and faces both trigger social recognition, but I meant to make the point that they operate in significantly different ways in the brain, and facial recognition is tuned to processing a lot of emotional and social cues that we aren't tuned to from text. I have tons of social associations with people's physical forms that are beyond simply their character.

(A language model helped me write this comment.)

I did find it and we sent him an email, hope he reads it and joins :)

I am fairly strongly against having faces, which I think boot up a lot of social instincts that I disprefer on LessWrong. LessWrong is a space where what matters is which argument is true, not who you like / have relationships with. I think some other sort of unique icon could be good.

Here's a quick mockup of what that might look like.

In my head you the voting UI is available after you click to expand and then scroll down to the bottom of the comment.

Added: Oops, I realize I did the quick takes, not the popular comments. Still, the relevant changes are very similar.

Most people do not have the constitution or agency for criminal murder

I think my model of people is that people are very much changed by the affordances that society gives them and the pressures they are under. In contrast with this statement, a lot of hunter-gatherer people had to be able to fight to the death, so I don't buy that it's entirely about the human constitution. I think if it was a known thing that you could hire an assassin on an employee and unless you messed up and left quite explicit evidence connecting you, you'd get away with it, then there'd be enough pressures to cause people in-extremis to do it a few times per year even in just high-stakes business settings. Also my impression is that business or political assassinations exist to this day in many countries; a little searching suggests Russia, Mexico, Venezuela, possibly Nigeria, and more.

I generally put a lot more importance on tracking which norms are actually being endorsed and enforced by the group / society as opposed to primarily counting on individual ethical reasoning or individual ethical consciences.

(TBC I also am not currently buying that this is an assassination in the US, but I didn't find this reasoning compelling.)

Done.

I wouldn't call this "AI lab watch." "Lab" has the connotation that these are small projects instead of multibillion dollar corporate behemoths.

This seems like a good point. Here's a quick babble of alts (folks could react with a thumbs-up on ones that they think are good).

AI Corporation Watch | AI Mega-Corp Watch | AI Company Watch | AI Industry Watch | AI Firm Watch | AI Behemoth Watch | AI Colossus Watch | AI Juggernaut Watch | AI Future Watch

I currently think "AI Corporation Watch" is more accurate. "Labs" feels like a research team, but I think these orgs are far far far more influenced by market forces than is suggested by "lab", and "corporation" communicates that. I also think the goal here is not to point to all companies that do anything with AI (e.g. midjourney) but to focus on the few massive orgs that are having the most influence on the path and standards of the industry, and to my eye "corporation" has that association more than "company". Definitely not sure though.

What's the chance of a 2nd LessOnline?

Um, one part of me is (as is not uncommon) really believes in this event and thinks it's going to be the best effort investments Lightcone's ever made (though this part of me currently has one or two other projects and ideas that it believes in maybe even more strongly), that's part of me is like "yeah this should absolutely happen every year", though as I say I get this feeling often about projects that often end up looking different to how I dreamed them when they finally show up in reality. I think that part would feel validated by the event turning out to be awesome and people finding it was worthwhile to come. Then there's the question of how much resources Lightcone actually has and whether we'll successfully fundraise and whether this will be one of the few projects we're investing a few staff-months in a year from now. I think my probabilities just went from 80% to 50% to 99% to... 30%. Overall it depends on how good this event is, which varies on a log scale.

I think there are worlds where we do it again and invest less effort into it, also there's worlds where we do it again and invest more effort into it. I think there's also a bunch of worlds where we're happy about this event but try a subtly different one next time (e.g. me and a teammate generated like 5 other serious event contenders before this one, including things more like workshops or academic conferences than like large festivals, and perhaps we'll try a different thing next). I think I like that this event is essentially open-invite and trying to be more big-tent, and I hope to do more things like this, so that even if we change what sorts of events we run anyone will just be able to buy a ticket. There's also worlds where we stop exploring having an events team for our campus and stop running events.

As one datapoint, in 2021 I organized a 60-person private event called the Sanity & Survival Summit for rationalist folk and folks working professionally on x-risk stuff, and I thought we'd maybe make that a yearly thing, and a year later we sort of last-minute/impromptu ran another version of it called Palmcone (it was in the Bahamas) for 80-100 people, and then we made the Lightcone Offices to try and get a more permanent version of the EA/x-risk things in the Bay, and then we scrapped the whole thing as we uninvested in the professional x-risk/EA ecosystem. That's a possible trajectory things could take.

1. I anticipate the vast majority of people going to each of the events will be locals to the state and landmass respectively, so I don't think it's actually particularly costly for them to overlap.
2. That's unfortunate that you are less likely to come, and I'm glad to get the feedback. I could primarily reply with reasons why I think it was the right call (e.g. helpful for getting the event off the ground, helpful for pinpointing the sort of ideas+writing the event is celebrating, I think it's prosocial for me to be open about info like this generally, etc) but I don't think that engages with the fact that it left you personally less likely to come. I still overall think if the event sounds like a good time to you (e.g. interesting conversations with people you'd like to talk to and/or exciting activities) and it's worth the cost to you then I hope you come :-)

I'm probably missing something simple, but what is 356? I was expecting a probability or a percent, but that number is neither.

Does anyone know if it's typically the case that people under gag orders about their NDAs can talk to other people who they know signed the same NDAs? That is, if a bunch of people quit a company and all have signed self-silencing NDAs, are they normally allowed to talk to each other about why they quit and commiserate about the costs of their silence?

Preface: I think this comment will be pretty unpopular here.

I think this is a very unhelpful frame for any discussion (especially so the more high-stakes it is) for the reasons that SlateStarCodex outlines in Against Bravery Debates, and I think your comment would be better with this removed.

Added: I appreciate the edit :)

Yep! My guess is I will send one out to people who bought tickets next week, along with various spreadsheets for signing up to activities with (e.g. giving a lightning talk).

(I personally strongly prefer slack for a bunch of UI reasons including threading and especially because I always find the conversational culture on discord disorienting, though I know manifest has a community discord so it might be worth using discord.)

Launched a few days ago, the plan is:

• Kids tickets are $50
• There's daycare purchasable on-site from 10am to 7pm, for like $10/hour if you book ahead of time or $30/hour if you use it on-the-day
• If you want connection to a nanny for outside of those hours we have a service that can help with that at $45/hour.

Happy to get feedback on this, still figuring out what exactly helps parents and how to set it up right.

[Added April 28th: In case someone reads my comment without this context: David has made a number of worthwhile contributions to discussions of biological existential risks (e.g. 1, 2, 3) as well as worked professionally in this area and his contributions on this topic are quite often well-worth engaging with. Here I just intended to add that in my opinion early on in the covid pandemic he messed up pretty badly in one or two critical discussions around mask effectiveness and censoring criticism of the CDC. Perhaps that's not saying much because the base rate for relevant experts dealing with Covid is also that they were very off-the-mark. Furthermore David's June 2020 post-mortem of his mistakes was a good public service even while I don't agree with his self-assessment in all cases. Overall I think his arguments are often well-worth engaging with.]

I'm not in touch with the ground truth in this case, but for those reading along without knowing the context, I'll mention that it wouldn't be the first time that David has misrepresented what people in the Effective Altruism Biorisk professional network believe^[1]. 

(I will mention that David later apologized for handling that situation poorly and wasting people's time^[2], which I think reflects positively on him.)

1. ^{^}

   See Habryka's response to Davidmanheim's comment here from March 7th 2020, such as this quote.

   Overall, my sense is that you made a prediction that people in biorisk would consider this post an infohazard that had to be prevented from spreading (you also reported this post to the admins, saying that we should "talk to someone who works in biorisk at at FHI, Openphil, etc. to confirm that this is a really bad idea").

   We have now done so, and in this case others did not share your assessment (and I expect most other experts would give broadly the same response).

2. ^{^}

   See David's own June 25th reply to the same comment.

The first sounds like the sort of thing that turns out to be surprisingly useful (nobody ever gives me health advice). Mm, maybe folks can agree-react to this sentence if they too want to go to such a session?

Nice, just had a good call with Alkjash, who is coming and will be preparing 2 layman-level math talks about questions he's been thinking about.

Other ideas we chatted about having at LessOnline include maybe having some discussions about doing research inside and outside of academia, and also about learning from GlowFic writers how to write well collaboratively. (Let me know if you'd be interested in either of these!)

I think on-site housing is pretty scarce, though we're going to make more high-density rooms in response to demand for that. Tickets aren't scarce, our venue could fit like a 700 person event, so I don't expect to hit the limits.

Still working on setting it up, once I have the details I'll announce them (e.g. pricing and whatnot).

I'm aiming to have childcare available in some form for the full 9-day LessOnline-to-Summer-Camp-to-Manifest period. I'm excited for folks to come with their full families.

Also here are some sessions tentatively scheduled (some may change):

1. Fighting Moloch in Politics talk/Q&A with Martin Sustrik
2. One-Shot 'Baba Is You' Rationality Exercises activity with Raymond Arnold
3. Write Your First Fact-Post activity led by Sarah Constantin
4. Currently Untitled Sequel to And All the Shoggoths Merely Players narrated by Zack Davis and John Wentworth
5. Write Your First Glowfic activity led by Alicorn
6. Podcast and Q&A with Alexander Wales (author of Worth the Candle) and Daystar Eld, moderated by Jamie Wahls
7. Wanted: People Who Want by Jacob Falkovich
8. Magic-The-Gathering Color Wheel for Writers talk by Duncan Sabien

Question: What is this event celebrating?

I've been trying to think a bit about the boundary of this event. Naturally I love LessWrong, but it's not simply about LessWrong.

This is not comprehensive, but here are three subcultures that I admire and am hoping to celebrate at this event.

1. Aspiring Rationalists – people interested in understanding and improving their cognitive algorithms
2. Rational Fiction Readers & Writers – people who write fiction that illustrate intelligent characters and lawful worlds
3. Public Worldview Builders and/or Polymaths – people earnestly trying to directly understand the world and explain what they’ve learned with online writing

Below are a few notes about each of them and what I hope from bringing them together.

Aspiring Rationalists

I continue to believe the mission to understand our cognitive algorithms and improve them is a worthwhile one, and there's still lots of people working on advancing this, whether it's finding new theoretical underpinnings for decision-making or coming up with pragmatic heuristics for improving the truth-tracking nature of discussions or building up models for how the world is trying to fight your ability to understand it or something quite different.

I'm hoping to bring folks together who are interested in this mission and give them the ability to get ideas from other people who have been working on this too.

Rational Fiction Readers & Writers

Near to the aspiring rationalists is Rational Fiction. For detailed pointers to it check the description in the r/rational subreddit sidebar or the TV Tropes Page, but broadly it really respects the characters' intelligence and the rules of how the fictional world works. It's the sort of story where a key plot point might involve doing a fermi estimate for how much steel is required to withstand a given magical blast, or where a key moment of dramatic tension involves the protagonist carefully reasoning about their situation and reaching a novel conclusion, one that in-principle the reader themselves could've deduced given the information that they had available.

This community is very active and large! r/rational has 25k subscribers, and the best works are extremely popular. At the time of publication Harry Potter and the Methods of Rationality had the most reviews (37k) of any work on FanFiction.net (now second most) and is the most followed HP-fic (22k), Mother of Learning has the most views (18M) on Royal Road, and Worth the Candle is the 3rd most read original fiction on Archive of Our Own (I started it a month ago and am now 220 chapters in, no spoilers please). There's so many more amazing works, I'd guess like 300-1,000 rational-fics (especially counting all of the Glow-Fics).

I think this often (not always) has a similar nature to the old hard sci-fi, where people try to set up rules of a world and really ask what would follow from these rules, and authors put in a lot of work to be very creative-and-lawful.

I’m hoping to bring people together to talk about stories and the ideas that go into them and come away with lots of new ideas for stories they could write.

Public Worldview Builders and/or Polymaths

(This bit is something Ray helped a lot in writing)

There’s a style of discourse that has felt pretty important to me. It used to be kinda The Central Thing going on on the internet. It’s become less so, with the rise of social media. That Thing is a bit hard to pin down, but it’s something like:  

“Longform, thoughtful writing, earnestly trying to figure stuff out.”  

LessOnline is a festival celebrating that kind of writing. Furthermore, I have a strong hunch that there’s a craft that everyone invited is involved with in some way — an art of paying attention to the information you get about the world, piecing things together into a coherent perspective, extracting strong arguments to show to others, writing in a way that’s engaging and clear, and leaving people with a little bit of a better understanding of the world, that altogether hopefully adds up to leave me and humanity with a much more accurate and more leveraged map of the world.

The idea here is to invite the part of the blogosphere that seems sort of like "kindred spirits" to LessWrong, approaching similar kinds of questions from different perspectives. I think one framing is "we're looking for people who are publicly building out cohesive worldview in public."

I’m hoping to bring such people together to share ideas they’ve been thinking about and knowledge they’ve learned, and also to share insights about the shared craft. 

An quick and incomplete list of other subcultures I'm excited about bringing together

Econ Blogosphere, Statistics Blogosphere, History Blogosphere, Finance Blogosphere, The Nerdy Cartoon-O-Sphere (e.g. XKCD, SMBC, Abstruse Goose, many more), Agent Foundations researchers, Genetic Enhancement researchers, more.

Not sure where the right place to raise this complaint, but having just seen it for the first time, really, "MARS"? I checked, this is not affiliated with MATS who have had like 6 programs and ~300 people go through it. To me this seems too close in branding space to me, and I'd recommend picking a more distinct name.

Oh cool! Um, first thought, register interest in this?

Update #1

Lots of info to share! Here's a bunch of awesome people confirmed as coming.

Two activities I'm personally quite excited to go to are Sarah Constantin's "Write Your First Fact Post" and Alicorn's "Write Your First GlowFic" (I want to try to do both of these things!).

Questions Thread

Thinking Thread

Updates Thread

I don't believe so. The gossip I've heard leads me to think that it was substantially downstream of attempts to scale the institute by GovAI and Research Scholars program (involving hiring lots of junior people, which the university doesn't like), as well as the university in general being a terrible bureaucracy (e.g. as incompetent and micromanage-y as "remove the plants you bought because the university cannot commit to watering them").

@Zach Stein-Perlman I'm not really sure why you gave a thumbs-down. Probably you're not trying to communicate that you think there shouldn't be deontological injunctions against genocide. I think someone renouncing any deontological injunctions against such devastating and irreversible actions would be both pretty scary and reprehensible. But I failed to come up with a different hypothesis for what you are communicating with a thumbs-down on that statement (to be clear I wouldn't be surprised if you provided one).

Done.

I have been around the community for a long time (since before Anthropic existed), in a very involved way, and I had also basically never heard of this before the Claude 3 release.... So... it was definitely not the case that this was just an obvious thing that everyone in the community knew about or that Anthropic senior staff were regularly saying

For the record a different Anthropic staffer told me confidently that it was widespread in 2022, the year before you joined, so I think you're wrong here.

(The staffer preferred that I not quote them verbatim in public so I've DM'd you a direct quote.)

Did you not read the discussion from when this came out? Your post doesn't engage with the primary criticism which is about Anthropic staff lying about the company's plans, including the evidence suggesting the CEO lied to one of their investors leaving the investor with the belief that Anthropic had a “commitment to not meaningfully advance the frontier with a launch”. In contrast to the private communications with an investor, here's what Anthropic's launch post says of their new model (emphasis added):

Today, we're announcing the Claude 3 model family, which sets new industry benchmarks across a wide range of cognitive tasks... Opus, our most intelligent model, outperforms its peers on most of the common evaluation benchmarks for AI systems, including undergraduate level expert knowledge (MMLU), graduate level expert reasoning (GPQA), basic mathematics (GSM8K), and more. It exhibits near-human levels of comprehension and fluency on complex tasks, leading the frontier of general intelligence.

Two obvious points:

1. It is deontologically more ethical to not yourself kill everyone in the world.
2. America has an incredible ability to set fashions, and if it took on these policies then I think a great number of others would follow suit.

That interview is hysterically funny. 

I think as a 15 year old I could've had as hard a time as Metz did there. I mean, not if it were a normal conversation, then I would be far more curious about the questions being brought up, but if it was a first connection and I was trying to build a relationship and interview someone for information in a high-stakes situation, then I can imagine me being very confused and just trying to placate (and at that age I may have given a false pretense of understanding my interlocutor).

Yet from my current position everything Vassar said was quite straightforward (though the annotations were helpful).

I would prefer not to have people reply to me about people's personal sexual activities/events (without exceptional reason such as a credible accusation of rape or other criminality). 

I also do not think that attendees of people's personal sexual events should be policed by others (nor be included when publicly discussing attendance of LW events).

I wouldn’t use that phrasing, but I live and work from Lighthaven, and a great number of large Berkeley x-risk network parties happen here, and I chat with the organizers, so I have a lot of interaction with events and organizers. I’m definitely more in contact with semi-professional events, like parties run by MATS and AI Impacts and Lightcone, and there’s of course many purely social events that happen in this extended network that I don’t know much about. I also go to larger non-organizational parties run by friends like 2x/month (e.g. 20-100 people).

I've never heard of such a thing happening.

This signature made me laugh:

Chicxulub - Kill many birds with one stone

Also, if you haven't done so before, I recommend googling 'Chicxulub'.

Screenshot for posterity.

Yes, but none of this require overt hostility to religion (as opposed to just rejection). I think that as long as religious people accept the conversational norms and culture on LW, them bringing in some new perspectives (that are still compatible with overall LW norms) ought to be welcome. 

I think I agree with not going out of one's way to be rude, I generally think politeness is worthwhile (and have worked to become more polite myself in recent years).^[1]

I also welcome people who adhere to any religion sharing insights that they have about the world here on LessWrong.

At the same time, I am 'hostile' to religions — or at least, I am 'hostile' to any religion that claims to have infallible leaders who receive the truth directly from God(s), or that have texts about history and science and ethics that are unalterable, where adherents to the religion are not allowed to disagree with them. 

I am 'hostile' in the sense that if (prior to me working on LessWrong) a group of devout Hindus were becoming moderators of LessWrong (and were intending to follow their ethical inside views in shaping the culture of the site) I would've taken active action to prevent them having that power (e.g. publicly written arguments against this decision, moved to collect signatures against this decision, etc). I also think that if I were hypothetically freely given the opportunity to lower the hard power that religion has in some ecosystem I cared about, such as removing a Catholic priest from having control over an existential risk grant-making institution, I would be willing to go out of my way to do so, and think that this was good.

Perhaps a better term is to say that I 'oppose' religions with (IMO) inherently corrupt epistemologies, and do not want them to have power over me or the things that I care about. 

Apart from that, there are many interesting individuals who adhere to religions who have valuable insight into how the world works, and I'm grateful to them when they share such insights openly, especially here on LessWrong.

1. ^{^}

   I want to mention that I don't wish to entirely police other people's hostility. I was not raised in a religious household, but I've met many who were and who were greatly hurt due to the religious practices and culture of their family and local community and I do not begrudge them their instinctive hostility to it when it appears in their environment.

Staggering it sounds kind of nice. It could allow there to be a solstice event on the actual solstice (Dec 21st) as well as small solstice celebrations in the week leading up for those who cannot be there on that date.

I'd be excited to try that at Lighthaven (if the solstice organizers wanted to give it a shot), though I also really like having a big get-together. Perhaps we could have a week-long solstice celebration at Lighthaven with multiple rituals and other little fun things for people to do.

Both our teams (LessOnline and Manifest) are putting together big events on the consecutive weekends, and my guess is that there'll be a number of folks who want to travel for both. So for everyone around in the interim we're offering housing on-site, food, and will be helping folks run their own sub-events.

I haven't got anything to announce there yet but I have ~3 events in the early stages of being organized, all of which I personally want to attend.

Well, this utterly sucks.

I've sent like 80 emails out, still tracking them all down! 

(If anyone has contact info for The Last Psychiatrist or Hotel Concierge or Christian Rudder from the OK Cupid blog, I'd appreciate a connection.)

I’d say most people assume I want “the answer” rather than “some bits of information”.

I don’t think it applies to safety researchers at AI Labs though, I am shocked how much those folks can make.

A common experience I have is that it takes like 1-2 paragraphs of explanation for why I want this info (e.g. "Well I'm wondering if so-and-so should fly in a day earlier to travel with me but it requires going to a different airport and I'm trying to figure out whether the time it'd take to drive to me would add up to too much and also..."), but if they just gave me their ~70% confidence interval when I asked then we could cut the whole context-sharing.

Often I am annoyed when I ask someone (who I believe has more information than me) a question and they say "I don't know". I'm annoyed because I want them to give me some information. Such as:

"How long does it take to drive to the conference venue?" 

"I don't know." 

"But is it more like 10 minutes or more like 2 hours?" 

"Oh it's definitely longer than 2 hours."

But perhaps I am the one making a mistake. For instance, the question "How many countries are there?" can be answered "I'd say between 150 and 400" or it can be answered "195", and the former is called "an estimate" and the latter is called "knowing the answer". There is a folk distinction here and perhaps it is reasonable for people to want to preserve the distinction between "an estimate" and "knowing the answer".

So in the future, to get what I want, I should say "Please can you give me an estimate for how long it takes to drive to the conference venue?".

And personally I should strive, when people ask me a question to which I don't know the answer, to say "I don't know the answer, but I'd estimate between X and Y."

I think it is pretty obviously a joke :P