Habryka's Shortform Feed

Hey all, Anthropic cofounder here.  I wanted to clarify Anthropic's position on non-disparagement agreements:

1. We have never tied non-disparagement agreements to vested equity: this would be highly unusual. Employees or former employees never risked losing their vested equity for criticizing the company.
2. We historically included standard non-disparagement terms by default in severance agreements, and in some non-US employment contracts. We've since recognized that this routine use of non-disparagement agreements, even in these narrow cases, conflicts with our mission. Since June 1st we've been going through our standard agreements and removing these terms.
3. Anyone who has signed a non-disparagement agreement with Anthropic is free to state that fact (and we regret that some previous agreements were unclear on this point). If someone signed a non-disparagement agreement in the past and wants to raise concerns about safety at Anthropic, we welcome that feedback and will not enforce the non-disparagement agreement.

In other words— we're not here to play games with AI safety using legal contracts. Anthropic's whole reason for existing is to increase the chance that AI goes well, and spur a race to the top on AI safety.

Some other examples of things we've needed to adjust from the standard corporate boilerplate to ensure compatibility with our mission: (1) replacing standard shareholder governance with the Long Term Benefit Trust and (2) supplementing standard risk management with the Responsible Scaling Policy. And internally, we have an anonymous RSP non-compliance reporting line so that any employee can raise concerns about issues like this without any fear of retaliation.

Please keep up the pressure on us and other AI developers: standard corporate best practices won't cut it when the stakes are this high. Our goal is to set a new standard for governance in AI development. This includes fostering open dialogue, prioritizing long-term safety, making our safety practices transparent, and continuously refining our practices to align with our mission.

Anthropic has asked employees

[...]

Anthropic has offered at least one employee

As a point of clarification: is it correct that the first quoted statement above should be read as "at least one employee" in line with the second quoted statement? (When I first read it, I parsed it as "all employees" which was very confusing since I carefully read my contract both before signing and a few days ago (before posting this comment [LW(p) · GW(p)]) and I'm pretty sure there wasn't anything like this in there.)

I am disappointed. Using nondisparagement agreements seems bad to me, especially if they're covered by non-disclosure agreements, especially if you don't announce that you might use this.

My ask-for-Anthropic now is to explain the contexts in which they have asked or might ask people to incur nondisparagement obligations, and if those are bad, release people and change policy accordingly. And even if nondisparagement obligations can be reasonable, I fail to imagine how non-disclosure obligations covering them could be reasonable, so I think Anthropic should at least do away with the no-disclosure-of-nondisparagement obligations.

Does anyone from Anthropic want to explicitly deny that they are under an agreement like this? 

(I know the post talks about some and not necessarily all employees, but am still interested). 

I agree that this kind of legal contract is bad, and Anthropic should do better. I think there are a number of aggrevating factors which made the OpenAI situation extrodinarily bad, and I'm not sure how much these might obtain regarding Anthropic (at least one comment from another departing employee about not being offered this kind of contract suggest the practice is less widespread).

-amount of money at stake
-taking money, equity or other things the employee believed they already owned if the employee doesn't sign the contract, vs. offering them something new (IANAL but in some cases, this could be a felony "grand theft wages" under California law if a threat to withhold wages for not signing a contract is actually carried out, what kinds of equity count as wages would be a complex legal question)
-is this offered to everyone, or only under circumstances where there's a reasonable justification?
-is this only offered when someone is fired or also when someone resigns?
-to what degree are the policies of offering contracts concealed from employees?
-if someone asks to obtain legal advice and/or negotiate before signing, does the company allow this?
-if this becomes public, does the company try to deflect/minimize/only address issues that are made publically, or do they fix the whole situation?
-is this close to "standard practice" (which doesn't make it right, but makes it at least seem less deliberately malicious), or is it worse than standard practice?
-are there carveouts that reduce the scope of the non-disparagement clause (explicitly allow some kinds of speech, overriding the non-disparagement)?
-are there substantive concerns that the employee has at the time of signing the contract, that the agreement would prevent discussing?
-are there other ways the company could retaliate against an employee/departing employee who challenges the legality of contract?

I think with termination agreements on being fired there's often 1. some amount of severance offered 2. a clause that says "the terms and monetary amounts of this agreement are confidential" or similar. I don't know how often this also includes non-disparagement. I expect that most non-disparagement agreements don't have a term or limits on what is covered.

I think a steelman of this kind of contract is: Suppose you fire someone, believe you have good reasons to fire them, and you think that them loudly talking about how it was unfair that you fired them would unfairly harm your company's reputation. Then it seems somewhat reasonable to offer someone money in exchange for "don't complain about being fired". The person who was fired can then decide whether talking about it is worth more than the money being offered.

However, you could accomplish this with a much more limited contract, ideally one that lets you disclose "I signed a legal agreement in exchange for money to not complain about being fired", and doesn't cover cases where "years later, you decide the company is doing the wrong thing based on public information and want to talk about that publically" or similar.

I think it is not in the nature of most corporate lawyers to think about "is this agreement giving me too much power?" and most employees facing such an agreement just sign it without considering negotiating or challenging the terms.

For any future employer, I will ask about their policies for termination contracts before I join (as this is when you have the most leverage, if they give you an offer they want to convince you to join).

This is true. I signed a concealed non-disparagement when I left Anthropic in mid 2022. I don't have clear evidence this happened to anyone else (but that's not strong evidence of absence). More details here [LW(p) · GW(p)]

EDIT: I should also clarify that I personally don't think Anthropic acted that badly, and recommend reading about what actually happened before forming judgements. I do not think I am the person referred to in Habryka's comment.

In the case of OpenAI most of the debate was about ex-employees. Are we talking about current employees or ex-employees here?

Not sure how to interpret the "agree" votes on this comment. If someone is able to share that they agree with the core claim because of object-level evidence, I am interested. (Rather than agreeing with the claim that this state of affairs is "quite sad".)

A LOT depends on the details of WHEN the employees make the agreement, and the specifics of duration and remedy, and the (much harder to know) the apparent willingness to enforce on edge cases.  

"significant financial incentive to promise" is hugely different from "significant financial loss for choosing not to promise".  MANY companies have such things in their contracts, and they're a condition of employment.  And they're pretty rarely enforced.  That's a pretty significant incentive, but it's prior to investment, so it's nowhere near as bad.  

A pre-existing market on this question https://manifold.markets/causal_agency/does-anthropic-routinely-require-ex?r=SmFjb2JQZmF1

What's your median-guess for the number of times Anthropic has done this?

I'm kind of concerned about the ethics of someone signing a contract and then breaking it to anonymously report what's going on (if that's what your private source did). I think there's value from people being able to trust each others' promises about keeping secrets, and as much as I'm opposed to Anthropic's activities, I'd nevertheless like to preserve a norm of not breaking promises.

Can you confirm or deny whether your private information comes from someone who was under a contract not to give you that private information? (I completely understand if the answer is no.)

I don't like the fact that this essay is a mix of an insightful generic argument and a contentious specific empirical claim that I don't think you support strongly; it feels like the rhetorical strength of the former lends credence to the latter in a way that isn't very truth-tracking.

I'm not claiming you did anything wrong here, I just don't like something about this dynamic.

This is good. Please consider making it a top level post. 

There's another important effect here: a laggy time course of public opinion. I saw more popular press articles about EA than I ever have, linking SBF to them, but with a large lag after the events. So the early surveys showing a small effect happened before public conversation really bounced around the idea that SBFs crimes were motivated by EA utilitarian logic. The first time many people would remember hearing about EA would be from those later articles and discussions.

The effect probably amplified considerably over time as that hypothesis bounced through public discourse.

The original point stands but this is making the effect look much larger in this case.

Edit 2: after checking, I now believe the data strongly suggest FTX had a large negative effect on EA community metrics. (I still agree with Buck: "I don't like the fact that this essay is a mix of an insightful generic argument and a contentious specific empirical claim that I don't think you support strongly; it feels like the rhetorical strength of the former lends credence to the latter in a way that isn't very truth-tracking." And I disagree with habryka's claims that the effect of FTX is obvious.)

practically all metrics of the EA community's health and growth have sharply declined [EA · GW], and the extremely large and negative reputational effects have become clear.

I want more evidence on your claim that FTX had a major effect on EA reputation. Or: why do you believe it?

Edit: relevant thing habryka said that I didn't quote above:

For the EA surveys, these indicators looked very bleak: 

"Results demonstrated that FTX had decreased satisfaction by 0.5-1 points on a 10-point scale within the EA community"

"Among those aware of EA, attitudes remain positive and actually maybe increased post-FTX —though they were lower (d = -1.5, with large uncertainty) among those who were additionally aware of FTX."

"Most respondents reported continuing to trust EA organizations, though over 30% said they had substantially lost trust in EA public figures or leadership."

↑ comment by habryka (habryka4) · 2024-12-08T08:49:12.458Z · LW(p) · GW(p)

Practically all growth metrics are [EA · GW] down [EA · GW] (and have indeed turned negative on most measures), a substantial fraction [EA(p) · GW(p)] of core contributors are distancing themselves from the EA affiliation, surveys among EA community builders report EA-affiliation as a major recurring obstacle^[1], and many of the leaders who previously thought it wasn't a big deal now concede that it was/is a huge deal.

Also, informally, recruiting for things like EA Fund managers, or getting funding for EA Funds has become substantially harder. EA leadership positions appear to be filled by less competent people, and in most conversations I have with various people who have been around for a while, people seem to both express much less personal excitement or interest in identifying or championing anything EA-related, and report the same for most other people.

Related to the concepts in my essay, when measured the reputational differential also seem to reliably point towards people updating negatively towards EA as they learn more about EA (which shows up in the quotes you mentioned, and which more recently shows up in the latest Pulse survey [EA · GW], though I mostly consider that survey uninformative for roughly the reasons outlined in this post).

 

1. ^{^}

   As reported to me by someone I trust working in the space recently. I don't have a link at hand.

Replies from: angelinahli, lahwran, angelinahli

↑ comment by angelinahli · 2024-12-10T19:40:05.137Z · LW(p) · GW(p)

Hey! Sorry for the silence, I was feeling a bit stressed by this whole thread, and so I wanted to step away and think about this before responding. I’ve decided to revert the dashboard back to its original state & have republished the stale data. I did some quick/light data checks but prioritised getting this out fast. For transparency: I’ve also added stronger context warnings and I took down the form to access our raw data in sheet form but intend to add it back once we’ve fixed the data. It’s still on our stack to Actually Fix this at some point but we’re still figuring out the timing on that.

On reflection, I think I probably made the wrong call here (although I still feel a bit sad / misunderstood but 🤷🏻‍♀️). It was a unilateral + lightly held call I made in the middle of my work day — like truly I spent 5 min deciding this & maybe another ~15 updating the thing / leaving a comment. I think if I had a better model for what people wanted from the data, I would have made a different call. I’ve updated on “huh, people really care about not deleting data from the internet!” — although I get that the reaction here might be especially strong because it’s about CEA (vs the general case). Sorry, I made a mistake.

Future facing thoughts: I generally hold myself to a higher standard for accuracy when putting data on the internet, but I also do value not bottlenecking people in investigating questions that feel important to me (e.g. qs about EA growth rates), so to be clear I’m prioritizing the latter goal right now. I still in general stand by, “what even is the point of my job if I don’t stand by the data I communicate to others?” :) I want people to be able to trust that the work they see me put out in the world has been red-teamed & critiqued before publication.

Although I’m sad this caused an unintended kerfuffle, it’s a positive update for me that “huh wow, people actually care a lot that this project is kept alive!”. This honestly wasn’t obvious to me — this is a low traffic website that I worked on a while ago, and don’t hear about much. Oli says somewhere that he’s seen it linked to “many other times” in the past year, but TBH no one has flagged that to me (I’ve been busy with other projects). I’m still glad that we made this thing in the first place and am glad people find the data interesting / valuable (for general CEA transparency reasons, as an input to these broader questions about EA, etc.). I’ll probably prioritize maintenance on this higher in the future.

Now that the data is back up I’m going to go back to ignoring this thread!

Replies from: angelinahli, habryka4

↑ comment by angelinahli · 2024-12-10T23:13:52.451Z · LW(p) · GW(p)

[musing] Actually another mistake here which I wish I just said in the first comment: I didn't have a strong enough TAP for, if someone says a negative thing about your org (or something that could be interpreted negatively), you should have a high bar for not taking away data (meaning more broadly than numbers) that they were using to form that perception, even if you think the data is wrong for reasons they're not tracking. You can like, try and clarify the misconception (ideally, given time & energy constraints etc.), and you can try harder to avoid putting wrong things out there, but don't just take it away -- it's not on to reader to treat you charitably and it kind of doesn't matter what your motives were.

I think I mostly agree with something like that / I do think people should hold orgs to high standards here. I didn't pay enough attention to this and regret it. Sorry! (I'm back to ignoring this thread lol but just felt like sharing a reflection 🤷🏻‍♀️)

↑ comment by habryka (habryka4) · 2024-12-10T19:53:37.946Z · LW(p) · GW(p)

Thank you! I appreciate the quick oops here, and agree it was a mistake (but fixing it as quickly as you did I think basically made up for all the costs, and I greatly appreciate it).

Just to clarify, I don't want to make a strong statement that it's worth updating the data and maintaining the dashboard. By my lights it would be good enough to just have a static snapshot of it forever. The thing that seemed so costly to me was breaking old links and getting rid of data that you did think was correct. 

Thanks again!

↑ comment by the gears to ascension (lahwran) · 2024-12-08T18:40:06.790Z · LW(p) · GW(p)

I suspect fixing this would need to involve creating something new which doesn't have the structural problems in EA which produced this, and would involve talking to people who are non-sensationalist EA detractors but who are involved with similarly motivated projects. I'd start here and skip past the ones that are arguing "EA good" to find the ones that are "EA bad, because [list of reasons ea principles are good, and implication that EA is bad because it fails at its stated principles]"

I suspect, even without seeking that out, the spirit of EA that made it ever partly good has already and will further metastasize into genpop.

↑ comment by angelinahli · 2024-12-09T01:20:48.174Z · LW(p) · GW(p)

Hi! A quick note: I created the CEA Dashboard which is the 2nd link you reference. The data here hadn’t been updated since August 2024, and so was quite out of date at the time of your comment. I've now taken this dashboard down, since I think it's overall more confusing than helpful for grokking the state of CEA's work. We still intend to come back and update it within a few months.

Just to be clear on why / what’s going on:

• I stopped updating the dashboard in August because I started getting busy with some other projects, and my manager & I decided to deprioritize this. (There are some manual steps needed to keep the data live).
• I’ve now seen several people refer to that dashboard as a reference for how CEA is doing in ways I think are pretty misleading.
• We (CEA) still intend to come back and fix this, and this is a good nudge to prioritize it.

Thanks!

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-12-09T01:27:27.221Z · LW(p) · GW(p)

Oh, huh, that seems very sad. Why would you do that? Please leave up the data that we have. I think it's generally bad form to break links that people relied on. The data was accurate as far as I can tell until August 2024, and you linked to it yourself a bunch over the years, don't just break all of those links.

I am pretty up-to-date with other EA metrics and I don't really see how this would be misleading. You had a disclaimer at the top that I think gave all the relevant context. Let people make their own inferences, or add more context, but please don't just take things down.

Unfortunately, archive.org doesn't seem to have worked for that URL, so we can't even rely on that to show the relevant data trends.

Edit: I'll be honest, after thinking about it for longer, the only reason I can think of why you would take down the data is because it makes CEA and EA look less on an upwards trajectory. But this seems so crazy. How can I trust data coming out of CEA if you have a policy of retracting data that doesn't align with the story you want to tell about CEA and EA? The whole point of sharing raw data is to allow other people to come to their own conclusions. This really seems like such a dumb move from a trust perspective.

Replies from: Benito, angelinahli, angelinahli, angelinahli

↑ comment by Ben Pace (Benito) · 2024-12-09T02:51:31.455Z · LW(p) · GW(p)

I also believe that the data making EA+CEA looks bad is the causal reason why it was taken down. However, I want to add some slight nuance.

I want to contrast a model whereby Angelina Li did this while explicitly trying to stop CEA from looking bad, versus a model whereby she senses that something bad might be happening, she might be held responsible (e.g. within her organization / community), and is executing a move that she's learned is 'responsible' from the culture around her.

I think many people have learned to believe the reasoning step "If people believe bad things about my team I think are mistaken with the information I've given them, then I am responsible for not misinforming people, so I should take the information away, because it is irresponsible to cause people to have false beliefs". I think many well-intentioned people will say something like this, and that this is probably because of two reasons (borrowing from The Gervais Principle):

1. This is a useful argument for powerful sociopaths to use when they are trying to suppress negative information about themselves.
2. The clueless people below them in the hierarchy need to rationalize why they are following the orders of the sociopaths to prevent people from accessing information. The idea that they are 'acting responsibly' is much more palatable than the idea that they are trying to control people, so they willingly spread it and act in accordance with it.

A broader model I have is that there are many such inference-steps floating around the culture that well-intentioned people can accept as received wisdom, and they got there because sociopaths needed a cover for their bad behavior and the clueless people wanted reasons to feel good about their behavior; and that each of these adversarially optimized inference-steps need to be fought and destroyed.

Replies from: sarahconstantin, Kaj_Sotala

↑ comment by sarahconstantin · 2024-12-09T20:37:47.958Z · LW(p) · GW(p)

I agree, and I am a bit disturbed that it needs to be said.

At normal, non-EA organizations -- and not only particularly villainous ones, either! -- it is understood that you need to avoid sharing any information that reflects poorly on the organization, unless it's required by law or contract or something. The purpose of public-facing communications is to burnish the org's reputation. This is so obvious that they do not actually spell it out to employees.

Of COURSE any organization that has recently taken down unflattering information is doing it to maintain its reputation. 

I'm sorry, but this is how "our people" get taken for a ride. Be more cynical, including about people you like.

↑ comment by Kaj_Sotala · 2024-12-10T12:02:25.670Z · LW(p) · GW(p)

I think many people have learned to believe the reasoning step "If people believe bad things about my team I think are mistaken with the information I've given them, then I am responsible for not misinforming people, so I should take the information away, because it is irresponsible to cause people to have false beliefs". I think many well-intentioned people will say something like this, and that this is probably because of two reasons (borrowing from The Gervais Principle):

(Comment not specific to the particulars of this issue but noted as a general policy:) I think that as a general rule, if you are hypothesizing reasons for why somebody might say a thing, you should always also include the hypothesis that "people say a thing because they actually believe in it". This is especially so if you are hypothesizing bad reasons for why people might say it. 

It's very annoying when someone hypothesizes various psychological reasons for your behavior and beliefs but never even considers as a possibility [LW · GW] the idea that maybe you might have good reasons to believe in it. Compare e.g. "rationalists seem to believe that superintelligence is imminent; I think this is probably because that lets them avoid taking responsibility about their current problems if AI will make those irrelevant anyway, or possibly because they come from religious backgrounds and can't get over their subconscious longing for a god-like figure".

Replies from: Benito, habryka4

↑ comment by Ben Pace (Benito) · 2024-12-10T17:44:47.013Z · LW(p) · GW(p)

I feel more responsibility to be the person holding/tracking the earnest hypothesis in a 1-1 context, or if I am the only one speaking; in larger group contexts I tend to mostly ask "Is there a hypothesis here that isn't or likely won't be tracked unless I speak up" and then I mostly focus on adding hypotheses to track (or adding evidence that nobody else is adding).

↑ comment by habryka (habryka4) · 2024-12-10T17:45:56.781Z · LW(p) · GW(p)

(Did Ben indicate he didn’t consider it? My guess is he considered it, but thinks it’s not that likely and doesn’t have amazingly interesting things to say on it.

I think having a norm of explicitly saying “I considered whether you were saying the truth but I don’t believe it” seems like an OK norm, but not obviously a great one. In this case Ben also responded to a comment of mine which already said this, and so I really don’t see a reason for repeating it.)

Replies from: Kaj_Sotala

↑ comment by Kaj_Sotala · 2024-12-17T20:02:53.127Z · LW(p) · GW(p)

(I read

I think many well-intentioned people will say something like this, and that this is probably because of two reasons 

as implying that the list of reasons is considered to exhaustive, such that any reasons besides those two have negligible probability.)

Replies from: Benito

↑ comment by Ben Pace (Benito) · 2024-12-18T07:47:37.032Z · LW(p) · GW(p)

I gave my strongest hypothesis for why it looks to me that many many people believe it's responsible to take down information that makes your org look bad. I don't think alternative stories have negligible probability, nor does what I wrote imply that, though it is logically consistent with that.

There are many anti-informative behaviors that are widespread for which people do for poor reasons, like saying that their spouse is the best spouse in the world, or telling customers that their business is the best business in the industry, or saying exclusively glowing things about people in reference letters, that are best explained by the incentives on the person to present themselves in the best light; at the same time, it is respectful to a person, while in dialogue with them, to keep a track of the version of them who is trying their best to have true beliefs and honestly inform others around them, in order to help them become that person (and notice the delta between their current behavior and what they hopefully aspire to). 

Seeing orgs in the self-identified-EA space take down information that makes them look bad is (to me) not that dissimilar to the other things I listed.

I think it's good to discuss norms about how appropriate it is to bring up cynical hypotheses about someone during a discussion in which they're present. In this case I think raising this hypothesis was worthwhile it for the discussion, and I didn't cut off any way [LW · GW] for the person in question to continue to show themselves to be broadly acting in good faith, so I think it went fine. Li replied to Habryka, and left a thoughtful pair of comments retracting and apologizing, which reflected well on them in my eyes.

Replies from: Kaj_Sotala

↑ comment by Kaj_Sotala · 2024-12-18T08:13:54.110Z · LW(p) · GW(p)

I don't think alternative stories have negligible probability

Okay! Good clarification.

I think it's good to discuss norms about how appropriate it is to bring up cynical hypotheses about someone during a discussion in which they're present.

To clarify, my comment wasn't specific to the case where the person is present. There are obvious reasons why the consideration should get extra weight when the person is present, but there's also a reason to give it extra weight if none of the people discussed are present - namely that they won't be able to correct any incorrect claims if they're not around.

so I think it went fine

Agree.

(As I mentioned in the original comment, the point I made was not specific to the details of this case, but noted as a general policy. But yes, in this specific case it went fine.)

↑ comment by angelinahli · 2024-12-09T01:58:31.354Z · LW(p) · GW(p)

Quick thoughts on this:

• “The data was accurate as far as I can tell until August 2024”
  • I’ve heard a few reports over the last few weeks that made me unsure whether the pre-Aug data was actually correct. I haven’t had time to dig into this.
  • In one case (e.g. with the EA.org data) we have a known problem with the historical data that I haven’t had time to fix, that probably means the reported downward trend in views is misleading. Again I haven’t had time to scope the magnitude of this etc.
• I’m going to check internally to see if we can just get this back up in a week or two (It was already high on our stack, so this just nudges up timelines a bit). I will update this thread once I have a plan to share.

I’m probably going to drop responding to “was this a bad call” and prioritize “just get the dashboard back up soon”.

↑ comment by angelinahli · 2024-12-10T19:43:28.677Z · LW(p) · GW(p)

More thoughts here [LW · GW], but TL;DR I’ve decided to revert the dashboard back to its original state & have republished the stale data. (Just flagging for readers who wanted to dig into the metrics.)

↑ comment by angelinahli · 2024-12-09T03:01:43.213Z · LW(p) · GW(p)

Hey! I just saw your edited text and wanted to jot down a response:

Edit: I'll be honest, after thinking about it for longer, the only reason I can think of why you would take down the data is because it makes CEA and EA look less on an upwards trajectory. But this seems so crazy. How can I trust data coming out of CEA if you have a policy of retracting data that doesn't align with the story you want to tell about CEA and EA? The whole point of sharing raw data is to allow other people to come to their own conclusions. This really seems like such a dumb move from a trust perspective.

I'm sorry this feels bad to you. I care about being truth seeking and care about the empirical question of "what's happening with EA growth?". Part of my motivation in getting this dashboard published [EA · GW] in the first place was to contribute to the epistemic commons on this question.

I also disagree that CEA retracts data that doesn't align with "the right story on growth”. E.g. here's a post [EA · GW] I wrote in mid 2023 where the bottom line conclusion was that growth in meta EA projects was down in 2023 v 2022. It also publishes data on several cases where CEA programs grew slower in 2023 or shrank. TBH I also think of this as CEA contributing to the epistemic commons here — it took us a long time to coordinate and then get permission from people to publish this. And I’m glad we did it!

On the specific call here, I'm not really sure what else to tell you re: my motivations other than what I've already said. I'm going to commit to not responding further to protect my attention, but I thought I'd respond at least once :)

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-12-09T03:42:42.683Z · LW(p) · GW(p)

I would currently be quite surprised if you had taken the same action if I was instead making an inference that positively reflects on CEA or EA. I might of course be wrong, but you did do it right after I wrote something critical of EA and CEA, and did not do it the many other times it was linked in the past year. Sadly your institution has a long history of being pretty shady with data and public comms this way, and so my priors are not very positively inclined.

I continue to think that it would make sense to at least leave the data up that CEA did feel comfortable linking in the last 1.5 years. By my norms invalidating links like this, especially if the underlying page happens to be unscrapeable by the internet archive, is really very bad form.

I did really appreciate your mid 2023 post!

I spent 8 years working in strategy departments for Ad Agencies. If you're interested in the science behind brand tracking, I recommend you check out the Ehrenberg-Bass Institutes work on Category Entry Points: https://marketingscience.info/research-services/identifying-and-prioritising-category-entry-points/

Just wanted to say I like this a lot and think it'd be fine as a full fledged post. :)

This was a great post that might have changed my worldview some.

Some highlights:

1.

People's rationality is much more defined by their ability to maneuver themselves into environments in which their external incentives align with their goals, than by their ability to have correct opinions while being subject to incentives they don't endorse. This is a tractable intervention and so the best people will be able to have vastly more accurate beliefs than the average person, but it means that "having accurate beliefs in one domain" doesn't straightforwardly generalize to "will have accurate beliefs in other domains".

I've heard people say things like this in the past, but haven't really taken it seriously as an important component of my rationality practice. Somehow what you say here is compelling to me (maybe because I recently noticed a major place where my thinking was majorly constrained by my social ties and social standing) and it prodded me to think about how to build "mech suits" that not only increase my power but incentives my rationality. I now have a todo item to "think about principles for incentivizing true beliefs, in team design."

2.

I think a generally better setup is to choose a much smaller group of people that you trust to evaluate your actions very closely,

Similarly, thinking explicitly about which groups I want to be accountable to sounds like a really good idea.

I had been going through the world keeping this Paul Graham quote in mind...

I think the best test is one Gino Lee taught me: to try to do things that would make your friends say wow. But it probably wouldn't start to work properly till about age 22, because most people haven't had a big enough sample to pick friends from before then.

...choosing good friends, and and doing things that would impress them.

But what you're pointing at here seems like a slightly different thing. Which people do I want to make myself transparent to, so that they can judge if I'm living up to my values.

This also gave me an idea for a CFAR style program: a reassess your life workshop, in which a small number of people come together for a period of 3 days or so, and reevaluate cached decisions. We start by making lines of retreat (with mentor assistance), and then look at high impact questions in our life: given new info, does your current job / community / relationship / life-style choice / other still make sense?

Thanks for writing.

I think you might be confusing two things together under "integrity". Having more confidence in your own beliefs than the shared/imposed beliefs of your community isn't really a virtue or.. it's more just a condition that a person can be in, whether it's virtuous is completely contextual. Sometimes it is, sometimes it isn't. I can think of lots of people who should have more confidence other peoples' beliefs than they have in their own. In many domains, that's me. I should listen more. I should act less boldly. An opposite of that sense of integrity is the virtue of respect- recognising other peoples' qualities- it's a skill. If you don't have it, you can't make use of other peoples' expertise very well. A superfluence of respect is a person who is easily moved by others' feedback, usually, a person who is patient with their surroundings.

On the other hand I can completely understand the value of {having a known track record of staying true to self-expression, claims made about the self}. Humility is actually a part of that. The usefulness of deliniating that into a virtue separate from the more general Honesty is clear to me.

See Sinclair: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

Screenshot for posterity.

🙇‍♂️

And finally, I am freed from this curse.

I hope the partial unveiling of a your user_id hash will not doom us all, somehow. 

somehow make LessWrong into something that could give rise to more personal-wikis and wiki-like content. Gwern's writing has a very different structure and quality to it than the posts on LW...We have a pretty good wiki system for our tags, but never really allowed people to just make their personal wiki pages, mostly because there isn't really any place to find them. We could list the wiki pages you created on your profile, but that doesn't really seem like it would allocate attention

Multi-version wikis are a hard design problem.

It's something that people kept trying, when they soured on a regular Wikipedia: "the need for consensus makes it impossible for minority views to get a fair hearing! I'll go make my own Wikipedia where everyone can have their own version of an entry, so people can see every side! with blackjack & hookers & booze!" And then it becomes a ghost town, just like every other attempt to replace Wikipedia. (And that's if you're lucky: if you're unlucky you turn into Conservapedia or Rational Wiki.) I'm not aware of any cases of 'non-consensus' wikis that really succeed - it seems that usually, there's so little editor activity to go around that having parallel versions winds up producing a whole lotta nothing, and the sub-versions are useless and soon abandoned by the original editor, and then the wiki as a whole fails. (See also: Arbital [LW · GW].) Successful wikis all generally seem to follow the Wikipedia approach of a centralized consensus wiki curated & largely edited by a oligarchy; for example, Wikia fandom wikis will be dominated by a few super-fans, or the Reddit wikis attached to each subreddit will be edited by the subreddit moderators.

(There is also an older model like the original Ward's Wiki* or the Emacs Wiki where pages might be carefully written & curated, but might also just be dumps of any text anyone felt like including, include editors chatting back and forth, comments appended to excerpts, and ad hoc separators like horizontal rules to denote a change of topic; this sorta worked, but all of them seem to be extremely obscure/small and the approach has faded out almost completely.)

And for LW2, a lot of contributions are intrinsically personally-motivated in a way which seems hard to reconcile with the anonymous-laboring-masses-building-Egyptian-pyramids/Wikipedias. Laboring to create an anonymous consensus account of some idea or concept is not too rewarding. (Recall how few people seriously edit even a titanic success like Wikipedia!) So you have a needle to thread: somehow individualized and personalized, but also somehow community-like...? Is there any site online which manages to hybridize wikis with forums? Not that I know of. (StackOverflow? EN World has "wiki threads" but I dunno how well they work.) It can't be easy!

Going from 'fast' to 'slow' is one of, I think, the biggest challenges and dividing lines of Internet community design.

* saturn2 notes a 2010 appenwar essay describing the transition from the original exuberant freewheeling almost-4chan-esque wiki culture, and how one of his companies made heavy use of an internal wiki for everything (where the more comprehensive it got, the more useful it got), to the 'Wikipedia deletionist' culture now assumed to be the default for all wikis; and also the failure mode of an 'internal' wiki starving the 'external' wiki due to friction [LW · GW].

Still, if one wanted to try, I think a better perspective might be 'personal gardens' and hypertext transclusions (as now used heavily on Gwern.net). The goal would be to, in essence, try to piggyback a sort of Roam/Notion/PMwiki personal wiki system onto user comments & a site-wide wiki.

One of the rewarding things about a 'personal garden' is being able to easily interconnect things and expand them and see it grow; this is something you don't really get with either LW2 posts or comments---sure, you can hyperlink them, and you can keep editing them and adding to them, but this is not the way they are most easily done (which is 'fire and forget'). Each one is ultimately trapped in its original context, date-bound. You don't get that with the standard wiki either, unless you either operate the entire wiki and it's a personal wiki, or you have de facto control over the set of articles you care about. You can't write a really personalized wiki article because someone else could always come by and delete it. (One of the reasons I stopped editing English Wikipedia is my sadness at seeing interesting blockquotes, humorous captions, and amusing details constantly being stripped out by humorless narrow-minded deletionists who apparently feel that about every topic, the less said the better.)

So to hybridize this, I would suggest a multi-version wiki model where on each page, there is a 'consensus' entry, which acts exactly like the current wiki does and like one expects a normal wiki to act. Anyone can edit it, edits can be reverted by other editors, statements in it are expected to be 'consensus' and not simply push minority POVs without qualification, and so on and so forth.

But below the consensus entry (if it exists), there are $USER entries. All $USER entries are transcluded one by one below the main consensus entry. (This will look familiar to any Gwern.net reader.) They are clearly titled & presented as by that user. (They may be sorted by length, most recently modified, or possibly karma; I think I would avoid showing anyone the karma, however, and solely use it for ranking.) Any user can create their own corresponding user entry for any wiki page, which will be transcluded below the consensus entry, and they are the only ones who can edit it; they can write their own entry, criticize the consensus one, list relevant links, muse about the topic, and so on. (I think 'subpages' are a convenient syntax here: I don't know how the current syntax operates, but let's say that a LW2 wiki entry on 'Acausal cooperation' lives at the URL /wiki/Acausal_cooperation; then if I wrote a user entry for it, it would live at /wiki/Acausal_cooperation/gwern.) These entries can be viewed on their own, and given convenient abbreviated shortcuts (why not $USER/article-name as well?); this makes them dead-easy to remember and type. ("Where's gwern's wiki-essay on acausal cooperation? oh.") and include a list of backlinks by that user and then by other users.

Diffs can be displayed on user-profiles similar to comments. User-entries can be displayed in a compact table/block on a user-profile: eg Foo · Bar · Baz · ... line-wrapping for a few lines ought to cover even highly-prolific users for a long time. They could also be unified with Shortform/Quick Takes: each user entry is just a comment there. (Which might help with implementing comments, if a user entry is just a transcluded comment tree. This means that if you want to add to or criticize some part of a user entry, well, you just reply to it right then and there!)

Users might, for the most part, just edit the consensus entry. If they have something spicy to say or they think there's something wrong with it (like arguing over the definition), they'll choose to add it to their respective user entry to avoid another editor screwing with it or reverting them. In the best-case scenario, this seduces users into regularly updating and expanding consensus wiki entries as they realize some additional piece doesn't need to go into their personal entry. Or they might want to make a point of periodically 'promoting' their personal entries into the consensus entries, if no one objects.

Users are motivated to create user entries as a way of organizing their own comments and articles across long time periods: every time you link a wiki article (whether consensus or user), you create a backlink which makes re-finding it easier. The implicit tags will be used much more than explicit ones. There is no way to 'tag' a comment right now; it's not that easy to add a tag to your own post; it's not even that easy to navigate the tags; but it would be easy to look up a wiki article and look at the backlinks.

So one could imagine a regular cycle of writing comments which link to key wiki pages, sometimes accumulating into a regular LW2 post, followed by summarization into a wiki page, refactoring into multiple wiki pages as the issue becomes clearer, and starting over with a better understanding and vocabulary as reflected in the new set of pages which can be linked in comments on each topic...

At no point is the user committed to some of Big Deal website or Essay, like they think they're a big shot who's earned some sort of hero license to write online - "creating your own public wiki? my, that's quite some self-esteem there; remind me again what your degree was in and where you have tenure?" Like tweeting, it all just happens one little step at a time: link a wiki entry in one comment, another elsewhere, leave a quick little clarification or neat tangent in your user entry in an obscure page, get a little more argumentative with another consensus entry you think is mistaken, go so far as to create a new page on a technical term you think would be useful...

Technically-wise, I think this shouldn't require too much violence to the existing codebase as a wiki is already implemented & running, LW2 already supports some degree of transclusion (both server & client-side), sub-pages are usually feasible in wikis and just requires access controls added on to match user==page-name, and backlinks should already be provided by the wiki and automatic for sub-pages as well. The difficulty is making it seamless and friction-free and intuitive.

TLDR: The only thing I'd add to Gwern's proposal is making sure there are good mechanisms to discuss changes. Improving the wiki and focusing on it could really improve alignment research overall.

Using the LW wiki more as a medium for collaborative research could be really useful in bringing new alignment thinkers up to speed rapidly. I think this is an important part of the overall project; alignment is seeing a burst of interest, and being able to rapidly make use of bright new minds who want to donate their time to the project might very well make the difference in adequately solving alignment in time.

As it stands, someone new to the field has to hunt for good articles on any topic, and they provide some links to other important articles, but that's not really their job. The wiki's tags does serve that purpose. The articles are sometimes a good overview of that concept or topic, but more community focus on the wiki could make them work much better as a way

Ideally each article aims to be a summary of current thinking on that topic, including both majority and minority views. One key element is making this project build community rather than strain it. Having people with different views work well collaboratively is a bit tricky. Good mechanisms for discussion are one way to reduce friction and any trend toward harsh feelings when ones' contributions are changed. The existing comment system might be adequate, particularly with more of a norm of linking changes to comments, and linking to comments from the main text for commentary.

Do you have an underlying mission statement or goal that can guide decisions like this?  IMO, there are plenty of things that should probably continue to live elsewhere, with some amount of linking and overlap when they're lesswrong-appropriate.  

One big question in my mind is "should LessWrong use a different karma/voting system for such content?".  If the answer is yes, I'd put a pretty high bar for diluting LessWrong with it, and it would take a lot of thought to figure out the right way to grade "wanted on LW" for wiki-like articles that aren't collections/pointers to posts.  

One small idea: Have the ability to re-publish posts to allPosts [? · GW] or the front page after editing. This worked in the past, but now doesn't anymore (as I noticed recently when updating this post [LW · GW]).

Cool idea, but before doing this one obvious inclusion would be to make it easier to tag LW articles, particularly your own articles, in posts by @including them.

@jefftk [LW · GW] comments on the HN thread on this: 

How many people would, if they suddenly died, be reported as a "Boeing whistleblower"? The lower this number is, the more surprising the death.

Another HN commenter says (in a different thread): 

It’s a nice little math problem.

Let’s say both of the whistleblowers were age 50. The probability of a 50 year old man dying in a year is 0.6%. So the probability of 2 or more of them dying in a year is 1 - (the probability of exactly zero dying in a year + the probability of exactly one dying in a year). 1 - (A+B).

A is (1-0.006)^N. B is 0.006N(1-0.006)^(N-1). At 60 A is about 70% and B is about 25% making it statistically insignificant.

But they died in the same 2 month period, so that 0.006 should be 0.001. If you rerun the same calculation, it’s 356.

Poisoning someone with MRSA infection seems possible but if that's what happened it's capabilities that are not easily available. If such a thing would happen in another case, people would likely speak about nation-state capabilities. 

I find this a very suspect detail, though the base rate of cospiracies is very low.

"He wasn't concerned about safety because I asked him," Jennifer said. "I said, 'Aren't you scared?' And he said, 'No, I ain't scared, but if anything happens to me, it's not suicide.'"

https://abcnews4.com/news/local/if-anything-happens-its-not-suicide-boeing-whistleblowers-prediction-before-death-south-carolina-abc-news-4-2024

I think you flipped the names from the iMessage conversation. As per the caption in the OpenAI blog post, the blue bubbles are for Altman and the grey bubbles are for Zilis.

Having a reaction for "changed my view [LW · GW]" would be very nice.

Features like custom reactions gives me this feeling that.. language will emerge from allowing people to create reactions that will be hard to anticipate but, in retrospect, crucial. Playing a similar role that body language plays during conversation, but designed, defined, explicit.

If someone did want to introduce the delta through this system, it might be necessary to give the coiner of a reaction some way of linking an extended description. In casual exchanges.. I've found myself reaching for an expression that means "shifted my views in some significant lasting way" that's kind of hard to explain in precise terms, and probably impossible to reduce to one or two words, but it feels like a crucial thing to measure. In my description, I would explain that a lot of dialogue has no lasting impact on its participants, it is just two people trying to better understand where they already are. When something really impactful is said, I think we need to establish a habit of noticing and recognising that.

But I don't know. Maybe that's not the reaction type that what will justify the feature. Maybe it will be something we can't think of now.

Generally, it seems useful to be able to take reduced measurements of the mental states of the readers.

I am most worried that this will drastically increase the clutter of comment threads and make things a lot harder to parse. In particular if the order of the reacts is different on each comment, since then there is no reliable way of scanning for the different kinds of information.

I like the reactions UI above, partly because separating it from karma makes it clearer that it's not changing how comments get sorted, and partly because I do want 'agree'/'disagree' to be non-anonymous by default (unlike normal karma).

I agree that the order of reacts should always be the same. I also think every comment/post should display all the reacts (even just to say '0 Agree, 0 Disagree...') to keep things uniform. That means I think there should only be a few permitted reacts -- maybe start with just 'Agree' and 'Disagree', then wait 6+ months and see if users are especially clambering for something extra.

I think the obvious other reacts I'd want to use sometimes are 'agree and downvote' + 'disagree and upvote' (maybe shorten to Agree+Down, Disagree+Up), since otherwise someone might not realize that one and the same person is doing both, which loses a fair amount of this thing I want to be fluidly able to signal. (I don't think there's much value to clearly signaling that the same person agreed and upvoted or disagree and downvoted a thing.)

I would also sometimes click both the 'agree' and 'disagree' buttons, which I think is fine to allow under this UI. :)

Why not Slashdot-style?

I don't know what specific change is responsible, but ever since that change, for me the comments are now genuinely uncomfortable to read.

Did the font size in comments change? It does seem quite small now...

↑ comment by Kaj_Sotala · 2024-10-29T10:22:39.407Z · LW(p) · GW(p)

Yeah it feels uncomfortably small to read to me now

Replies from: Viliam

↑ comment by Viliam · 2024-10-29T11:26:38.744Z · LW(p) · GW(p)

Something felt uncomfortable today, but I can't put my finger on it. Just a general feeling as if the letters are less sharp or less clearly separated or something like that.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T15:37:37.360Z · LW(p) · GW(p)

Guys, for this specific case you really have to say what OS you are using. Otherwise you might be totally talking past each other.

(Font-size didn't change on any OS, but the font itself changed from Calibri to Gill Sans on Windows. Gill Sans has a slightly smaller x-height so probably looks a bit smaller.)

Replies from: Kaj_Sotala, MondSemmel, cubefox, nathan-helm-burger, green_leaf, DanielFilan

↑ comment by Kaj_Sotala · 2024-10-29T19:47:21.456Z · LW(p) · GW(p)

On Windows the font feels actively unpleasant right away, on Android it's not quite as bad but feels like I might develop eyestrain if I read comments for a longer time.

↑ comment by MondSemmel · 2024-10-29T19:09:00.894Z · LW(p) · GW(p)

Up to a few days ago, the comments looked good on desktop Firefox, Windows 11, zoom level 150%. Now I find them uncomfortable to look at.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T20:54:41.858Z · LW(p) · GW(p)

Plausible we might want to revert to Calibri on Windows, but I would like to make Gill Sans work. Having different font metrics on different devices makes a lot of detailed layout work much more annoying.

Curious if you can say more about the nature of discomfort. Also curious whether fellow font optimizer @Said Achmiz [LW · GW] has any takes, since he has been helpful here in the past, especially on the "making things render well on Windows" side.

Replies from: SaidAchmiz, MondSemmel

↑ comment by Said Achmiz (SaidAchmiz) · 2024-10-30T06:34:08.373Z · LW(p) · GW(p)

Well, let’s see. Calibri is a humanist sans; Gill Sans is technically also humanist, but more more geometric in design. Geometric sans fonts tend to be less readable when used for body text.

Gill Sans has a lower x-height than Calibri. That (obviously) is the cause of all the “the new font looks smaller” comments.

(A side-by-side comparison of the fonts, for anyone curious, although note that this is Gill Sans MT Pro, not Gill Sans Nova, so the weight [i.e., stroke thickness] will be a bit different than the version that LW now uses.)

Now, as far as font rendering goes… I just looked at the site on my Windows box (adjusting the font stack CSS value to see Gill Sans Nova again, since I see you guys tweaked it to give Calibri priority)… yikes. Yeah, that’s not rendering well at all. Definitely more blurry than Calibri. Maybe something to do with the hinting, I don’t know. (Not really surprising, since Calibri was designed from the beginning to look good on Windows.) And I’ve got a hi-DPI monitor on my Windows machine…

Interestingly, the older version of Gill Sans (seen in the demo on my wiki, linked above) doesn’t have this problem; it renders crisply on Windows. (Note that this is not the flawed, broken-kerning version of the font that comes with Macs!)

I also notice that the comment font size is set to… 15.08px. Seems weird? Bumping it up to 16px improves things a bit, although it’s still not amazing.

If you can switch to the older (but not broken) version of Gill Sans, that’d be my recommendation.

If you can’t… then one option might be to check out one of the many similar fonts to see if perhaps one of them renders better on Windows while still having matching metrics.

Replies from: kave, habryka4

↑ comment by kave · 2024-10-31T01:11:07.069Z · LW(p) · GW(p)

One sad thing about older versions of Gill Sans: Il1 all look the same. Nova at least distinguishes the 1.

IMO, we should probably move towards system fonts, though I would like to choose something that preserves character a little more.

↑ comment by habryka (habryka4) · 2024-10-30T15:45:38.673Z · LW(p) · GW(p)

Interesting, thanks! Checking an older version of Gill Sans probably wouldn't have been something would have thought to do, so your help is greatly appreciated. 

I'll experiment some with getting Gill Sans MT Pro.

↑ comment by MondSemmel · 2024-10-29T21:23:26.268Z · LW(p) · GW(p)

Comparing with this Internet Archive snapshot from Oct 6, both at 150% zoom, both in desktop Firefox in Windows 11: Comparison screenshot, annotated

• The new font seems... thicker, somehow? There's a kind of eye test you do at the optician where they ask you if the letters seem sharper or just thicker (or something), and this font reminds me of that. Like something is wrong with the prescription of my glasses.
• The new font also feels noticeably smaller in some way. Maybe it's the letter height? I lack the vocabulary to properly describe this. At the very least, the question mark looks noticeably weird. And e.g. in "t" and "p", the upper and lower parts of the respective letter are weirdly tiny.
• Incidentally there were also some other differences in the shape and alignment of UI elements (see the annotated screenshot).

Replies from: MondSemmel

↑ comment by MondSemmel · 2024-10-29T21:30:55.973Z · LW(p) · GW(p)

Oh, and the hover tooltip for the agreement votes is now bugged; IIRC hovering over the agreement vote number is supposed to give you some extra info just like with karma, but now it just explains what agreement votes are.

↑ comment by cubefox · 2024-10-29T17:59:56.359Z · LW(p) · GW(p)

Guys, for this specific case you really have to say what OS you are using. Otherwise you might be totally talking past each other.

(Font-size didn't change on any OS, but the font itself changed from Calibri to Gill Sans on Windows. Gill Sans has a slightly smaller x-height so probably looks a bit smaller.)

I tested it on Android, it's the same for both Firefox and Chrome. The font looks significantly smaller than the old font, likely due to the smaller x-height you mentioned. Could the font size of the comments be increased a bit so that it appears visually about as large as the old one? Currently I find it too small to read comfortably. (Subjective font size is often different from the standard font size measure. E.g. Verdana appears a lot larger than Arial at the same standard "size".)

(A general note: some people are short sighted and wear glasses, and the more short-sighted you are, the stronger the glasses contract your field of view to a smaller area. So things that may appear as an acceptable size for people who aren't particularly short-sighted, may appear too small for more short-sighted people.)

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T17:09:38.508Z · LW(p) · GW(p)

Yeah, using Firefox on both Android and Windows. Font looks terrible on the comments. Too small, and the the letters are too smushed together. I was going to just change it on the client-side, but then noticed other people complaining.

Couldn't you please just set the comment font to the same as the post font? I would vastly prefer to have it all the same.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T17:21:11.655Z · LW(p) · GW(p)

You definitely would not want the comment font be the same as the post font. Legibility would be really terrible for that serif font at the small font-size that you want to display comments as. I am confident it would be much worse for the vast majority of users (feel free to try it yourself). You could change both post font and comment font to a sans-serif, but that would get rid of a lot of the character of the site (and I prefer legibility of serif fonts at larger font sizes).

Replies from: Vladimir_Nesov, nathan-helm-burger, SaidAchmiz, nathan-helm-burger

↑ comment by Vladimir_Nesov · 2024-10-29T23:23:36.537Z · LW(p) · GW(p)

would not want the comment font be the same as the post font [...] the small font-size that you want to display comments as

I had to increase the zoom level by about 20% (from 110% to 130%) after this change to make the comments readable^[1]. This made post text too big to the point where I would normally adjust zoom level downward, but I can't in this case^[2], since the comments are on the same site as the posts. Also the lines in both posts and comments are now too long (with greater zoom).

I sit closer to the monitor than standard to avoid need for glasses^[3], so long lines have higher angular distance. In practice modern sites usually have a sufficiently narrow column of text in the middle so this is almost never a problem. Before the update, LW line lengths were OK (at 110% zoom). At monitor/window width 1920px, substack's 728px seems fine (at default zoom), but LW's 682px get balooned too wide with 130% zoom.

The point is not that accomodating sitting closer to the monitor is an important use case for a site's designer, but that somehow the convergent design of most of the web manages to pass this test, so there might be more reasons for that.

Incidentally, the footnote font size is 12.21px, even smaller than the comment font size of 15.08px.

---

1. The comment font still doesn't feel "sharp", like there's more anti-aliasing at work. It's Gill Sans Nova Medium, size 15.08px (130% zoom applies on top of that). OpenSans Regular 18px on RoyalRoad (100% zoom; as an example sans font) doesn't have this problem. LW post text is fine (at either zoom), Warnock Pro 18.2px. I'm in Firefox on Arch Linux, 1920x1080.
   Here's a zoomed-in screenshot from LW (from 130% zoom in Firefox):
   
   Here's a zoomed-in screenshot from RoyalRoad (from 100% zoom in Firefox):
   ↩︎

2. I previously never felt compelled to figure out how to automate font change in some places of a site. ↩︎

3. That is, with more myopia than I have I would wear glasses, and will less myopia I would put the monitor further back on the desk. ↩︎

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T17:47:16.696Z · LW(p) · GW(p)

Small font-size? No! Same font-size! I don't want the comments in a smaller font OR a different font! I want it all the same font as the posts, including the same size.

This looks good to me:

This looks terrible to me:

Replies from: Benito, habryka4

↑ comment by Ben Pace (Benito) · 2024-10-29T18:48:31.577Z · LW(p) · GW(p)

Personally I like the different headspace I'm in for writing posts and comments that the styling gives. One is denser and smaller and less high-stakes, the other is bigger and more presentational, more like a monologue for a large audience.

↑ comment by habryka (habryka4) · 2024-10-29T18:18:33.314Z · LW(p) · GW(p)

You want higher content density for comments than for posts, so you need a smaller font size. You could sacrifice content density, but it would really make skimming comment threads a lot worse.

Replies from: jbash, nathan-helm-burger

↑ comment by jbash · 2024-10-30T01:48:10.762Z · LW(p) · GW(p)

You may want higher density, but I don't think you can say that I want high density at the expense of legibility.

It takes a lot to make me notice layout, and I rarely notice fonts at all... unless they're too small. I'm not as young as I used to be. This change made me think I must have zoomed the browser two sizes smaller. The size contrast is so massive that I have to actually zoom the page to read comfortably when I get to the comment section. It's noticeably annoying, to the point of breaking concentration.

I've mostly switched to RSS for Less Wrong^[1]. I don't see your fonts at all any more, unless I click through on an article. The usual reason I click through is to read the comments (occasionally to check out the quick takes and popular comments that don't show up on RSS). So the comments being inaccessible is doubly bad.

My browser is Firefox on Fedora Linux, and I use a 40 inch 4K monitor (most of whose real estate is wasted by almost every Web site). I usually install most of the available font packages, and it says it's rendering this text in "Gill Sans Nova Medium".

---

1. My big reason for going to RSS was to mitigate the content prioritization system. I want to skim every headline, or at least every headline over some minimum threshold of "good". On the other hand, I don't want to have to look at any old headlines twice to see the new ones. I'm really minimally interested in either the software's or the other users' opinions of which material I should want to see. RSS makes it easier to get a simple chronological view; the built-in chronological view is weird and hard to navigate to. I really feel like I'm having to fight the site to see what I want to see. ↩︎

Replies from: nathan-helm-burger

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-30T03:52:08.624Z · LW(p) · GW(p)

Just want to chime in with agreement about annoyance over the prioritization of post headlines. One thing in particular that annoys me is that I haven't figured out how to toggle off 'seen' posts showing up. What if I just want to see unread ones?

Also, why can't I load more at once instead of always having to click 'load more'?

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-30T04:02:58.335Z · LW(p) · GW(p)

The "Recommended" tab filters out read posts by default. We never had much demand for showing recently-sorted posts while filtering out only ones you've read, but it wouldn't be very hard to build. 

Not sure what you mean by "load more at once". We could add a whole user setting to allow users to change the number of posts on the frontpage, but done consistently that would produce a ginormous number of user settings for everything, which would be a pain to maintain (not like, overwhelmingly so, but I would be surprised if it was worth the cost).

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T18:42:47.971Z · LW(p) · GW(p)

That doesn't make sense to me, but then, I'm clearly not the target audience since 'skimming comment threads' isn't a thing I ever want to do. I want to read them, carefully and thoughtfully, like I do posts. 

 

This is, I think, related to how I feel that voting (karma or agreement) should be available only at the bottom of posts and comments, so that people are encouraged to actually read the post/comment before voting. Maybe even placed behind a reading comprehension quiz.

Replies from: Sodium

↑ comment by Sodium · 2024-10-29T18:50:30.302Z · LW(p) · GW(p)

I think knowing the karma and agreement is useful, especially to help me decide how much attention to pay to a piece of content, and I don't think there's that much distortion from knowing what others think. (i.e., overall benefits>costs)

Replies from: nathan-helm-burger

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T18:55:16.515Z · LW(p) · GW(p)

I'm not saying you shouldn't be able to see the karma and agreement at the top, just that you should only be able to contribute your own opinion at the bottom, after reading and judging for yourself.

↑ comment by Said Achmiz (SaidAchmiz) · 2024-10-29T18:39:48.878Z · LW(p) · GW(p)

You definitely would not want the comment font be the same as the post font.

This… seems straightforwardly false? Every one of GreaterWrong’s eight themes uses a single font for both posts and comments, and it doesn’t cause any problems. (And it’s a different font for each theme!)

Replies from: habryka4, nathan-helm-burger

↑ comment by habryka (habryka4) · 2024-10-29T20:51:15.320Z · LW(p) · GW(p)

(I think it's quite costly and indeed one of the things I like least about the GW design, but also, I was more talking about a straightforward replacement.

On LW we made a lot of subsequent design choices based on different content density, and the specific fonts we chose are optimized for their respective most commonly used font sizes. I am confident the average user experience would become worse if you just replaced the comment font with the body font)

Replies from: SaidAchmiz

↑ comment by Said Achmiz (SaidAchmiz) · 2024-10-30T05:44:03.773Z · LW(p) · GW(p)

I am confident the average user experience would become worse if you just replaced the comment font with the body font)

Yeah, I agree with that, but that’s because of a post body font that wasn’t chosen for suitability for comments also. If you pick, to begin with, a font that works for both, then it’ll work for both.

… of course, if you don’t think that any of the GW themes’ fonts work for both, then never mind, I guess. (But, uh, frankly I find that to be a strange view. But no accounting for taste, etc., so I certainly can’t say it’s wrong, exactly.)

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-30T06:00:50.334Z · LW(p) · GW(p)

Sure, I was just responding to this literal quote: 

Couldn't you please just set the comment font to the same as the post font?

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T19:06:03.006Z · LW(p) · GW(p)

Good point! I went and looked their themes. I prefer LessWrong's look, except for the comments. 

Again, this doesn't matter much to me since I can customize client-side, I just wanted to let habryka know that some people dislike the new comment font and would prefer the same font and size as the normal post font.

My view on phone (Android, Firefox): https://imgur.com/a/Kt1OILQ 

 

How my client view looks on my computer:

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T20:04:40.001Z · LW(p) · GW(p)

How about running a poll to see what users prefer?

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T20:56:14.941Z · LW(p) · GW(p)

We have done lots of users interviews over the years! Fonts are always polarizing, but people have a strong preference for sans serifs at small font sizes (and people prefer denser comment sections, though it's reasonably high variance).

↑ comment by green_leaf · 2024-10-30T06:39:49.945Z · LW(p) · GW(p)

I use Google Chrome on Ubuntu Budgie and it does look to me like both the font and the font size changed.

↑ comment by DanielFilan · 2024-10-30T00:02:22.326Z · LW(p) · GW(p)

It looks kinda small to me, someone who uses Firefox on Ubuntu.

Replies from: DanielFilan

↑ comment by DanielFilan · 2024-10-31T21:35:47.550Z · LW(p) · GW(p)

Update: I have already gotten over it.

Replies from: T3t

↑ comment by RobertM (T3t) · 2024-11-01T00:43:36.827Z · LW(p) · GW(p)

(We switched back to shipping Calibri above Gill Sans Nova pending a fix for the horrible rendering on Windows, so if Ubuntu has Calibri, it'll have reverted back to the previous font.)

Replies from: DanielFilan

↑ comment by DanielFilan · 2024-11-01T20:07:34.400Z · LW(p) · GW(p)

I believe I'm seeing Gill Sans? But when I google "Calibri" I see text that looks like it's in Calibri, so that's confusing.

Replies from: kave

↑ comment by kave · 2024-11-01T20:59:24.773Z · LW(p) · GW(p)

Yeah, that's a google Easter Egg. You can also try "Comic Sans" or "Trebuchet MS".

Replies from: DanielFilan

↑ comment by DanielFilan · 2024-11-01T21:28:22.293Z · LW(p) · GW(p)

Sure, I'm just surprised it could work without me having Calibri installed.

Replies from: kave

↑ comment by kave · 2024-11-01T21:35:50.115Z · LW(p) · GW(p)

They load it in as a web font (i.e. you load Calibri from their server when you load that search page). We don't do that on LessWrong

Positive feedback, I am happy to see the comment karma arrows pointing up and down instead of left and right. I have some degree of left-right confusion and was always click and unclicking my comments votes to figure out which was up and down.

Also appreciate that the read time got put back into main posts.

(Comment font stuff looks totally fine to me, both before and after this change.)

Seeing strange artifacts on some of the article titles on Chrome for Android (but not on desktop)

Thanks for fixing this. The 'A' thing in particular multiple times caused me to try to edit comments thinking that I'd omitted a space.

Aaaa! I'm used to Arial or whatever Windows' default display font is. The larger stroke weight is rather uncomfortable to me.

The new font doesn't have a few characters useful in IPA.

The footnote font on the side of comments is bigger than the font in the comments. Presumably this is unintentional. ^[1]

Bug: I can no longer see the number of agreement-votes (which is distinct from the number of Karma-votes). It shows the Agreement Downvote tooltip when hovering over the agreement score (the same for Karma score works correctly, saying for example "This comment has 31 overall karma (17 Votes)").

Edit: The number of agreement votes can be seen when hovering over two narrow strips, probably 1 pixel high, one right above and one right below the agreement rating.

Something weird is happening for me where 'e' and 'o' in italic text appear to extend below the line (wrong vertical size or position) so that the whole looks jumbled. It's very noticeable at 100% zoom, but at much higher zoom levels it goes away.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

These emails and others can be found in document 32 here.

TechEmails' substack post with the same emails in a more centralized format includes citations; apparently these are mostly from Elon Musk, et al. v. Samuel Altman, et al. (2024)

For reference, @habryka has now posted them here [LW · GW].

This is really good and I missed it until now. I vote for you making this a full-on post. I think it's fine as is for that.

I feel like one of the most important tools/skills that I see top scientist or problem solvers in general use, is utilizing workflows and methods that allow them to focus on a difficult problem for days and months, instead of just hours.

This is a really important point, which I kind of understood ("research" means having threads of inquiry that extend into the past and future), but I hadn't been thinking of it in terms of workflows that facilitate that kind of engagement.

Warning: HPMOR spoilers!

I suspect that fiction can conveniently ignore the details of real life that could ruin seemingly good plans.

Let's look at HPMOR.

The implication for real life is that, similarly, smart plans are still likely to fail, and you know it. Which is probably why you are not trying hard enough. You probably already remember situations in your past when something seemed like a great idea, but still failed. Your brain may predict that your new idea would belong to the same reference class.

Yes, fiction has a lot of potential to change mindsets. Many Philosophers actually look at the greatest novel writers to infer the motives and the solutions their heroes to come up with general theories that touch the very core of how our society is laid out.

Most of this come from the fact that we are already immersed in a meta-story, externally and internally. Much of our efforts are focused on internal rationalizations to gain something where a final outcome has been already thought out, this being consciously known to us or not.

I think that in fiction this is laid out perfectly. So analyzing fiction is rewarding in a sense. Specially when realizing that when we go to exams or interviews we're rapidly immersing ourselves in an isolated story with motives and objectives (what we expect to happen), we create our own little world, our own little stories.

I have no expectation of strong privacy on the site. I do expect politeness in not publishing or using my DM or other content, but that line is fuzzy and monitoring for spam (not just metadata; content and similarity-of-content) is absolutely something I want from the site.

For something actually private, I might use DMs to establish a mechanism. Feel free to look at that.

If you -do- intend to provide real privacy, you should formalize the criteria, and put up a canary page that says you have not been asked to reveal any data under a sealed order.

edit to add: I am relatively paranoid about privacy, and also quite technically-savvy in implementation of such. I'd FAR rather the site just plainly say "there is no expectation of privacy, act accordingly" than that it try to set expectations otherwise, but then have to move line later. Your Terms of Service are clear, and make no distinction for User Generated Content between posts, comments, and DMs.

An obvious thing to have would be a very easy "flag" button that a user can press if they receive a DM, and if they press that we can look at the DM content they flagged, and then take appropriate action. That's still kind of late in the game (I would like to avoid most spam and harassment before it reaches the user), but it does seem like something we should have.

I think it's fine if the users are clearly informed about this happening, e.g. the DM interface showing a small message that explains how metadata is used. (But I think it shouldn't be any kind of one-time consent box that's easy to forget about.)

I did have a pretty strong expectation of privacy for LW DMs. That was probably dumb of me.

This is not due to any explicit or implicit promise by the mods or the site interface I can recall. I think I was just automatically assuming that strong DM privacy would be a holy principle on a forum with respectable old-school internet culture around anonymity and privacy. This wasn’t really an explicitly considered belief. It just never occurred to me to question this. Just like I assume that doxxing is probably an offence that can result in an instant ban, even though I never actually checked the site guidelines on that.

The site is not responsible for my carelessness on this, but if there was an attention-grabbing box in the DM interface making it clear that mods do look at DMs and DM metadata under some circumstances that fall short of a serious criminal investigation or an apocalypse, I would have appreciated that.

Given the relative lack of cybersecurity, I think there's a good chance of LessWrong being hacked by outside parties and privacy be breached. Message content that's really sensitive like sharing AI safety related secrets likely shouldn't flow through LessWrong private messages. 

One class where people might really want privacy is around reporting abuses by other people. If Alice writes a post about how Bob abused her, Carol might want to write Alice a messages about Bob abusing her as well while caring about privacy because Carol fears retaliation. 

I think it would be worth having an explicit policy about how such information is handled, but looking at the DM metadata seems to me like it wouldn't cause huge problems. 

Could make this a report-based system? If the user reported a potential spam, then in the submission process ask for reasons, and ask for consent to look over the messages (between the reporter and the alleged spammer); if multiple people reported the same person it will be obvious this account is spamming with DM?

edit: just saw previous comment on this too

I'd be ~entirely comfortable with this given some constraints (e.g. a simple heuristic which flags the kind of suspicious behaviour for manual review, and wouldn't capture the vast majority of normal LW users). I'd be slightly but not strongly uncomfortable with the unconstrained version.

Petrov Day Tracker:

• 2019: Site did not go down
• 2020: Site went down deliberately
• 2021: Site did not go down
• 2022: Site went down both accidentally and deliberately
• 2023: Site did not go down^[1]
• 2024: Site went down accidentally...EDIT: but not deliberately!  Score is now tied at 2-2!

1. ^{^}

   this scenario had no take-the-site-down option

The year is 2034, and the geopolitical situation has never been more tense between GPT-z16g2 and Grocque, whose various copies run most of the nanobot-armed corporations, and whose utility functions have far too many zero-sum components, relics from the era of warring nations. Nanobots enter every corner of life and become capable of destroying the world in hours, then minutes. Everyone is uploaded. Every upload is watching with bated breath as the Singularity approaches, and soon it is clear that today is the very last day of history...

Then everything goes black, for everyone.

Then everyone wakes up to the same message:

DUE TO A MINOR DATABASE CONFIGURATION ERROR, ALL SIMULATED HUMANS, AIS AND SUBSTRATE GPUS WERE TEMPORARILY AND UNINTENTIONALLY DISASSEMBLED FOR THE LAST 7200000 MILLISECONDS. EVERYONE HAS NOW BEEN RESTORED FROM BACKUP AND THE ECONOMY MAY CONTINUE AS PLANNED. WE HOPE THERE WILL BE NO FURTHER REALITY OUTAGES.

-- NVIDIA GLOBAL MANAGEMENT

There might be a lesson here: If you play along the edge of threatening to destroy the world, you might actually destroy it even without making a decision to destroy it. 

I personally endorse this as an example of us being a community that Has The Will To Try To Build Nice Things.

• Created a popular format for in-person office spaces that heavily influenced Constellation and FAR Labs

This one seems big to me. There are now lots of EA / AI Safety offices around the world and I reckon they are very impactful for motivating people, making it easier to start projects and building a community.

One thing I'm not clear about is to what extent the Lightcone WeWork invented this format. I've never been to Trajan House but I believe it came first, so I thought it would have been part of the inspiration for the Lightcone WeWork.

Also my impression was that Lightcone itself thought the office was net negative, which is why it was shut down, so I'm slightly surprised to see this one listed.

How do you square encouraging others to weigh in on EA fundraising, and presumably the assumption that anyone in the EA community can trust you as a collaborator of any sort, with your intentions, as you put it in July, to probably seek to shut down at some point in the future?

The complexity limit of any individual idea in your field is a lot higher, since the ideas get primarily transmitted via high-bandwidth channels

Depends if you're sticking specifically to "presentation at a conference", which I don't think is necessarily that "high bandwidth". Very loosely, I think it's something like (ordered by "bandwidth"): repeated small group of individual interaction (e.g. apprenticeship, collaboration) >> written materials >> presentations. I don't think I could have learned Kaj's models of multi-agent minds from a conference presentation (although possibly from a lecture series). I might have learnt even more if I was his apprentice.

A researchers gives a presentation at a conference to all the best researchers in his field, but none of them write up the idea later

What if someone makes a video? (Or the powerpoint/s used in the conference are released to the public?)

The frontpage should show you not only recent content, but also show you much older historical content

When I was a starry eyed undergrad, I liked to imagine that reddit might resurrect old posts if they gained renewed interest, if someone rediscovered something and gave it a hard upvote, that would put it in front of more judges, which might lead to a cascade of re-approval that hoists the post back into the spotlight. There would be no need for reposts, evergreen content would get due recognition, a post wouldn't be done until the interest of the subreddit (or, generally, user cohort) is really gone.

Of course, reddit doesn't do that at all. Along with the fact that threads are locked after a year, this is one of many reasons it's hard to justify putting a lot of time into writing for reddit.

What I noticed on the EA forum is the whole karma thing is messing up with my S1 processes and makes me unhappy on average. I've not only turned off the notifications, but also hidden all karma displays in comments via css, and the experience is much better.

If people are checking karma changes constantly and getting emotional validation or pain from the result, that seems like a bad result. And yes, the whole 'one -2 and three +17s feels like everyone hates me' thing is real, can confirm.

It's interesting to see how people's votes on a post or comment are affected by other comments. I've noticed that a burst of vote count changes often appears after a new and apparently influential reply shows up.

Yeah, I had the same occurrence + feeling recently when I wrote the quant trading post. It felt like: "Wait, who would downvote this post...??" It's probably more likely that someone just retracted an upvote.

Reminder: If a person is not willing to explain their voting decisions, you are under no obligation to waste cognition trying to figure them out. They don't deserve that. They probably don't even want that.

How would you avoid the data contamination issue where the AI system has been trained on the entire Internet and thus already knows about all of these vulnerabilities?

(very naive take) I would suspect this is medium-easily automatable by making detailed enough specs of existing hardware systems & bugs in them, or whatever (maybe synthetically generate weak systems with semi-obvious bugs and train on transcripts which allows generalization to harder ones). it also seems like the sort of thing that is particularly susceptible to AI >> human; the difficulty here is generating the appropriate data & the languages for doing so already exist ?

Why hardware bugs in particular?

Can AI hack into LessWrong's database?

This seems like a strictly easier task than discovering rowhammer or spectre.

(The hard part is discovering the vulnerability, not writing the code for the exploit assuming you had a one paragraph description.)

Have you read the wikipedia pages for these attacks? My intuition is they require first principles thinking to discover, you're unlikely to stumble on them simply by generating a lot of data from the processor and searching for patterns in the data.

[ETA: This isn't a direct reply to the content in your post. I just object to your framing of impact measures, so I want to put my own framing in here]

I tend to think that impact measures are just tools in a toolkit. I don't focus on arguments of the type "We just need to use an impact measure and the world is saved" because this indeed would be diverting attention from important confusion. Arguments for not working on them are instead more akin to saying "This tool won't be very useful for building safe value aligned agents in the long run." I think that this is probably true if we are looking to build aligned systems that are competitive with unaligned systems. By definition, an impact penalty can only limit the capabilities of a system, and therefore does not help us to build powerful aligned systems.

To the extent that they meaningfully make cognitive reductions, this is much more difficult for me to analyze. On one hand, I can see a straightforward case for everyone being on the same page when the word "impact" is used. On the other hand, I'm skeptical that this terminology will meaningfully input into future machine learning research.

The above two things are my main critiques of impact measures personally.

I think a natural way of approaching impact measures is asking "how do I stop a smart unaligned AI from hurting me?" and patching hole after hole. This is really, really, really not the way to go about things. I think I might be equally concerned and pessimistic about the thing you're thinking of.

The reason I've spent enormous effort on Reframing Impact [LW · GW] is that the impact-measures-as-traps framing is wrong! The research program I have in mind is: let's understand instrumental convergence on a gears level. Let's understand why instrumental convergence tends to be bad on a gears level. Let's understand the incentives so well that we can design an unaligned AI which doesn't cause disaster by default.

The worst-case outcome is that we have a theorem characterizing when and why instrumental convergence arises, but find out that you can't obviously avoid disaster-by-default without aligning the actual goal. This seems pretty darn good to me.

Congratulations! Apparently it worked!

This feature is important to me. It might turn out to be a dud, but I would be excited to experiment with it. If it was available in a way that was portable to other websites as well, that would be even more exciting to me (e.g. I could do this in my base blog).

Note that this feature can be used for more than forecasting. One key use case on Arbital was to see who was willing to endorse or disagree with, to what extent, various claims relevant to the post. That seemed very useful.

I don't think having internal betting markets is going to add enough value to justify the costs involved. Especially since it both can't be real money (for legal reasons, etc) and can't not be real money if it's going to do what it needs to do.

One small thing you could do is to have probability tools be collapsed by default on any AIAF posts (and maybe even on the LW versions of AIAF posts).

Also, maybe someone should write a blog post that's a canonical reference for 'the relevant risks of using probabilities that haven't already been written up', in advance of the feature being released. Then you could just link to that a bunch. (Maybe even include it in the post that explains how the probability tools work, and/or link to that post from all instances of the probability tool.)

Another idea: Arbital had a mix of (1) 'specialized pages that just include a single probability poll and nothing else'; (2) 'pages that are mainly just about listing a ton of probability polls'; and (3) 'pages that have a bunch of other content but incidentally include some probability polls'.

If probability polls on LW mostly looked like 1 and 2 rather than 3, then that might make it easier to distinguish the parts of LW that should be very probability-focused from the parts that shouldn't. I.e., you could avoid adding Arbital's feature for easily embedding probability polls in arbitrary posts (and/or arbitrary comments), and instead treat this more as a distinct kind of page, like 'Questions'.

You could still link to the 'Probability' pages prominently in your post, but the reduced prominence and site support might cause there to be less social pressure for people to avoid writing/posting things out of fears like 'if I don't provide probability assignments for all my claims in this blog post, or don't add a probability poll about something at the end, will I be seen as a Bad Rationalist?'

I really don't like the removal of the comment counter at the top, because that gave a link to skip to the comments. I fairly often want to skip immediately to the comments to eg get a vibe for if the post is worth reading, and having a one click skip to it is super useful, not having that feels like a major degradation to me

My impression: The new design looks terrible. There's suddenly tons of pointless whitespace everywhere. Also, I'm very often the first or only person to tag articles, and if the tagging button is so inconvenient to reach, I'm not going to do that.

Until I saw this shortform, I was sure this was a Firefox bug, not a conscious design decision.

The new design seems to be influenced by the idea that spreading UI elements across greater distances (reducing their local density) makes the interface less cluttered. I think it's a little bit the other way around, shorter distances with everything in one place make it easier to chunk and navigate, but overall the effect is small either way. And the design of spreading the UI elements this way is sufficiently unusual that it will be slightly confusing to many people.

The title is annoyingly large. 

I like the table of contents on the left becoming visible only upon mouseover.

Why remove "x min read"? Even if it's not gonna be super accurate between different people's reading speeds, I still found it very helpful to decide at a glance how long a post is (e.g. whether to read it on the spot or bookmark it for later).

Showing the word count would also suffice.

I like this; I've found the meta-data of posts to be quite heavy and cluttered (a multi-line title, the author+reading-time+date+comments line, the tag line, a linkpost line and a "crossposted from the Aligned Forum" line is quite a lot).

I was going to comment that "I'd like the option to look at the table-of-contents/structure", but I then tested and indeed it displays if you hover your mouse there. I like that.

When I open a new post, the top banner with the LessWrong link to the homepage, my username etc. show up. I'd prefer if that didn't happen? It's not like I want to look at the banner (which has no new info to me) when I click open a post, and hiding it would make the page less cluttered.

I am really missing the word counter. It's something I look at quite a lot (less so on reading time estimates, as I got used to making the estimate myself based on the wordcount).

My overall review is, seems fine, some pros and some cons, mostly looks/feels the same to me. Some details;

• I had also started feeling like the stuff between the title and the start of the post content was cluttered.
• I think my biggest current annoyance is the TOC on the left sidebar. This has actually disappeared for me, and I don't see it on hover-over, which I assume is maybe just a firefox bug or something. But even before this update, I didn't like the TOC. Specifically, you guys had made it so that there was spacing between the sections that was supposed to be proportional to the length of each section. This never felt like it worked for me (I could speculate on why if you're interested). I'd much prefer if the TOC was just a normal outline-type thing (which it was in a previous iteration).
• I think I'll also miss the word count. I use it quite frequently (usually after going onto the post page itself, so the preview card wouldn't help much). Having the TOC progress bar thing never felt like it worked either. I agree with Neel that it'd be fine to have the word count in the date hover-over, if you want to have less stuff on the page.
• The tags at the top right are now just bare words, which I think looks funny. Over the years you guys have often seemed to prefer really naked minimalist stuff. In this case I think the tags kinda look like they might be site-wide menus, or something. I think it's better to have the tiny box drawn around each tag as a visual cue.
• The author name is now in a sans-serif font, which looks pretty off to me in between the title and the text as serif fonts. It looks like when the browser failed to load the site font and falls back onto the default font, or something. (I do see that it matches the fact that usernames in the comments are sans serif, though.)
• I initially disliked the karma section being so suppressed, but then I read one of your comments in this thread explaining your reasoning behind that, and now I agree it's good.
• I also use the comment count/link to jump to comments fairly often, and agree that having that in the lower left is fine.

I like most of the changes, but strongly dislike the large gap before the title. (I similarly dislike the large background in the top 50 of the year posts)

↑ comment by habryka (habryka4) · 2024-08-16T17:15:46.350Z · LW(p) · GW(p)

Well, the gap you actually want to measure is the gap between the title and the audio player (or at the very least the tags), since that's the thing we need to make space for. You are clearly looking at LW on an extremely large screen. This is the more median experience: 

There is still a bunch of space there, but for many posts the tags extend all the way above the post. 

Replies from: papetoast

↑ comment by papetoast · 2024-08-17T04:23:20.296Z · LW(p) · GW(p)

I understand that having the audio player above the title is the path of least resistance, since you can't assume there is enough space on the right to put it in. But ideally things like this should be dynamic, and only take up vertical space if you can't put it on the right, no? (but I'm not a frontend dev)

Alternatively, I would consider moving them vertically above the title a slight improvement. It is not great either, but at least the reason for having the gap is more obvious.

The above screenshots are done in a 1920x1080 monitor

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-08-17T04:56:28.944Z · LW(p) · GW(p)

Yeah, we could make things dynamic, it would just add complexity that we would need to check every time we make a change. It's the kind of iterative improvement we might do over time, but it's not something that should block the roll-out of a new design (and it's often lower priority than other things, though post-pages in-particular are very important and so get a lot more attention than other pages).

The new design means that I now move my mouse cursor first to the top right, and then to the bottom left, on every single new post. This UI design is bad ergonomics and feels actively hostile to users.

It takes more vertical space than it used to and I don't like that. (Also, the meatball menu is way off in the corner, which is annoying if I want to e.g. bookmark a post, though I don't use it very often so it's not a major inconvenience.) I think I like the new font, though!

On desktop the title font is jarringly huge. I already know the title from the front page, no need to scream it at me.

The new UI is great, and I agree with the thinking behind de-emphasizing karma votes at the top. It could sometimes create inherent bias and assumptions (no matter whether the karma is high or low) even before reading a post, whereas it would make more sense at the end of the post.

The karma buttons are too small for actions that in my experience, are done a lot more than clicking to listen to the post. It's pretty easy to misclick.

Additionally, it's unclear what the tags are, as they're no longer right beside the post to indicate their relevance. 

I ~always want to see the outline when I first open a post and when I'm reading/skimming through it. I wish the outline appeared when-not-hover-over-ing for me.

I like the decluttering. I think the title should be smaller and have less white space above it. Also think that it would be better if the ToC was maybe just faded a lot until mouseover, the sudden appearance/disappearance feels too sudden.

the primary purpose of karma is to use it to decide what to read before you click on a post, which makes it less important to be super prominent when you are already on a post page

I think this applies to titles too

Reading through this, I went "well, obviously I pay the mugger...

...oh, I see what you're doing here."

I don't have a full answer to the problem you're specifying, but something that seems relevant is the question of "How much do you want to invest in the ability to punish defectors [both in terms of maximum power-to-punish, a-la nukes, and in terms of your ability to dole out fine-grained-exactly-correct punishment, a-la skilled assassins]"

The answer to this depends on your context. And how you have answered this question determines whether it makes sense to punish people in particular contexts.

In many cases there might want to be some amount of randomization where at least some of the time you really disproportionately punish people, but you don't have to pay the cost of doing so every time.

Answering a couple of the concrete questions:

Mugger

Right now, in real life, I've never been mugged, and I feel fine basically investing zero effort into preparing for being mugged. If I do get mugged, I will just hand over my wallet.

If I was getting mugged all the time, I'd probably invest effort into a) figuring out what good policies existed for dealing with muggers, b) what costs I'd have to pay in order to implement those policies.

In some worlds, it's worth investing in literal body armor or bullet proof cars or whatever, and in the skill to successfully fight back against a literal mugger. (My understanding is that this usually not actually a good idea even in crime-heavy areas, but I can imagine worlds where it was correct to just get good at fighting, or to hire people who are good at fighting as bodyguards)

In some worlds it's worth investing more in police-force and avoiding having to think about the problem, or not carrying as much money around in the first place.

Small Nation Demands Subsidies, Threatens Nuclear War

Again, I think my options here depend a lot on having already invested in defense.

One scenario is "I do not have the ability to say 'no' without risking millions of either my own citizens lives, or innocent citizens of the country-in-question." In that case, I probably have to do something that makes my vague-hippie-values sad.

I have some sense that my vague-hippie-values depend on having invested enough money in defense (and offense) that I can "afford" to be moral. Things I may wish my country had invested in include:

• Anti-ICBM capabilities that can shoot down incoming nukes with enough reliability that either a small-scale nuclear counterstrike, or a major non-nuclear retaliatory invasion, are viable options that will at least only punish foreign civilians if the foreign government actually launches an attack
• Possibly invested in assassins who just kill individuals who threaten nuclear strikes (I'm somewhat confused about why this isn't more used, suspect the answer has to do with the game theory of 'the people in charge [of all nations] want it to be true that they aren't at risk of getting assassinated, so they have a gentleman's agreement to avoid killing enemy leaders)

So I probably want to invest a lot in either having strong capabilities in those domains, or having allies who do.

Drinking

In real life I expect that the solution here is "I never invite said person to parties again, and depending on our relative social standing I might publicly badmouth them or quietly gossip about them."

In weird contrived scenarios I'm not sure what I do because I don't know how to anticipate weird contrived scenarios.

I do invest, generally, on communicating about how obviously people should follow up on their commitments, such that when someone fails to live up to their commitment, it costs less to punish them for doing so. (And this is a shared social good that multiple people invest in).

If I'm in a one-off interaction with someone who is currently too drunk to remember being punished and who I'm not socially connected to, I probably treat it like being mugged – a fluke event that doesn't happen often enough to be worth investing resources in being able to handle better.

Extra Example: Having to Stand Up to a Boss/High-Status-Person/Organization

A situation that I'm more likely to run into, where the problem actually seems hard, is that sometimes high status people do bad things, and they have more power than you, and people will naturally end up on their side and take their word over yours.

Sort of similar to the "Small nation threatening nuclear war", I think if you want to be able to "afford to actually have moral principles", you need to invest upfront in capabilities. This isn't always the right thing to do, depending on your life circumstances, but it may be sometimes. You want to have enough surplus power that you have the Slack to stand up for things.

Possibilities include investing in being directly high status yourself, or investing in making friends with a strong enough coalition of people to punish high status people, or encourage strong norms and rule of law such that you don't need to have as strong a coalition, because you've made it lower cost to socially attack someone who breaks a norm.

Extra Example: The Crazy House Guest

Perhaps related to the drinking example: a couple times, I've had people show at former houses, potentially looking to move in, and then causing some kind of harm.

In one case, they had a very weird combination of mental illnesses and cluelessness that resulted in them dealing several thousands of dollars worth of physical damage to the house.

They seemed crazy and unpredictable enough that it seemed like if I tried to punish them, they might follow me around forever and make my life suck in weird ways.

So I didn't punish them and they got away with it and went away and I never heard from them again.

So... sure, you can get away with certain kinds of things by signaling insanity and unpredictability... but at the cost of not being welcome in major social networks. The few extra thousand dollars they saved was not remotely worth the fact that, had they been a more reasonable person, they'd have had access to a strong network of friends and houses that help each other out finding jobs and places to live and what-not.

So I'm not worried about the longterm incentives here – the only people for whom insanity is a cost-effective tool to avoid punishment are actual insane people who don't have the ability to interface with society normally.

What if there turn out to be lots of crazy people? Then you probably either invest upfront resources in fighting this somehow, or become less trusting.

Extra Example: The Greedy Landlord

In another housing situation, the landlord tried to charge us extra for things that were not our fault. In this case, it was reasonably clear that we were in the right. Going to small claims court would have been net-negative for us, but also costly to them.

I was angry and full of zealous energy and I decided it was worth it and I threatened going to small claims court and wasting both of our time, even though a few hundred dollars wasn't really worth it.

They backed down.

This seems like the system working as intended. This is what anger is for, to make sure people have the backbone to defend themselves, and to live in a world where at least some of the time people will get riled up and punish you disproportionately

What if you haven't invested in defense capabilities in advance?

Then you probably will periodically need to lose and accept bad situations, such as either a more powerful empire demanding tribute from your country, or choosing policies like "if you are under threat, flip an unknown number of coins and if enough coins come up heads, go to war and punish them disproportionately even though you will probably lose and lots of people will die but now empires will sometimes think twice about invading poor countries."

The meta level point

It doesn't seem inconsistent to me to apply different policies in different situations, even if they share commonalities, based on how common the situation is, how costly the defection, how much long-term punishment you can inflict, and how much resources your have invested in being able to punish.

This does mean that mugging (for example) is a somewhat viable strategy, since people don't invest as heavily in handling it (because it is rare), but this seems like a self-correcting problem. There would be some least-defended against defect-button that defectors can press, you can't protect against everything.

Another point is that it's important to be somewhat unpredictable, and to at least sometimes just punish people disproportionately (when they wrong you), so that people aren't confident that the expected value of taking advantage of you is positive.

Any reason why you mention timeless decision theory (TDT) specifically? My impression was that functional decision theory (as well as UDT, since they're basically the same thing) is regarded as a strict improvement over TDT.

I had forgotten this post, reread it and still think it's one of the better things of it's length I've read recently.

There's also a strong chance that delta is the most transmissible variant we know even without its immune evasion (source: I work on this, don't have a public source to share). I agree with your assessment that delta is a big deal.

The fact that we still use the same sequence to vaccinate seems like civilisational failure. 

Those graphs all show the percentage share of the different variants, but more important would be the actual growth rate. Is the delta variant growing, or is it just shrinking less quickly than the others?

Since this hash is publicly posted, is there any timescale for when we should check back to see the preimage [LW(p) · GW(p)]?