Habryka's Shortform Feed

Hey all, Anthropic cofounder here.  I wanted to clarify Anthropic's position on non-disparagement agreements:

1. We have never tied non-disparagement agreements to vested equity: this would be highly unusual. Employees or former employees never risked losing their vested equity for criticizing the company.
2. We historically included standard non-disparagement terms by default in severance agreements, and in some non-US employment contracts. We've since recognized that this routine use of non-disparagement agreements, even in these narrow cases, conflicts with our mission. Since June 1st we've been going through our standard agreements and removing these terms.
3. Anyone who has signed a non-disparagement agreement with Anthropic is free to state that fact (and we regret that some previous agreements were unclear on this point). If someone signed a non-disparagement agreement in the past and wants to raise concerns about safety at Anthropic, we welcome that feedback and will not enforce the non-disparagement agreement.

In other words— we're not here to play games with AI safety using legal contracts. Anthropic's whole reason for existing is to increase the chance that AI goes well, and spur a race to the top on AI safety.

Some other examples of things we've needed to adjust from the standard corporate boilerplate to ensure compatibility with our mission: (1) replacing standard shareholder governance with the Long Term Benefit Trust and (2) supplementing standard risk management with the Responsible Scaling Policy. And internally, we have an anonymous RSP non-compliance reporting line so that any employee can raise concerns about issues like this without any fear of retaliation.

Please keep up the pressure on us and other AI developers: standard corporate best practices won't cut it when the stakes are this high. Our goal is to set a new standard for governance in AI development. This includes fostering open dialogue, prioritizing long-term safety, making our safety practices transparent, and continuously refining our practices to align with our mission.

Anthropic has asked employees

[...]

Anthropic has offered at least one employee

As a point of clarification: is it correct that the first quoted statement above should be read as "at least one employee" in line with the second quoted statement? (When I first read it, I parsed it as "all employees" which was very confusing since I carefully read my contract both before signing and a few days ago (before posting this comment [LW(p) · GW(p)]) and I'm pretty sure there wasn't anything like this in there.)

I am disappointed. Using nondisparagement agreements seems bad to me, especially if they're covered by non-disclosure agreements, especially if you don't announce that you might use this.

My ask-for-Anthropic now is to explain the contexts in which they have asked or might ask people to incur nondisparagement obligations, and if those are bad, release people and change policy accordingly. And even if nondisparagement obligations can be reasonable, I fail to imagine how non-disclosure obligations covering them could be reasonable, so I think Anthropic should at least do away with the no-disclosure-of-nondisparagement obligations.

Does anyone from Anthropic want to explicitly deny that they are under an agreement like this? 

(I know the post talks about some and not necessarily all employees, but am still interested). 

I agree that this kind of legal contract is bad, and Anthropic should do better. I think there are a number of aggrevating factors which made the OpenAI situation extrodinarily bad, and I'm not sure how much these might obtain regarding Anthropic (at least one comment from another departing employee about not being offered this kind of contract suggest the practice is less widespread).

-amount of money at stake
-taking money, equity or other things the employee believed they already owned if the employee doesn't sign the contract, vs. offering them something new (IANAL but in some cases, this could be a felony "grand theft wages" under California law if a threat to withhold wages for not signing a contract is actually carried out, what kinds of equity count as wages would be a complex legal question)
-is this offered to everyone, or only under circumstances where there's a reasonable justification?
-is this only offered when someone is fired or also when someone resigns?
-to what degree are the policies of offering contracts concealed from employees?
-if someone asks to obtain legal advice and/or negotiate before signing, does the company allow this?
-if this becomes public, does the company try to deflect/minimize/only address issues that are made publically, or do they fix the whole situation?
-is this close to "standard practice" (which doesn't make it right, but makes it at least seem less deliberately malicious), or is it worse than standard practice?
-are there carveouts that reduce the scope of the non-disparagement clause (explicitly allow some kinds of speech, overriding the non-disparagement)?
-are there substantive concerns that the employee has at the time of signing the contract, that the agreement would prevent discussing?
-are there other ways the company could retaliate against an employee/departing employee who challenges the legality of contract?

I think with termination agreements on being fired there's often 1. some amount of severance offered 2. a clause that says "the terms and monetary amounts of this agreement are confidential" or similar. I don't know how often this also includes non-disparagement. I expect that most non-disparagement agreements don't have a term or limits on what is covered.

I think a steelman of this kind of contract is: Suppose you fire someone, believe you have good reasons to fire them, and you think that them loudly talking about how it was unfair that you fired them would unfairly harm your company's reputation. Then it seems somewhat reasonable to offer someone money in exchange for "don't complain about being fired". The person who was fired can then decide whether talking about it is worth more than the money being offered.

However, you could accomplish this with a much more limited contract, ideally one that lets you disclose "I signed a legal agreement in exchange for money to not complain about being fired", and doesn't cover cases where "years later, you decide the company is doing the wrong thing based on public information and want to talk about that publically" or similar.

I think it is not in the nature of most corporate lawyers to think about "is this agreement giving me too much power?" and most employees facing such an agreement just sign it without considering negotiating or challenging the terms.

For any future employer, I will ask about their policies for termination contracts before I join (as this is when you have the most leverage, if they give you an offer they want to convince you to join).

This is true. I signed a concealed non-disparagement when I left Anthropic in mid 2022. I don't have clear evidence this happened to anyone else (but that's not strong evidence of absence). More details here [LW(p) · GW(p)]

EDIT: I should also clarify that I personally don't think Anthropic acted that badly, and recommend reading about what actually happened before forming judgements. I do not think I am the person referred to in Habryka's comment.

In the case of OpenAI most of the debate was about ex-employees. Are we talking about current employees or ex-employees here?

Not sure how to interpret the "agree" votes on this comment. If someone is able to share that they agree with the core claim because of object-level evidence, I am interested. (Rather than agreeing with the claim that this state of affairs is "quite sad".)

A LOT depends on the details of WHEN the employees make the agreement, and the specifics of duration and remedy, and the (much harder to know) the apparent willingness to enforce on edge cases.  

"significant financial incentive to promise" is hugely different from "significant financial loss for choosing not to promise".  MANY companies have such things in their contracts, and they're a condition of employment.  And they're pretty rarely enforced.  That's a pretty significant incentive, but it's prior to investment, so it's nowhere near as bad.  

A pre-existing market on this question https://manifold.markets/causal_agency/does-anthropic-routinely-require-ex?r=SmFjb2JQZmF1

What's your median-guess for the number of times Anthropic has done this?

I'm kind of concerned about the ethics of someone signing a contract and then breaking it to anonymously report what's going on (if that's what your private source did). I think there's value from people being able to trust each others' promises about keeping secrets, and as much as I'm opposed to Anthropic's activities, I'd nevertheless like to preserve a norm of not breaking promises.

Can you confirm or deny whether your private information comes from someone who was under a contract not to give you that private information? (I completely understand if the answer is no.)

Just wanted to say I like this a lot and think it'd be fine as a full fledged post. :)

This was a great post that might have changed my worldview some.

Some highlights:

1.

People's rationality is much more defined by their ability to maneuver themselves into environments in which their external incentives align with their goals, than by their ability to have correct opinions while being subject to incentives they don't endorse. This is a tractable intervention and so the best people will be able to have vastly more accurate beliefs than the average person, but it means that "having accurate beliefs in one domain" doesn't straightforwardly generalize to "will have accurate beliefs in other domains".

I've heard people say things like this in the past, but haven't really taken it seriously as an important component of my rationality practice. Somehow what you say here is compelling to me (maybe because I recently noticed a major place where my thinking was majorly constrained by my social ties and social standing) and it prodded me to think about how to build "mech suits" that not only increase my power but incentives my rationality. I now have a todo item to "think about principles for incentivizing true beliefs, in team design."

2.

I think a generally better setup is to choose a much smaller group of people that you trust to evaluate your actions very closely,

Similarly, thinking explicitly about which groups I want to be accountable to sounds like a really good idea.

I had been going through the world keeping this Paul Graham quote in mind...

I think the best test is one Gino Lee taught me: to try to do things that would make your friends say wow. But it probably wouldn't start to work properly till about age 22, because most people haven't had a big enough sample to pick friends from before then.

...choosing good friends, and and doing things that would impress them.

But what you're pointing at here seems like a slightly different thing. Which people do I want to make myself transparent to, so that they can judge if I'm living up to my values.

This also gave me an idea for a CFAR style program: a reassess your life workshop, in which a small number of people come together for a period of 3 days or so, and reevaluate cached decisions. We start by making lines of retreat (with mentor assistance), and then look at high impact questions in our life: given new info, does your current job / community / relationship / life-style choice / other still make sense?

Thanks for writing.

I think you might be confusing two things together under "integrity". Having more confidence in your own beliefs than the shared/imposed beliefs of your community isn't really a virtue or.. it's more just a condition that a person can be in, whether it's virtuous is completely contextual. Sometimes it is, sometimes it isn't. I can think of lots of people who should have more confidence other peoples' beliefs than they have in their own. In many domains, that's me. I should listen more. I should act less boldly. An opposite of that sense of integrity is the virtue of respect- recognising other peoples' qualities- it's a skill. If you don't have it, you can't make use of other peoples' expertise very well. A superfluence of respect is a person who is easily moved by others' feedback, usually, a person who is patient with their surroundings.

On the other hand I can completely understand the value of {having a known track record of staying true to self-expression, claims made about the self}. Humility is actually a part of that. The usefulness of deliniating that into a virtue separate from the more general Honesty is clear to me.

See Sinclair: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

Screenshot for posterity.

🙇‍♂️

And finally, I am freed from this curse.

I hope the partial unveiling of a your user_id hash will not doom us all, somehow. 

somehow make LessWrong into something that could give rise to more personal-wikis and wiki-like content. Gwern's writing has a very different structure and quality to it than the posts on LW...We have a pretty good wiki system for our tags, but never really allowed people to just make their personal wiki pages, mostly because there isn't really any place to find them. We could list the wiki pages you created on your profile, but that doesn't really seem like it would allocate attention

Multi-version wikis are a hard design problem.

It's something that people kept trying, when they soured on a regular Wikipedia: "the need for consensus makes it impossible for minority views to get a fair hearing! I'll go make my own Wikipedia where everyone can have their own version of an entry, so people can see every side! with blackjack & hookers & booze!" And then it becomes a ghost town, just like every other attempt to replace Wikipedia. (And that's if you're lucky: if you're unlucky you turn into Conservapedia or Rational Wiki.) I'm not aware of any cases of 'non-consensus' wikis that really succeed - it seems that usually, there's so little editor activity to go around that having parallel versions winds up producing a whole lotta nothing, and the sub-versions are useless and soon abandoned by the original editor, and then the wiki as a whole fails. (See also: Arbital [LW · GW].) Successful wikis all generally seem to follow the Wikipedia approach of a centralized consensus wiki curated & largely edited by a oligarchy; for example, Wikia fandom wikis will be dominated by a few super-fans, or the Reddit wikis attached to each subreddit will be edited by the subreddit moderators.

(There is also an older model like the original Ward's Wiki* or the Emacs Wiki where pages might be carefully written & curated, but might also just be dumps of any text anyone felt like including, include editors chatting back and forth, comments appended to excerpts, and ad hoc separators like horizontal rules to denote a change of topic; this sorta worked, but all of them seem to be extremely obscure/small and the approach has faded out almost completely.)

And for LW2, a lot of contributions are intrinsically personally-motivated in a way which seems hard to reconcile with the anonymous-laboring-masses-building-Egyptian-pyramids/Wikipedias. Laboring to create an anonymous consensus account of some idea or concept is not too rewarding. (Recall how few people seriously edit even a titanic success like Wikipedia!) So you have a needle to thread: somehow individualized and personalized, but also somehow community-like...? Is there any site online which manages to hybridize wikis with forums? Not that I know of. (StackOverflow? EN World has "wiki threads" but I dunno how well they work.) It can't be easy!

Going from 'fast' to 'slow' is one of, I think, the biggest challenges and dividing lines of Internet community design.

* saturn2 notes a 2010 appenwar essay describing the transition from the original exuberant freewheeling almost-4chan-esque wiki culture, and how one of his companies made heavy use of an internal wiki for everything (where the more comprehensive it got, the more useful it got), to the 'Wikipedia deletionist' culture now assumed to be the default for all wikis; and also the failure mode of an 'internal' wiki starving the 'external' wiki due to friction [LW · GW].

Still, if one wanted to try, I think a better perspective might be 'personal gardens' and hypertext transclusions (as now used heavily on Gwern.net). The goal would be to, in essence, try to piggyback a sort of Roam/Notion/PMwiki personal wiki system onto user comments & a site-wide wiki.

One of the rewarding things about a 'personal garden' is being able to easily interconnect things and expand them and see it grow; this is something you don't really get with either LW2 posts or comments---sure, you can hyperlink them, and you can keep editing them and adding to them, but this is not the way they are most easily done (which is 'fire and forget'). Each one is ultimately trapped in its original context, date-bound. You don't get that with the standard wiki either, unless you either operate the entire wiki and it's a personal wiki, or you have de facto control over the set of articles you care about. You can't write a really personalized wiki article because someone else could always come by and delete it. (One of the reasons I stopped editing English Wikipedia is my sadness at seeing interesting blockquotes, humorous captions, and amusing details constantly being stripped out by humorless narrow-minded deletionists who apparently feel that about every topic, the less said the better.)

So to hybridize this, I would suggest a multi-version wiki model where on each page, there is a 'consensus' entry, which acts exactly like the current wiki does and like one expects a normal wiki to act. Anyone can edit it, edits can be reverted by other editors, statements in it are expected to be 'consensus' and not simply push minority POVs without qualification, and so on and so forth.

But below the consensus entry (if it exists), there are $USER entries. All $USER entries are transcluded one by one below the main consensus entry. (This will look familiar to any Gwern.net reader.) They are clearly titled & presented as by that user. (They may be sorted by length, most recently modified, or possibly karma; I think I would avoid showing anyone the karma, however, and solely use it for ranking.) Any user can create their own corresponding user entry for any wiki page, which will be transcluded below the consensus entry, and they are the only ones who can edit it; they can write their own entry, criticize the consensus one, list relevant links, muse about the topic, and so on. (I think 'subpages' are a convenient syntax here: I don't know how the current syntax operates, but let's say that a LW2 wiki entry on 'Acausal cooperation' lives at the URL /wiki/Acausal_cooperation; then if I wrote a user entry for it, it would live at /wiki/Acausal_cooperation/gwern.) These entries can be viewed on their own, and given convenient abbreviated shortcuts (why not $USER/article-name as well?); this makes them dead-easy to remember and type. ("Where's gwern's wiki-essay on acausal cooperation? oh.") and include a list of backlinks by that user and then by other users.

Diffs can be displayed on user-profiles similar to comments. User-entries can be displayed in a compact table/block on a user-profile: eg Foo · Bar · Baz · ... line-wrapping for a few lines ought to cover even highly-prolific users for a long time. They could also be unified with Shortform/Quick Takes: each user entry is just a comment there. (Which might help with implementing comments, if a user entry is just a transcluded comment tree. This means that if you want to add to or criticize some part of a user entry, well, you just reply to it right then and there!)

Users might, for the most part, just edit the consensus entry. If they have something spicy to say or they think there's something wrong with it (like arguing over the definition), they'll choose to add it to their respective user entry to avoid another editor screwing with it or reverting them. In the best-case scenario, this seduces users into regularly updating and expanding consensus wiki entries as they realize some additional piece doesn't need to go into their personal entry. Or they might want to make a point of periodically 'promoting' their personal entries into the consensus entries, if no one objects.

Users are motivated to create user entries as a way of organizing their own comments and articles across long time periods: every time you link a wiki article (whether consensus or user), you create a backlink which makes re-finding it easier. The implicit tags will be used much more than explicit ones. There is no way to 'tag' a comment right now; it's not that easy to add a tag to your own post; it's not even that easy to navigate the tags; but it would be easy to look up a wiki article and look at the backlinks.

So one could imagine a regular cycle of writing comments which link to key wiki pages, sometimes accumulating into a regular LW2 post, followed by summarization into a wiki page, refactoring into multiple wiki pages as the issue becomes clearer, and starting over with a better understanding and vocabulary as reflected in the new set of pages which can be linked in comments on each topic...

At no point is the user committed to some of Big Deal website or Essay, like they think they're a big shot who's earned some sort of hero license to write online - "creating your own public wiki? my, that's quite some self-esteem there; remind me again what your degree was in and where you have tenure?" Like tweeting, it all just happens one little step at a time: link a wiki entry in one comment, another elsewhere, leave a quick little clarification or neat tangent in your user entry in an obscure page, get a little more argumentative with another consensus entry you think is mistaken, go so far as to create a new page on a technical term you think would be useful...

Technically-wise, I think this shouldn't require too much violence to the existing codebase as a wiki is already implemented & running, LW2 already supports some degree of transclusion (both server & client-side), sub-pages are usually feasible in wikis and just requires access controls added on to match user==page-name, and backlinks should already be provided by the wiki and automatic for sub-pages as well. The difficulty is making it seamless and friction-free and intuitive.

TLDR: The only thing I'd add to Gwern's proposal is making sure there are good mechanisms to discuss changes. Improving the wiki and focusing on it could really improve alignment research overall.

Using the LW wiki more as a medium for collaborative research could be really useful in bringing new alignment thinkers up to speed rapidly. I think this is an important part of the overall project; alignment is seeing a burst of interest, and being able to rapidly make use of bright new minds who want to donate their time to the project might very well make the difference in adequately solving alignment in time.

As it stands, someone new to the field has to hunt for good articles on any topic, and they provide some links to other important articles, but that's not really their job. The wiki's tags does serve that purpose. The articles are sometimes a good overview of that concept or topic, but more community focus on the wiki could make them work much better as a way

Ideally each article aims to be a summary of current thinking on that topic, including both majority and minority views. One key element is making this project build community rather than strain it. Having people with different views work well collaboratively is a bit tricky. Good mechanisms for discussion are one way to reduce friction and any trend toward harsh feelings when ones' contributions are changed. The existing comment system might be adequate, particularly with more of a norm of linking changes to comments, and linking to comments from the main text for commentary.

Do you have an underlying mission statement or goal that can guide decisions like this?  IMO, there are plenty of things that should probably continue to live elsewhere, with some amount of linking and overlap when they're lesswrong-appropriate.  

One big question in my mind is "should LessWrong use a different karma/voting system for such content?".  If the answer is yes, I'd put a pretty high bar for diluting LessWrong with it, and it would take a lot of thought to figure out the right way to grade "wanted on LW" for wiki-like articles that aren't collections/pointers to posts.  

One small idea: Have the ability to re-publish posts to allPosts [? · GW] or the front page after editing. This worked in the past, but now doesn't anymore (as I noticed recently when updating this post [LW · GW]).

Cool idea, but before doing this one obvious inclusion would be to make it easier to tag LW articles, particularly your own articles, in posts by @including them.

@jefftk [LW · GW] comments on the HN thread on this: 

How many people would, if they suddenly died, be reported as a "Boeing whistleblower"? The lower this number is, the more surprising the death.

Another HN commenter says (in a different thread): 

It’s a nice little math problem.

Let’s say both of the whistleblowers were age 50. The probability of a 50 year old man dying in a year is 0.6%. So the probability of 2 or more of them dying in a year is 1 - (the probability of exactly zero dying in a year + the probability of exactly one dying in a year). 1 - (A+B).

A is (1-0.006)^N. B is 0.006N(1-0.006)^(N-1). At 60 A is about 70% and B is about 25% making it statistically insignificant.

But they died in the same 2 month period, so that 0.006 should be 0.001. If you rerun the same calculation, it’s 356.

Poisoning someone with MRSA infection seems possible but if that's what happened it's capabilities that are not easily available. If such a thing would happen in another case, people would likely speak about nation-state capabilities. 

I find this a very suspect detail, though the base rate of cospiracies is very low.

"He wasn't concerned about safety because I asked him," Jennifer said. "I said, 'Aren't you scared?' And he said, 'No, I ain't scared, but if anything happens to me, it's not suicide.'"

https://abcnews4.com/news/local/if-anything-happens-its-not-suicide-boeing-whistleblowers-prediction-before-death-south-carolina-abc-news-4-2024

Having a reaction for "changed my view [LW · GW]" would be very nice.

Features like custom reactions gives me this feeling that.. language will emerge from allowing people to create reactions that will be hard to anticipate but, in retrospect, crucial. Playing a similar role that body language plays during conversation, but designed, defined, explicit.

If someone did want to introduce the delta through this system, it might be necessary to give the coiner of a reaction some way of linking an extended description. In casual exchanges.. I've found myself reaching for an expression that means "shifted my views in some significant lasting way" that's kind of hard to explain in precise terms, and probably impossible to reduce to one or two words, but it feels like a crucial thing to measure. In my description, I would explain that a lot of dialogue has no lasting impact on its participants, it is just two people trying to better understand where they already are. When something really impactful is said, I think we need to establish a habit of noticing and recognising that.

But I don't know. Maybe that's not the reaction type that what will justify the feature. Maybe it will be something we can't think of now.

Generally, it seems useful to be able to take reduced measurements of the mental states of the readers.

I am most worried that this will drastically increase the clutter of comment threads and make things a lot harder to parse. In particular if the order of the reacts is different on each comment, since then there is no reliable way of scanning for the different kinds of information.

I like the reactions UI above, partly because separating it from karma makes it clearer that it's not changing how comments get sorted, and partly because I do want 'agree'/'disagree' to be non-anonymous by default (unlike normal karma).

I agree that the order of reacts should always be the same. I also think every comment/post should display all the reacts (even just to say '0 Agree, 0 Disagree...') to keep things uniform. That means I think there should only be a few permitted reacts -- maybe start with just 'Agree' and 'Disagree', then wait 6+ months and see if users are especially clambering for something extra.

I think the obvious other reacts I'd want to use sometimes are 'agree and downvote' + 'disagree and upvote' (maybe shorten to Agree+Down, Disagree+Up), since otherwise someone might not realize that one and the same person is doing both, which loses a fair amount of this thing I want to be fluidly able to signal. (I don't think there's much value to clearly signaling that the same person agreed and upvoted or disagree and downvoted a thing.)

I would also sometimes click both the 'agree' and 'disagree' buttons, which I think is fine to allow under this UI. :)

Why not Slashdot-style?

I don't know what specific change is responsible, but ever since that change, for me the comments are now genuinely uncomfortable to read.

Did the font size in comments change? It does seem quite small now...

↑ comment by Kaj_Sotala · 2024-10-29T10:22:39.407Z · LW(p) · GW(p)

Yeah it feels uncomfortably small to read to me now

Replies from: Viliam

↑ comment by Viliam · 2024-10-29T11:26:38.744Z · LW(p) · GW(p)

Something felt uncomfortable today, but I can't put my finger on it. Just a general feeling as if the letters are less sharp or less clearly separated or something like that.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T15:37:37.360Z · LW(p) · GW(p)

Guys, for this specific case you really have to say what OS you are using. Otherwise you might be totally talking past each other.

(Font-size didn't change on any OS, but the font itself changed from Calibri to Gill Sans on Windows. Gill Sans has a slightly smaller x-height so probably looks a bit smaller.)

Replies from: Kaj_Sotala, MondSemmel, cubefox, nathan-helm-burger, green_leaf, DanielFilan

↑ comment by Kaj_Sotala · 2024-10-29T19:47:21.456Z · LW(p) · GW(p)

On Windows the font feels actively unpleasant right away, on Android it's not quite as bad but feels like I might develop eyestrain if I read comments for a longer time.

↑ comment by MondSemmel · 2024-10-29T19:09:00.894Z · LW(p) · GW(p)

Up to a few days ago, the comments looked good on desktop Firefox, Windows 11, zoom level 150%. Now I find them uncomfortable to look at.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T20:54:41.858Z · LW(p) · GW(p)

Plausible we might want to revert to Calibri on Windows, but I would like to make Gill Sans work. Having different font metrics on different devices makes a lot of detailed layout work much more annoying.

Curious if you can say more about the nature of discomfort. Also curious whether fellow font optimizer @Said Achmiz [LW · GW] has any takes, since he has been helpful here in the past, especially on the "making things render well on Windows" side.

Replies from: SaidAchmiz, MondSemmel

↑ comment by Said Achmiz (SaidAchmiz) · 2024-10-30T06:34:08.373Z · LW(p) · GW(p)

Well, let’s see. Calibri is a humanist sans; Gill Sans is technically also humanist, but more more geometric in design. Geometric sans fonts tend to be less readable when used for body text.

Gill Sans has a lower x-height than Calibri. That (obviously) is the cause of all the “the new font looks smaller” comments.

(A side-by-side comparison of the fonts, for anyone curious, although note that this is Gill Sans MT Pro, not Gill Sans Nova, so the weight [i.e., stroke thickness] will be a bit different than the version that LW now uses.)

Now, as far as font rendering goes… I just looked at the site on my Windows box (adjusting the font stack CSS value to see Gill Sans Nova again, since I see you guys tweaked it to give Calibri priority)… yikes. Yeah, that’s not rendering well at all. Definitely more blurry than Calibri. Maybe something to do with the hinting, I don’t know. (Not really surprising, since Calibri was designed from the beginning to look good on Windows.) And I’ve got a hi-DPI monitor on my Windows machine…

Interestingly, the older version of Gill Sans (seen in the demo on my wiki, linked above) doesn’t have this problem; it renders crisply on Windows. (Note that this is not the flawed, broken-kerning version of the font that comes with Macs!)

I also notice that the comment font size is set to… 15.08px. Seems weird? Bumping it up to 16px improves things a bit, although it’s still not amazing.

If you can switch to the older (but not broken) version of Gill Sans, that’d be my recommendation.

If you can’t… then one option might be to check out one of the many similar fonts to see if perhaps one of them renders better on Windows while still having matching metrics.

Replies from: kave, habryka4

↑ comment by kave · 2024-10-31T01:11:07.069Z · LW(p) · GW(p)

One sad thing about older versions of Gill Sans: Il1 all look the same. Nova at least distinguishes the 1.

IMO, we should probably move towards system fonts, though I would like to choose something that preserves character a little more.

↑ comment by habryka (habryka4) · 2024-10-30T15:45:38.673Z · LW(p) · GW(p)

Interesting, thanks! Checking an older version of Gill Sans probably wouldn't have been something would have thought to do, so your help is greatly appreciated. 

I'll experiment some with getting Gill Sans MT Pro.

↑ comment by MondSemmel · 2024-10-29T21:23:26.268Z · LW(p) · GW(p)

Comparing with this Internet Archive snapshot from Oct 6, both at 150% zoom, both in desktop Firefox in Windows 11: Comparison screenshot, annotated

• The new font seems... thicker, somehow? There's a kind of eye test you do at the optician where they ask you if the letters seem sharper or just thicker (or something), and this font reminds me of that. Like something is wrong with the prescription of my glasses.
• The new font also feels noticeably smaller in some way. Maybe it's the letter height? I lack the vocabulary to properly describe this. At the very least, the question mark looks noticeably weird. And e.g. in "t" and "p", the upper and lower parts of the respective letter are weirdly tiny.
• Incidentally there were also some other differences in the shape and alignment of UI elements (see the annotated screenshot).

Replies from: MondSemmel

↑ comment by MondSemmel · 2024-10-29T21:30:55.973Z · LW(p) · GW(p)

Oh, and the hover tooltip for the agreement votes is now bugged; IIRC hovering over the agreement vote number is supposed to give you some extra info just like with karma, but now it just explains what agreement votes are.

↑ comment by cubefox · 2024-10-29T17:59:56.359Z · LW(p) · GW(p)

Guys, for this specific case you really have to say what OS you are using. Otherwise you might be totally talking past each other.

(Font-size didn't change on any OS, but the font itself changed from Calibri to Gill Sans on Windows. Gill Sans has a slightly smaller x-height so probably looks a bit smaller.)

I tested it on Android, it's the same for both Firefox and Chrome. The font looks significantly smaller than the old font, likely due to the smaller x-height you mentioned. Could the font size of the comments be increased a bit so that it appears visually about as large as the old one? Currently I find it too small to read comfortably. (Subjective font size is often different from the standard font size measure. E.g. Verdana appears a lot larger than Arial at the same standard "size".)

(A general note: some people are short sighted and wear glasses, and the more short-sighted you are, the stronger the glasses contract your field of view to a smaller area. So things that may appear as an acceptable size for people who aren't particularly short-sighted, may appear too small for more short-sighted people.)

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T17:09:38.508Z · LW(p) · GW(p)

Yeah, using Firefox on both Android and Windows. Font looks terrible on the comments. Too small, and the the letters are too smushed together. I was going to just change it on the client-side, but then noticed other people complaining.

Couldn't you please just set the comment font to the same as the post font? I would vastly prefer to have it all the same.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T17:21:11.655Z · LW(p) · GW(p)

You definitely would not want the comment font be the same as the post font. Legibility would be really terrible for that serif font at the small font-size that you want to display comments as. I am confident it would be much worse for the vast majority of users (feel free to try it yourself). You could change both post font and comment font to a sans-serif, but that would get rid of a lot of the character of the site (and I prefer legibility of serif fonts at larger font sizes).

Replies from: Vladimir_Nesov, nathan-helm-burger, SaidAchmiz, nathan-helm-burger

↑ comment by Vladimir_Nesov · 2024-10-29T23:23:36.537Z · LW(p) · GW(p)

would not want the comment font be the same as the post font [...] the small font-size that you want to display comments as

I had to increase the zoom level by about 20% (from 110% to 130%) after this change to make the comments readable^[1]. This made post text too big to the point where I would normally adjust zoom level downward, but I can't in this case^[2], since the comments are on the same site as the posts. Also the lines in both posts and comments are now too long (with greater zoom).

I sit closer to the monitor than standard to avoid need for glasses^[3], so long lines have higher angular distance. In practice modern sites usually have a sufficiently narrow column of text in the middle so this is almost never a problem. Before the update, LW line lengths were OK (at 110% zoom). At monitor/window width 1920px, substack's 728px seems fine (at default zoom), but LW's 682px get balooned too wide with 130% zoom.

The point is not that accomodating sitting closer to the monitor is an important use case for a site's designer, but that somehow the convergent design of most of the web manages to pass this test, so there might be more reasons for that.

Incidentally, the footnote font size is 12.21px, even smaller than the comment font size of 15.08px.

---

1. The comment font still doesn't feel "sharp", like there's more anti-aliasing at work. It's Gill Sans Nova Medium, size 15.08px (130% zoom applies on top of that). OpenSans Regular 18px on RoyalRoad (100% zoom; as an example sans font) doesn't have this problem. LW post text is fine (at either zoom), Warnock Pro 18.2px. I'm in Firefox on Arch Linux, 1920x1080.
   Here's a zoomed-in screenshot from LW (from 130% zoom in Firefox):
   
   Here's a zoomed-in screenshot from RoyalRoad (from 100% zoom in Firefox):
   ↩︎

2. I previously never felt compelled to figure out how to automate font change in some places of a site. ↩︎

3. That is, with more myopia than I have I would wear glasses, and will less myopia I would put the monitor further back on the desk. ↩︎

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T17:47:16.696Z · LW(p) · GW(p)

Small font-size? No! Same font-size! I don't want the comments in a smaller font OR a different font! I want it all the same font as the posts, including the same size.

This looks good to me:

This looks terrible to me:

Replies from: Benito, habryka4

↑ comment by Ben Pace (Benito) · 2024-10-29T18:48:31.577Z · LW(p) · GW(p)

Personally I like the different headspace I'm in for writing posts and comments that the styling gives. One is denser and smaller and less high-stakes, the other is bigger and more presentational, more like a monologue for a large audience.

↑ comment by habryka (habryka4) · 2024-10-29T18:18:33.314Z · LW(p) · GW(p)

You want higher content density for comments than for posts, so you need a smaller font size. You could sacrifice content density, but it would really make skimming comment threads a lot worse.

Replies from: jbash, nathan-helm-burger

↑ comment by jbash · 2024-10-30T01:48:10.762Z · LW(p) · GW(p)

You may want higher density, but I don't think you can say that I want high density at the expense of legibility.

It takes a lot to make me notice layout, and I rarely notice fonts at all... unless they're too small. I'm not as young as I used to be. This change made me think I must have zoomed the browser two sizes smaller. The size contrast is so massive that I have to actually zoom the page to read comfortably when I get to the comment section. It's noticeably annoying, to the point of breaking concentration.

I've mostly switched to RSS for Less Wrong^[1]. I don't see your fonts at all any more, unless I click through on an article. The usual reason I click through is to read the comments (occasionally to check out the quick takes and popular comments that don't show up on RSS). So the comments being inaccessible is doubly bad.

My browser is Firefox on Fedora Linux, and I use a 40 inch 4K monitor (most of whose real estate is wasted by almost every Web site). I usually install most of the available font packages, and it says it's rendering this text in "Gill Sans Nova Medium".

---

1. My big reason for going to RSS was to mitigate the content prioritization system. I want to skim every headline, or at least every headline over some minimum threshold of "good". On the other hand, I don't want to have to look at any old headlines twice to see the new ones. I'm really minimally interested in either the software's or the other users' opinions of which material I should want to see. RSS makes it easier to get a simple chronological view; the built-in chronological view is weird and hard to navigate to. I really feel like I'm having to fight the site to see what I want to see. ↩︎

Replies from: nathan-helm-burger

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-30T03:52:08.624Z · LW(p) · GW(p)

Just want to chime in with agreement about annoyance over the prioritization of post headlines. One thing in particular that annoys me is that I haven't figured out how to toggle off 'seen' posts showing up. What if I just want to see unread ones?

Also, why can't I load more at once instead of always having to click 'load more'?

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-30T04:02:58.335Z · LW(p) · GW(p)

The "Recommended" tab filters out read posts by default. We never had much demand for showing recently-sorted posts while filtering out only ones you've read, but it wouldn't be very hard to build. 

Not sure what you mean by "load more at once". We could add a whole user setting to allow users to change the number of posts on the frontpage, but done consistently that would produce a ginormous number of user settings for everything, which would be a pain to maintain (not like, overwhelmingly so, but I would be surprised if it was worth the cost).

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T18:42:47.971Z · LW(p) · GW(p)

That doesn't make sense to me, but then, I'm clearly not the target audience since 'skimming comment threads' isn't a thing I ever want to do. I want to read them, carefully and thoughtfully, like I do posts. 

 

This is, I think, related to how I feel that voting (karma or agreement) should be available only at the bottom of posts and comments, so that people are encouraged to actually read the post/comment before voting. Maybe even placed behind a reading comprehension quiz.

Replies from: Sodium

↑ comment by Sodium · 2024-10-29T18:50:30.302Z · LW(p) · GW(p)

I think knowing the karma and agreement is useful, especially to help me decide how much attention to pay to a piece of content, and I don't think there's that much distortion from knowing what others think. (i.e., overall benefits>costs)

Replies from: nathan-helm-burger

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T18:55:16.515Z · LW(p) · GW(p)

I'm not saying you shouldn't be able to see the karma and agreement at the top, just that you should only be able to contribute your own opinion at the bottom, after reading and judging for yourself.

↑ comment by Said Achmiz (SaidAchmiz) · 2024-10-29T18:39:48.878Z · LW(p) · GW(p)

You definitely would not want the comment font be the same as the post font.

This… seems straightforwardly false? Every one of GreaterWrong’s eight themes uses a single font for both posts and comments, and it doesn’t cause any problems. (And it’s a different font for each theme!)

Replies from: habryka4, nathan-helm-burger

↑ comment by habryka (habryka4) · 2024-10-29T20:51:15.320Z · LW(p) · GW(p)

(I think it's quite costly and indeed one of the things I like least about the GW design, but also, I was more talking about a straightforward replacement.

On LW we made a lot of subsequent design choices based on different content density, and the specific fonts we chose are optimized for their respective most commonly used font sizes. I am confident the average user experience would become worse if you just replaced the comment font with the body font)

Replies from: SaidAchmiz

↑ comment by Said Achmiz (SaidAchmiz) · 2024-10-30T05:44:03.773Z · LW(p) · GW(p)

I am confident the average user experience would become worse if you just replaced the comment font with the body font)

Yeah, I agree with that, but that’s because of a post body font that wasn’t chosen for suitability for comments also. If you pick, to begin with, a font that works for both, then it’ll work for both.

… of course, if you don’t think that any of the GW themes’ fonts work for both, then never mind, I guess. (But, uh, frankly I find that to be a strange view. But no accounting for taste, etc., so I certainly can’t say it’s wrong, exactly.)

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-30T06:00:50.334Z · LW(p) · GW(p)

Sure, I was just responding to this literal quote: 

Couldn't you please just set the comment font to the same as the post font?

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T19:06:03.006Z · LW(p) · GW(p)

Good point! I went and looked their themes. I prefer LessWrong's look, except for the comments. 

Again, this doesn't matter much to me since I can customize client-side, I just wanted to let habryka know that some people dislike the new comment font and would prefer the same font and size as the normal post font.

My view on phone (Android, Firefox): https://imgur.com/a/Kt1OILQ 

 

How my client view looks on my computer:

↑ comment by Nathan Helm-Burger (nathan-helm-burger) · 2024-10-29T20:04:40.001Z · LW(p) · GW(p)

How about running a poll to see what users prefer?

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-10-29T20:56:14.941Z · LW(p) · GW(p)

We have done lots of users interviews over the years! Fonts are always polarizing, but people have a strong preference for sans serifs at small font sizes (and people prefer denser comment sections, though it's reasonably high variance).

↑ comment by green_leaf · 2024-10-30T06:39:49.945Z · LW(p) · GW(p)

I use Google Chrome on Ubuntu Budgie and it does look to me like both the font and the font size changed.

↑ comment by DanielFilan · 2024-10-30T00:02:22.326Z · LW(p) · GW(p)

It looks kinda small to me, someone who uses Firefox on Ubuntu.

Replies from: DanielFilan

↑ comment by DanielFilan · 2024-10-31T21:35:47.550Z · LW(p) · GW(p)

Update: I have already gotten over it.

Replies from: T3t

↑ comment by RobertM (T3t) · 2024-11-01T00:43:36.827Z · LW(p) · GW(p)

(We switched back to shipping Calibri above Gill Sans Nova pending a fix for the horrible rendering on Windows, so if Ubuntu has Calibri, it'll have reverted back to the previous font.)

Replies from: DanielFilan

↑ comment by DanielFilan · 2024-11-01T20:07:34.400Z · LW(p) · GW(p)

I believe I'm seeing Gill Sans? But when I google "Calibri" I see text that looks like it's in Calibri, so that's confusing.

Replies from: kave

↑ comment by kave · 2024-11-01T20:59:24.773Z · LW(p) · GW(p)

Yeah, that's a google Easter Egg. You can also try "Comic Sans" or "Trebuchet MS".

Replies from: DanielFilan

↑ comment by DanielFilan · 2024-11-01T21:28:22.293Z · LW(p) · GW(p)

Sure, I'm just surprised it could work without me having Calibri installed.

Replies from: kave

↑ comment by kave · 2024-11-01T21:35:50.115Z · LW(p) · GW(p)

They load it in as a web font (i.e. you load Calibri from their server when you load that search page). We don't do that on LessWrong

Positive feedback, I am happy to see the comment karma arrows pointing up and down instead of left and right. I have some degree of left-right confusion and was always click and unclicking my comments votes to figure out which was up and down.

Also appreciate that the read time got put back into main posts.

(Comment font stuff looks totally fine to me, both before and after this change.)

Seeing strange artifacts on some of the article titles on Chrome for Android (but not on desktop)

Thanks for fixing this. The 'A' thing in particular multiple times caused me to try to edit comments thinking that I'd omitted a space.

Aaaa! I'm used to Arial or whatever Windows' default display font is. The larger stroke weight is rather uncomfortable to me.

The new font doesn't have a few characters useful in IPA.

The footnote font on the side of comments is bigger than the font in the comments. Presumably this is unintentional. ^[1]

Bug: I can no longer see the number of agreement-votes (which is distinct from the number of Karma-votes). It shows the Agreement Downvote tooltip when hovering over the agreement score (the same for Karma score works correctly, saying for example "This comment has 31 overall karma (17 Votes)").

Edit: The number of agreement votes can be seen when hovering over two narrow strips, probably 1 pixel high, one right above and one right below the agreement rating.

Something weird is happening for me where 'e' and 'o' in italic text appear to extend below the line (wrong vertical size or position) so that the whole looks jumbled. It's very noticeable at 100% zoom, but at much higher zoom levels it goes away.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

These emails and others can be found in document 32 here.

TechEmails' substack post with the same emails in a more centralized format includes citations; apparently these are mostly from Elon Musk, et al. v. Samuel Altman, et al. (2024)

For reference, @habryka has now posted them here [LW · GW].

This is really good and I missed it until now. I vote for you making this a full-on post. I think it's fine as is for that.

I feel like one of the most important tools/skills that I see top scientist or problem solvers in general use, is utilizing workflows and methods that allow them to focus on a difficult problem for days and months, instead of just hours.

This is a really important point, which I kind of understood ("research" means having threads of inquiry that extend into the past and future), but I hadn't been thinking of it in terms of workflows that facilitate that kind of engagement.

Warning: HPMOR spoilers!

I suspect that fiction can conveniently ignore the details of real life that could ruin seemingly good plans.

Let's look at HPMOR.

The implication for real life is that, similarly, smart plans are still likely to fail, and you know it. Which is probably why you are not trying hard enough. You probably already remember situations in your past when something seemed like a great idea, but still failed. Your brain may predict that your new idea would belong to the same reference class.

Yes, fiction has a lot of potential to change mindsets. Many Philosophers actually look at the greatest novel writers to infer the motives and the solutions their heroes to come up with general theories that touch the very core of how our society is laid out.

Most of this come from the fact that we are already immersed in a meta-story, externally and internally. Much of our efforts are focused on internal rationalizations to gain something where a final outcome has been already thought out, this being consciously known to us or not.

I think that in fiction this is laid out perfectly. So analyzing fiction is rewarding in a sense. Specially when realizing that when we go to exams or interviews we're rapidly immersing ourselves in an isolated story with motives and objectives (what we expect to happen), we create our own little world, our own little stories.

Petrov Day Tracker:

• 2019: Site did not go down
• 2020: Site went down deliberately
• 2021: Site did not go down
• 2022: Site went down both accidentally and deliberately
• 2023: Site did not go down^[1]
• 2024: Site went down accidentally...EDIT: but not deliberately!  Score is now tied at 2-2!

1. ^{^}

   this scenario had no take-the-site-down option

The year is 2034, and the geopolitical situation has never been more tense between GPT-z16g2 and Grocque, whose various copies run most of the nanobot-armed corporations, and whose utility functions have far too many zero-sum components, relics from the era of warring nations. Nanobots enter every corner of life and become capable of destroying the world in hours, then minutes. Everyone is uploaded. Every upload is watching with bated breath as the Singularity approaches, and soon it is clear that today is the very last day of history...

Then everything goes black, for everyone.

Then everyone wakes up to the same message:

DUE TO A MINOR DATABASE CONFIGURATION ERROR, ALL SIMULATED HUMANS, AIS AND SUBSTRATE GPUS WERE TEMPORARILY AND UNINTENTIONALLY DISASSEMBLED FOR THE LAST 7200000 MILLISECONDS. EVERYONE HAS NOW BEEN RESTORED FROM BACKUP AND THE ECONOMY MAY CONTINUE AS PLANNED. WE HOPE THERE WILL BE NO FURTHER REALITY OUTAGES.

-- NVIDIA GLOBAL MANAGEMENT

There might be a lesson here: If you play along the edge of threatening to destroy the world, you might actually destroy it even without making a decision to destroy it. 

I personally endorse this as an example of us being a community that Has The Will To Try To Build Nice Things.

• Created a popular format for in-person office spaces that heavily influenced Constellation and FAR Labs

This one seems big to me. There are now lots of EA / AI Safety offices around the world and I reckon they are very impactful for motivating people, making it easier to start projects and building a community.

One thing I'm not clear about is to what extent the Lightcone WeWork invented this format. I've never been to Trajan House but I believe it came first, so I thought it would have been part of the inspiration for the Lightcone WeWork.

Also my impression was that Lightcone itself thought the office was net negative, which is why it was shut down, so I'm slightly surprised to see this one listed.

How do you square encouraging others to weigh in on EA fundraising, and presumably the assumption that anyone in the EA community can trust you as a collaborator of any sort, with your intentions, as you put it in July, to probably seek to shut down at some point in the future?

The complexity limit of any individual idea in your field is a lot higher, since the ideas get primarily transmitted via high-bandwidth channels

Depends if you're sticking specifically to "presentation at a conference", which I don't think is necessarily that "high bandwidth". Very loosely, I think it's something like (ordered by "bandwidth"): repeated small group of individual interaction (e.g. apprenticeship, collaboration) >> written materials >> presentations. I don't think I could have learned Kaj's models of multi-agent minds from a conference presentation (although possibly from a lecture series). I might have learnt even more if I was his apprentice.

A researchers gives a presentation at a conference to all the best researchers in his field, but none of them write up the idea later

What if someone makes a video? (Or the powerpoint/s used in the conference are released to the public?)

The frontpage should show you not only recent content, but also show you much older historical content

When I was a starry eyed undergrad, I liked to imagine that reddit might resurrect old posts if they gained renewed interest, if someone rediscovered something and gave it a hard upvote, that would put it in front of more judges, which might lead to a cascade of re-approval that hoists the post back into the spotlight. There would be no need for reposts, evergreen content would get due recognition, a post wouldn't be done until the interest of the subreddit (or, generally, user cohort) is really gone.

Of course, reddit doesn't do that at all. Along with the fact that threads are locked after a year, this is one of many reasons it's hard to justify putting a lot of time into writing for reddit.

What I noticed on the EA forum is the whole karma thing is messing up with my S1 processes and makes me unhappy on average. I've not only turned off the notifications, but also hidden all karma displays in comments via css, and the experience is much better.

If people are checking karma changes constantly and getting emotional validation or pain from the result, that seems like a bad result. And yes, the whole 'one -2 and three +17s feels like everyone hates me' thing is real, can confirm.

It's interesting to see how people's votes on a post or comment are affected by other comments. I've noticed that a burst of vote count changes often appears after a new and apparently influential reply shows up.

Yeah, I had the same occurrence + feeling recently when I wrote the quant trading post. It felt like: "Wait, who would downvote this post...??" It's probably more likely that someone just retracted an upvote.

Reminder: If a person is not willing to explain their voting decisions, you are under no obligation to waste cognition trying to figure them out. They don't deserve that. They probably don't even want that.

[ETA: This isn't a direct reply to the content in your post. I just object to your framing of impact measures, so I want to put my own framing in here]

I tend to think that impact measures are just tools in a toolkit. I don't focus on arguments of the type "We just need to use an impact measure and the world is saved" because this indeed would be diverting attention from important confusion. Arguments for not working on them are instead more akin to saying "This tool won't be very useful for building safe value aligned agents in the long run." I think that this is probably true if we are looking to build aligned systems that are competitive with unaligned systems. By definition, an impact penalty can only limit the capabilities of a system, and therefore does not help us to build powerful aligned systems.

To the extent that they meaningfully make cognitive reductions, this is much more difficult for me to analyze. On one hand, I can see a straightforward case for everyone being on the same page when the word "impact" is used. On the other hand, I'm skeptical that this terminology will meaningfully input into future machine learning research.

The above two things are my main critiques of impact measures personally.

I think a natural way of approaching impact measures is asking "how do I stop a smart unaligned AI from hurting me?" and patching hole after hole. This is really, really, really not the way to go about things. I think I might be equally concerned and pessimistic about the thing you're thinking of.

The reason I've spent enormous effort on Reframing Impact [LW · GW] is that the impact-measures-as-traps framing is wrong! The research program I have in mind is: let's understand instrumental convergence on a gears level. Let's understand why instrumental convergence tends to be bad on a gears level. Let's understand the incentives so well that we can design an unaligned AI which doesn't cause disaster by default.

The worst-case outcome is that we have a theorem characterizing when and why instrumental convergence arises, but find out that you can't obviously avoid disaster-by-default without aligning the actual goal. This seems pretty darn good to me.

Congratulations! Apparently it worked!

This feature is important to me. It might turn out to be a dud, but I would be excited to experiment with it. If it was available in a way that was portable to other websites as well, that would be even more exciting to me (e.g. I could do this in my base blog).

Note that this feature can be used for more than forecasting. One key use case on Arbital was to see who was willing to endorse or disagree with, to what extent, various claims relevant to the post. That seemed very useful.

I don't think having internal betting markets is going to add enough value to justify the costs involved. Especially since it both can't be real money (for legal reasons, etc) and can't not be real money if it's going to do what it needs to do.

One small thing you could do is to have probability tools be collapsed by default on any AIAF posts (and maybe even on the LW versions of AIAF posts).

Also, maybe someone should write a blog post that's a canonical reference for 'the relevant risks of using probabilities that haven't already been written up', in advance of the feature being released. Then you could just link to that a bunch. (Maybe even include it in the post that explains how the probability tools work, and/or link to that post from all instances of the probability tool.)

Another idea: Arbital had a mix of (1) 'specialized pages that just include a single probability poll and nothing else'; (2) 'pages that are mainly just about listing a ton of probability polls'; and (3) 'pages that have a bunch of other content but incidentally include some probability polls'.

If probability polls on LW mostly looked like 1 and 2 rather than 3, then that might make it easier to distinguish the parts of LW that should be very probability-focused from the parts that shouldn't. I.e., you could avoid adding Arbital's feature for easily embedding probability polls in arbitrary posts (and/or arbitrary comments), and instead treat this more as a distinct kind of page, like 'Questions'.

You could still link to the 'Probability' pages prominently in your post, but the reduced prominence and site support might cause there to be less social pressure for people to avoid writing/posting things out of fears like 'if I don't provide probability assignments for all my claims in this blog post, or don't add a probability poll about something at the end, will I be seen as a Bad Rationalist?'

I really don't like the removal of the comment counter at the top, because that gave a link to skip to the comments. I fairly often want to skip immediately to the comments to eg get a vibe for if the post is worth reading, and having a one click skip to it is super useful, not having that feels like a major degradation to me

My impression: The new design looks terrible. There's suddenly tons of pointless whitespace everywhere. Also, I'm very often the first or only person to tag articles, and if the tagging button is so inconvenient to reach, I'm not going to do that.

Until I saw this shortform, I was sure this was a Firefox bug, not a conscious design decision.

The new design seems to be influenced by the idea that spreading UI elements across greater distances (reducing their local density) makes the interface less cluttered. I think it's a little bit the other way around, shorter distances with everything in one place make it easier to chunk and navigate, but overall the effect is small either way. And the design of spreading the UI elements this way is sufficiently unusual that it will be slightly confusing to many people.

The title is annoyingly large. 

I like the table of contents on the left becoming visible only upon mouseover.

Why remove "x min read"? Even if it's not gonna be super accurate between different people's reading speeds, I still found it very helpful to decide at a glance how long a post is (e.g. whether to read it on the spot or bookmark it for later).

Showing the word count would also suffice.

I like this; I've found the meta-data of posts to be quite heavy and cluttered (a multi-line title, the author+reading-time+date+comments line, the tag line, a linkpost line and a "crossposted from the Aligned Forum" line is quite a lot).

I was going to comment that "I'd like the option to look at the table-of-contents/structure", but I then tested and indeed it displays if you hover your mouse there. I like that.

When I open a new post, the top banner with the LessWrong link to the homepage, my username etc. show up. I'd prefer if that didn't happen? It's not like I want to look at the banner (which has no new info to me) when I click open a post, and hiding it would make the page less cluttered.

I am really missing the word counter. It's something I look at quite a lot (less so on reading time estimates, as I got used to making the estimate myself based on the wordcount).

My overall review is, seems fine, some pros and some cons, mostly looks/feels the same to me. Some details;

• I had also started feeling like the stuff between the title and the start of the post content was cluttered.
• I think my biggest current annoyance is the TOC on the left sidebar. This has actually disappeared for me, and I don't see it on hover-over, which I assume is maybe just a firefox bug or something. But even before this update, I didn't like the TOC. Specifically, you guys had made it so that there was spacing between the sections that was supposed to be proportional to the length of each section. This never felt like it worked for me (I could speculate on why if you're interested). I'd much prefer if the TOC was just a normal outline-type thing (which it was in a previous iteration).
• I think I'll also miss the word count. I use it quite frequently (usually after going onto the post page itself, so the preview card wouldn't help much). Having the TOC progress bar thing never felt like it worked either. I agree with Neel that it'd be fine to have the word count in the date hover-over, if you want to have less stuff on the page.
• The tags at the top right are now just bare words, which I think looks funny. Over the years you guys have often seemed to prefer really naked minimalist stuff. In this case I think the tags kinda look like they might be site-wide menus, or something. I think it's better to have the tiny box drawn around each tag as a visual cue.
• The author name is now in a sans-serif font, which looks pretty off to me in between the title and the text as serif fonts. It looks like when the browser failed to load the site font and falls back onto the default font, or something. (I do see that it matches the fact that usernames in the comments are sans serif, though.)
• I initially disliked the karma section being so suppressed, but then I read one of your comments in this thread explaining your reasoning behind that, and now I agree it's good.
• I also use the comment count/link to jump to comments fairly often, and agree that having that in the lower left is fine.

I like most of the changes, but strongly dislike the large gap before the title. (I similarly dislike the large background in the top 50 of the year posts)

↑ comment by habryka (habryka4) · 2024-08-16T17:15:46.350Z · LW(p) · GW(p)

Well, the gap you actually want to measure is the gap between the title and the audio player (or at the very least the tags), since that's the thing we need to make space for. You are clearly looking at LW on an extremely large screen. This is the more median experience: 

There is still a bunch of space there, but for many posts the tags extend all the way above the post. 

Replies from: papetoast

↑ comment by papetoast · 2024-08-17T04:23:20.296Z · LW(p) · GW(p)

I understand that having the audio player above the title is the path of least resistance, since you can't assume there is enough space on the right to put it in. But ideally things like this should be dynamic, and only take up vertical space if you can't put it on the right, no? (but I'm not a frontend dev)

Alternatively, I would consider moving them vertically above the title a slight improvement. It is not great either, but at least the reason for having the gap is more obvious.

The above screenshots are done in a 1920x1080 monitor

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-08-17T04:56:28.944Z · LW(p) · GW(p)

Yeah, we could make things dynamic, it would just add complexity that we would need to check every time we make a change. It's the kind of iterative improvement we might do over time, but it's not something that should block the roll-out of a new design (and it's often lower priority than other things, though post-pages in-particular are very important and so get a lot more attention than other pages).

The new design means that I now move my mouse cursor first to the top right, and then to the bottom left, on every single new post. This UI design is bad ergonomics and feels actively hostile to users.

It takes more vertical space than it used to and I don't like that. (Also, the meatball menu is way off in the corner, which is annoying if I want to e.g. bookmark a post, though I don't use it very often so it's not a major inconvenience.) I think I like the new font, though!

On desktop the title font is jarringly huge. I already know the title from the front page, no need to scream it at me.

The new UI is great, and I agree with the thinking behind de-emphasizing karma votes at the top. It could sometimes create inherent bias and assumptions (no matter whether the karma is high or low) even before reading a post, whereas it would make more sense at the end of the post.

The karma buttons are too small for actions that in my experience, are done a lot more than clicking to listen to the post. It's pretty easy to misclick.

Additionally, it's unclear what the tags are, as they're no longer right beside the post to indicate their relevance. 

I ~always want to see the outline when I first open a post and when I'm reading/skimming through it. I wish the outline appeared when-not-hover-over-ing for me.

I like the decluttering. I think the title should be smaller and have less white space above it. Also think that it would be better if the ToC was maybe just faded a lot until mouseover, the sudden appearance/disappearance feels too sudden.

the primary purpose of karma is to use it to decide what to read before you click on a post, which makes it less important to be super prominent when you are already on a post page

I think this applies to titles too

Reading through this, I went "well, obviously I pay the mugger...

...oh, I see what you're doing here."

I don't have a full answer to the problem you're specifying, but something that seems relevant is the question of "How much do you want to invest in the ability to punish defectors [both in terms of maximum power-to-punish, a-la nukes, and in terms of your ability to dole out fine-grained-exactly-correct punishment, a-la skilled assassins]"

The answer to this depends on your context. And how you have answered this question determines whether it makes sense to punish people in particular contexts.

In many cases there might want to be some amount of randomization where at least some of the time you really disproportionately punish people, but you don't have to pay the cost of doing so every time.

Answering a couple of the concrete questions:

Mugger

Right now, in real life, I've never been mugged, and I feel fine basically investing zero effort into preparing for being mugged. If I do get mugged, I will just hand over my wallet.

If I was getting mugged all the time, I'd probably invest effort into a) figuring out what good policies existed for dealing with muggers, b) what costs I'd have to pay in order to implement those policies.

In some worlds, it's worth investing in literal body armor or bullet proof cars or whatever, and in the skill to successfully fight back against a literal mugger. (My understanding is that this usually not actually a good idea even in crime-heavy areas, but I can imagine worlds where it was correct to just get good at fighting, or to hire people who are good at fighting as bodyguards)

In some worlds it's worth investing more in police-force and avoiding having to think about the problem, or not carrying as much money around in the first place.

Small Nation Demands Subsidies, Threatens Nuclear War

Again, I think my options here depend a lot on having already invested in defense.

One scenario is "I do not have the ability to say 'no' without risking millions of either my own citizens lives, or innocent citizens of the country-in-question." In that case, I probably have to do something that makes my vague-hippie-values sad.

I have some sense that my vague-hippie-values depend on having invested enough money in defense (and offense) that I can "afford" to be moral. Things I may wish my country had invested in include:

• Anti-ICBM capabilities that can shoot down incoming nukes with enough reliability that either a small-scale nuclear counterstrike, or a major non-nuclear retaliatory invasion, are viable options that will at least only punish foreign civilians if the foreign government actually launches an attack
• Possibly invested in assassins who just kill individuals who threaten nuclear strikes (I'm somewhat confused about why this isn't more used, suspect the answer has to do with the game theory of 'the people in charge [of all nations] want it to be true that they aren't at risk of getting assassinated, so they have a gentleman's agreement to avoid killing enemy leaders)

So I probably want to invest a lot in either having strong capabilities in those domains, or having allies who do.

Drinking

In real life I expect that the solution here is "I never invite said person to parties again, and depending on our relative social standing I might publicly badmouth them or quietly gossip about them."

In weird contrived scenarios I'm not sure what I do because I don't know how to anticipate weird contrived scenarios.

I do invest, generally, on communicating about how obviously people should follow up on their commitments, such that when someone fails to live up to their commitment, it costs less to punish them for doing so. (And this is a shared social good that multiple people invest in).

If I'm in a one-off interaction with someone who is currently too drunk to remember being punished and who I'm not socially connected to, I probably treat it like being mugged – a fluke event that doesn't happen often enough to be worth investing resources in being able to handle better.

Extra Example: Having to Stand Up to a Boss/High-Status-Person/Organization

A situation that I'm more likely to run into, where the problem actually seems hard, is that sometimes high status people do bad things, and they have more power than you, and people will naturally end up on their side and take their word over yours.

Sort of similar to the "Small nation threatening nuclear war", I think if you want to be able to "afford to actually have moral principles", you need to invest upfront in capabilities. This isn't always the right thing to do, depending on your life circumstances, but it may be sometimes. You want to have enough surplus power that you have the Slack to stand up for things.

Possibilities include investing in being directly high status yourself, or investing in making friends with a strong enough coalition of people to punish high status people, or encourage strong norms and rule of law such that you don't need to have as strong a coalition, because you've made it lower cost to socially attack someone who breaks a norm.

Extra Example: The Crazy House Guest

Perhaps related to the drinking example: a couple times, I've had people show at former houses, potentially looking to move in, and then causing some kind of harm.

In one case, they had a very weird combination of mental illnesses and cluelessness that resulted in them dealing several thousands of dollars worth of physical damage to the house.

They seemed crazy and unpredictable enough that it seemed like if I tried to punish them, they might follow me around forever and make my life suck in weird ways.

So I didn't punish them and they got away with it and went away and I never heard from them again.

So... sure, you can get away with certain kinds of things by signaling insanity and unpredictability... but at the cost of not being welcome in major social networks. The few extra thousand dollars they saved was not remotely worth the fact that, had they been a more reasonable person, they'd have had access to a strong network of friends and houses that help each other out finding jobs and places to live and what-not.

So I'm not worried about the longterm incentives here – the only people for whom insanity is a cost-effective tool to avoid punishment are actual insane people who don't have the ability to interface with society normally.

What if there turn out to be lots of crazy people? Then you probably either invest upfront resources in fighting this somehow, or become less trusting.

Extra Example: The Greedy Landlord

In another housing situation, the landlord tried to charge us extra for things that were not our fault. In this case, it was reasonably clear that we were in the right. Going to small claims court would have been net-negative for us, but also costly to them.

I was angry and full of zealous energy and I decided it was worth it and I threatened going to small claims court and wasting both of our time, even though a few hundred dollars wasn't really worth it.

They backed down.

This seems like the system working as intended. This is what anger is for, to make sure people have the backbone to defend themselves, and to live in a world where at least some of the time people will get riled up and punish you disproportionately

What if you haven't invested in defense capabilities in advance?

Then you probably will periodically need to lose and accept bad situations, such as either a more powerful empire demanding tribute from your country, or choosing policies like "if you are under threat, flip an unknown number of coins and if enough coins come up heads, go to war and punish them disproportionately even though you will probably lose and lots of people will die but now empires will sometimes think twice about invading poor countries."

The meta level point

It doesn't seem inconsistent to me to apply different policies in different situations, even if they share commonalities, based on how common the situation is, how costly the defection, how much long-term punishment you can inflict, and how much resources your have invested in being able to punish.

This does mean that mugging (for example) is a somewhat viable strategy, since people don't invest as heavily in handling it (because it is rare), but this seems like a self-correcting problem. There would be some least-defended against defect-button that defectors can press, you can't protect against everything.

Another point is that it's important to be somewhat unpredictable, and to at least sometimes just punish people disproportionately (when they wrong you), so that people aren't confident that the expected value of taking advantage of you is positive.

Any reason why you mention timeless decision theory (TDT) specifically? My impression was that functional decision theory (as well as UDT, since they're basically the same thing) is regarded as a strict improvement over TDT.

I had forgotten this post, reread it and still think it's one of the better things of it's length I've read recently.

There's also a strong chance that delta is the most transmissible variant we know even without its immune evasion (source: I work on this, don't have a public source to share). I agree with your assessment that delta is a big deal.

The fact that we still use the same sequence to vaccinate seems like civilisational failure. 

Those graphs all show the percentage share of the different variants, but more important would be the actual growth rate. Is the delta variant growing, or is it just shrinking less quickly than the others?

Since this hash is publicly posted, is there any timescale for when we should check back to see the preimage [LW(p) · GW(p)]?