[LW Team] Request for User-Interviews about Tagging/Search/Wikis 2019-11-14T22:16:35.840Z · score: 14 (4 votes)
LW Team Updates - November 2019 (Subscriptions & More) 2019-11-08T02:39:29.498Z · score: 28 (12 votes)
[Team Update] Why we spent Q3 optimizing for karma 2019-11-07T23:39:55.274Z · score: 63 (18 votes)
[Site Update] Subscriptions, Bookmarks, & Pingbacks 2019-10-29T04:32:31.109Z · score: 95 (24 votes)
Open & Welcome Thread - October 2019 2019-10-01T23:10:57.782Z · score: 10 (3 votes)
LW Team Updates - October 2019 2019-10-01T23:08:18.283Z · score: 32 (11 votes)
Novum Organum: Introduction 2019-09-19T22:34:21.223Z · score: 79 (23 votes)
Open & Welcome Thread - September 2019 2019-09-03T02:53:21.771Z · score: 10 (4 votes)
LW Team Updates - September 2019 2019-08-29T22:12:55.747Z · score: 43 (12 votes)
[Resource Request] What's the sequence post which explains you should continue to believe things about a particle moving that's moving beyond your ability to observe it? 2019-08-04T22:31:37.063Z · score: 7 (1 votes)
Open & Welcome Thread - August 2019 2019-08-02T23:56:26.343Z · score: 13 (5 votes)
Do you fear the rock or the hard place? 2019-07-20T22:01:48.392Z · score: 43 (14 votes)
Why did we wait so long for the bicycle? 2019-07-17T18:45:09.706Z · score: 49 (19 votes)
Causal Reality vs Social Reality 2019-06-24T23:50:19.079Z · score: 37 (28 votes)
LW2.0: Technology Platform for Intellectual Progress 2019-06-19T20:25:20.228Z · score: 27 (7 votes)
LW2.0: Community, Culture, and Intellectual Progress 2019-06-19T20:25:08.682Z · score: 28 (5 votes)
Discussion Thread: The AI Does Not Hate You by Tom Chivers 2019-06-17T23:43:00.297Z · score: 36 (10 votes)
Welcome to LessWrong! 2019-06-14T19:42:26.128Z · score: 94 (48 votes)
LessWrong FAQ 2019-06-14T19:03:58.782Z · score: 58 (17 votes)
An attempt to list out my core values and virtues 2019-06-09T20:02:43.122Z · score: 26 (6 votes)
Feedback Requested! Draft of a New About/Welcome Page for LessWrong 2019-06-01T00:44:58.977Z · score: 30 (5 votes)
A Brief History of LessWrong 2019-06-01T00:43:59.408Z · score: 20 (12 votes)
The LessWrong Team 2019-06-01T00:43:31.545Z · score: 24 (7 votes)
Site Guide: Personal Blogposts vs Frontpage Posts 2019-05-31T23:08:07.363Z · score: 34 (9 votes)
A Quick Taxonomy of Arguments for Theoretical Engineering Capabilities 2019-05-21T22:38:58.739Z · score: 29 (6 votes)
Could humanity accomplish everything which nature has? Why might this not be the case? 2019-05-21T21:03:28.075Z · score: 8 (2 votes)
Could humanity ever achieve atomically precise manufacturing (APM)? What about a much-smarter-than-human-level intelligence? 2019-05-21T21:00:30.562Z · score: 8 (2 votes)
Data Analysis of LW: Activity Levels + Age Distribution of User Accounts 2019-05-14T23:53:54.332Z · score: 27 (9 votes)
How do the different star-types in the universe (red dwarf, etc.) related to habitability for human-like life? 2019-05-11T01:01:52.202Z · score: 6 (1 votes)
How many "human" habitable planets/stars are in the universe? 2019-05-11T00:59:59.648Z · score: 6 (1 votes)
How many galaxies could we reach traveling at 0.5c, 0.8c, and 0.99c? 2019-05-08T23:39:16.337Z · score: 6 (1 votes)
How many humans could potentially live on Earth over its entire future? 2019-05-08T23:33:21.368Z · score: 9 (3 votes)
Claims & Assumptions made in Eternity in Six Hours 2019-05-08T23:11:30.307Z · score: 46 (13 votes)
What speeds do you need to achieve to colonize the Milky Way? 2019-05-07T23:46:09.214Z · score: 6 (1 votes)
Could a superintelligent AI colonize the galaxy/universe? If not, why not? 2019-05-07T21:33:20.288Z · score: 6 (1 votes)
Is it definitely the case that we can colonize Mars if we really wanted to? Is it reasonable to believe that this is technically feasible for a reasonably advanced civilization? 2019-05-07T20:08:32.105Z · score: 8 (2 votes)
Why is it valuable to know whether space colonization is feasible? 2019-05-07T19:58:59.570Z · score: 6 (1 votes)
What are the claims/arguments made in Eternity in Six Hours? 2019-05-07T19:54:32.061Z · score: 6 (1 votes)
Which parts of the paper Eternity in Six Hours are iffy? 2019-05-06T23:59:16.777Z · score: 18 (5 votes)
Space colonization: what can we definitely do and how do we know that? 2019-05-06T23:05:55.300Z · score: 31 (9 votes)
What is corrigibility? / What are the right background readings on it? 2019-05-02T20:43:45.303Z · score: 6 (1 votes)
Speaking for myself (re: how the LW2.0 team communicates) 2019-04-25T22:39:11.934Z · score: 47 (17 votes)
[Answer] Why wasn't science invented in China? 2019-04-23T21:47:46.964Z · score: 78 (26 votes)
Agency and Sphexishness: A Second Glance 2019-04-16T01:25:57.634Z · score: 27 (14 votes)
On the Nature of Agency 2019-04-01T01:32:44.660Z · score: 30 (10 votes)
Why Planning is Hard: A Multifaceted Model 2019-03-31T02:33:05.169Z · score: 37 (15 votes)
List of Q&A Assumptions and Uncertainties [LW2.0 internal document] 2019-03-29T23:55:41.168Z · score: 25 (5 votes)
Review of Q&A [LW2.0 internal document] 2019-03-29T23:15:57.335Z · score: 25 (4 votes)
Plans are Recursive & Why This is Important 2019-03-10T01:58:12.649Z · score: 61 (24 votes)
Motivation: You Have to Win in the Moment 2019-03-01T00:26:07.323Z · score: 49 (21 votes)


Comment by ruby on What Comes After Epistemic Spot Checks? · 2019-11-20T01:39:06.446Z · score: 9 (4 votes) · LW · GW


In my mind, Elizabeth's work on Epistemic Spot Checks (ESC's) generally, and this post in particular, are exactly the kinds of effort to innovate in truth-seeking that are core to LessWrong.

It's been written that the proper use of the word rational is to not simply mean "true" or "optimal", but instead to describe algorithms that systematically increase map-territory correspondence. I love the work on ESC and this post asking what comes next because they concern precisely this level of question: what algorithms systematically get us to truth? What processes can we follow.

I'm excited for more work in this direction and the LW community developing expertise not just in talking about things, but really, actually getting at what's is the case in the territory. 

Would love to hear people weigh in more with their thoughts.

Comment by ruby on Benito's Shortform Feed · 2019-11-14T08:40:30.169Z · score: 4 (2 votes) · LW · GW

That clarifies it and makes a lot of sense. Seems my objection rested upon a misunderstanding of your true intention. In short, no worries.

I look forwards to figuring this out together.

Comment by ruby on Benito's Shortform Feed · 2019-11-14T08:25:16.492Z · score: 2 (1 votes) · LW · GW

I'm currently working through my own thoughts and vision for tagging.

If and when we end up building a tagging system on LessWrong, the goal will be to distinguish the main types of post people are interested in viewing, and create a limited number of tags that determine this. I think building this will mainly help users who are viewing new content on the frontpage, and that for much more granular sorting of historical content, a wiki is better placed.

I'm pretty sure I disagree with this and object to you making an assertion that makes it sound like the team is definitely decided about what the goal of tagging system will be.

I'll write a proper response tomorrow.

Comment by ruby on Raemon's Scratchpad · 2019-11-12T00:09:41.303Z · score: 8 (3 votes) · LW · GW

I really like the green-unread on post pages. On Recent Discussion I have so much of it that I think I don't really pay attention to it.

Comment by ruby on LW Team Updates - November 2019 (Subscriptions & More) · 2019-11-11T20:01:04.128Z · score: 2 (1 votes) · LW · GW

Fixed, thanks!

Comment by ruby on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-09T02:07:24.155Z · score: 4 (2 votes) · LW · GW

I find this compelling (along with the "finding out which things matter that you didn't realize mattered) and think this is a reason for us to begin doing A/B testing sometime in not too distant future.

Comment by ruby on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-09T02:04:04.996Z · score: 12 (4 votes) · LW · GW

I see the spirit of what you're saying and think there's something to it though it doesn't feel completely correct. That said, I don't think anyone on the team has experience with that kind of A/B testing loop and given that lack of experience, we should try it out for at least a while on some projects.

To date, I've been working just to get us to have more of an analytics-mindset plus basic thorough analytics throughout the app, e.g. tracking on each of the features/buttons we build, etc. (This wasn't trivial to do with e.g. Google Tag Manager so we've ended up building stuff in-house.) I think trying out A/B testing would likely make sense soon, but as above, I think there's a lot of value even before it with more dumb/naive analytics.

We trialled FullStory for a few weeks and I agree it's good, but also we just weren't using it enough to justify it. LogRocket offers monthly subscription though and likely we'll sign up for that soon. (Once we're actually using it fully, not just trialling, we'll need to post about it properly, build opt-out, etc. and be good around privacy - already in trial we hid e.g. voting, usernames.)

To come back to the opening points in the OP, we probably shouldn't get too bogged down trying to optimize specific simple metrics by getting all the buttons perfect, etc., given the uncertainty over which metrics are even correct to focus on. For example, there isn't any clear metric (that I can think of) that definitely answers how much to focus on bringing in new users and getting them up to speed vs building tools for existing users already producing good intellectual progress. I think it's correct that have to use high-level models and fuzzier techniques to think about big project prioritization. A/B tests won't resolve the most crucial uncertainties we have though I do think they're likely to hugely helpful in refining our design sense.

Comment by ruby on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-09T01:51:26.354Z · score: 6 (3 votes) · LW · GW

This is roughly the procedure we usually follow.

Comment by ruby on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-08T02:33:31.878Z · score: 8 (5 votes) · LW · GW
Trying to optimize a metric without even having a test framework in place adds a lot of evidence to that story - certainly in my own start-up experience, we never had any idea what we were doing until well after the test framework was in place (at any of the companies I've worked at). Analytics more generally were also always crucial for figuring out where the low-hanging fruit was and which projects to prioritize, and it sounds like you guys are currently still flying blind in that department.

I think I agree with the general spirit here. Throughout my year with the LessWrong team, I've been progressively building out analytics infrastructure to reduce my sense of the "flying blind" you speak of. We're not done yet, but I've now got a lot of data at my fingertips. I think the disagreement here would be over whether anything short of A/B testing is valuable. I'm pretty sure that it is.

Comment by ruby on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-08T02:28:05.611Z · score: 9 (5 votes) · LW · GW

A number of these projects were already on our docket, but less visible is the projects which were delayed and the fact that those selected might not have been done now otherwise. For example, if we hadn't been doing metric quarter, I'd like have spent more of my time continuing work on the Open Questions platform and much less of my time doing interviews and talking to authors. Admittedly, subscriptions and the new editor are projects we were already committed to and had been working on, but if we hadn't thought they'd help with the metric, we'd have delayed it to the next quarter the way we did with many of other project ideas.

We did brainstorm however, but as Oli said, it wasn't easy to come with any ideas which were obviously much better.

Comment by ruby on [Team Update] Why we spent Q3 optimizing for karma · 2019-11-08T02:21:26.451Z · score: 13 (5 votes) · LW · GW

Heckling appreciated. I'll add a bit more to Habryka's response.

Separate from the question of whether A/B would have been applicable to our projects, I'm not sure why think it's pointless to try to make inferences without them. True, A/B tests are cleaner and more definitive, and what we observed is plausibly what would have happened even with different activities, but that isn't to say we don't learn a lot when the outcome is one of a) metric/growth stays flat, b) small decrease, c) small increase, d) large decrease, e) large increase. In particular, the growth we saw (increase in absolute and rate) is suggestive of doing something real and also strong evidence against the hypothesis that it'd be very easy to drive a lot of growth.

Generally, it's at least suggestive that the first quarter where we explicitly we focus on growth is one where we see 40% growth from last quarter (compared to 20% in the previous quarter to the one before). It could be a coincidence, but I feel like there are still likelihood ratios here.

When it comes to attribution too, with some of these projects it's easy to get much more of an idea even without A/B testing. I can look at the posts from authors who we contacted and reasonably believe counterfactually would not have otherwise posted and see how much karma that generated. Same from Petrov Days and MSFP.

Comment by ruby on Aella on Rationality and the Void · 2019-10-31T22:42:47.959Z · score: 4 (2 votes) · LW · GW


Comment by ruby on steve2152's Shortform · 2019-10-31T16:47:05.216Z · score: 2 (6 votes) · LW · GW

Because they're interested in weapons and making people distinctly not safe.

Comment by ruby on steve2152's Shortform · 2019-10-31T14:51:15.946Z · score: 9 (5 votes) · LW · GW

A friend in the AI space who visited Washington told me that military leaders distinctly do not like the term "safety".

Comment by ruby on On Internal Family Systems and multi-agent minds: a reply to PJ Eby · 2019-10-30T22:35:06.850Z · score: 2 (1 votes) · LW · GW

Recently clarified guidelines:

Comment by ruby on Theater Tickets, Sleeping Pills, and the Idiosyncrasies of Delegated Risk Management · 2019-10-30T19:57:31.997Z · score: 3 (2 votes) · LW · GW

But sleeping pills aren't just about stress, they're very concretely about putting you to sleep which they do.

Comment by ruby on Turning air into bread · 2019-10-30T19:48:32.656Z · score: 18 (5 votes) · LW · GW

No matter how many false physical constraints we overturn the second law of thermodynamics seems to guarantee (this is debatable) that we will eventually hit a wall . . .

Granted that we will eventually hit a wall, there's a good chance the wall is so unbelievably far off that it might as not exist for another million or billion years and allow for astronomical (literally) amounts of growth. Heck, even what we get out of the Earth alone could be increased multiple orders of magnitude. Suppose there's a point at thinking about slowing down, I think that point is very far away.

I'll quote a bit from my summary of Eternity in Six Hours, which I find credible:

  • Travelling at 50c% there are 116 million galaxies reachable; at 80% there are 762 million galaxies reachable; at 99%c, you get 4.13 billion galaxies.
    • For reference, there are 100 to 400 billion stars in the Milky Way, and from a quick check it might be reasonable to assume 100 billion is the average galaxy.
      • The ability to colonize the universe as opposed to just the Milky Way is the difference between ~10^8 stars and ~10^16 or ~10^17 starts. A factor of 100 million.

Similarly, the sun's estimated energy output is 3.8x10^26W (Joules per Second) whereas civilization's current energy usage is estimate at ~10^24J/year in a recent year (2012 or 2014?). That's something like 9 orders of magnitude of more energy that's being expended than we currently use (simplifying a whole bunch). 

There's finite and there's finite, some of those finite's are freaking huge. I say let's get 'em.

But being more serious, if we think about the EV of different strategies, I think the EV continuing to pursue growth (as jasoncrawford defines it) for the foreseeable future is better than very prematurely trying to limit growth and be "sustainable" notwithstanding the risks that eventually there will be some kind of crunch. 

Admittedly, I could be wrong about the limits of potential technological capabilities. If for some reason we hit a a limit of what we can do far earlier, then there might be a wall far sooner than when we run out of energy. But even such a wall seems at least quite a ways off.

Comment by ruby on Theater Tickets, Sleeping Pills, and the Idiosyncrasies of Delegated Risk Management · 2019-10-30T19:09:16.342Z · score: 4 (2 votes) · LW · GW

Good post. I like these examples, though I find the names you've given to each type to not that be that evocative and I expect to struggle to remember them or explain them to others. For example, sleeping pills do something real. Perhaps you mean something like sugar (placebo) pills? Or antibiotics for a cold?

Comment by ruby on Rugby & Regression Towards the Mean · 2019-10-30T18:25:00.648Z · score: 4 (2 votes) · LW · GW

I really appreciate seeing this kind of applied statistical analysis to a stray interesting-sounding fact you heard.

Comment by ruby on [Site Update] Subscriptions, Bookmarks, & Pingbacks · 2019-10-30T18:19:58.027Z · score: 6 (4 votes) · LW · GW


Comment by ruby on How do you assess the quality / reliability of a scientific study? · 2019-10-30T05:46:17.605Z · score: 6 (2 votes) · LW · GW
I've probably read about 1000 papers. Lessons learned the hard way...

Very cool. How have these been split across different fields/domains?

Comment by ruby on How do you assess the quality / reliability of a scientific study? · 2019-10-29T18:46:21.121Z · score: 25 (6 votes) · LW · GW

I think this is a really important question and I'm eager to see answers. I'm willing to put up $100 of my personal money as a prize for what I think is the best answer and another $50 for what I think is the best summary of multiple answers. (I'm willing to combine these of the best answer includes summary of other answers.)

This isn't a proper impact certificate, but if it were, I might be awarding this prize at 5% or less of the true value of the impact. So in offering $100, I'm saying the real impact could be worth like $2000 or more in my mind if it's a really good answer.

Assuming Eli is okay with this, I'll evaluate in two weeks, ending November 13 at 12:00 AM, and pledge to award within three weeks (for each day I'm late, I'll increase the prize amounts by 5% compounding).

A thing I would be interested in here is also people mentioning how they gained their ability to assess papers, e.g. "I can do this because I have a statistics degree" or "I can do this because of my domain expertise" and further bonus points on listing resources people could use to enhance their ability to assess research.

Comment by ruby on Ruby's Public Drafts & Working Notes · 2019-10-24T20:40:49.565Z · score: 4 (2 votes) · LW · GW

Great quote from Francis Bacon (Novum Organum Book 2:8):

Don’t be afraid of large numbers or tiny fractions. In dealing with numbers it is as easy to write or think a thousand or a thousandth as to write or think one.
Comment by ruby on Infinite Certainty · 2019-10-08T01:58:33.918Z · score: 2 (1 votes) · LW · GW

You might want to see How to Convince Me That 2 + 2 = 3

Even if you believe that mathematical truths are necessarily true, you can still ask why you believe that they are necessarily true. What caused you to believe it? Likely whatever process it is is fallible.

I'll quote you what I commented elsewhere on this topic:

Let's suppose you believe that 2+2=4 follows axiomatically from Peano axioms or something. The question is what kind of evidence should convince you that 2+2=4 doesn't follow from those axioms? According the post, it'd be exactly the same kind of evidence that convinced you 2+2=4 does follow from the axioms. Perhaps you wake up one day and find that when you sit down to apply the axioms, working through them step by step, you get 2+2=3. And when you open up a textbook it shows the same thing, and when you ask your math professor friend, and when you just think about it in your head.
I suppose the point is that how you interact with mathematical proofs isn't much different from how you interact with the rest of the world. Mathematical results follow in some logically necessary ways, but there's a process of evidence that causes you to have contingent beliefs even about things that themselves are seemingly necessarily could only be one way.
Cf. logical omniscience and related lines of inquiry.

I realize I haven't engage with your Epsilon scenario. It does seem pretty hard to imagine and assign probabilities to, but actually assigning I seems like a mistake.

Comment by ruby on Discussion Thread: The AI Does Not Hate You by Tom Chivers · 2019-10-07T20:17:41.234Z · score: 9 (4 votes) · LW · GW

Scott Aaronson has now written a review.

Comment by ruby on LW Team Updates - October 2019 · 2019-10-04T04:34:03.123Z · score: 2 (1 votes) · LW · GW


Comment by ruby on LW Team Updates - October 2019 · 2019-10-02T17:56:18.372Z · score: 2 (1 votes) · LW · GW


Comment by ruby on LW Team Updates - October 2019 · 2019-10-01T23:26:08.339Z · score: 7 (1 votes) · LW · GW

Hints & Tips: The All Posts Page

LessWrong has a powerful All Posts page which can serve as an alternative to the frontpage. It's URL is The All Posts page has:

  • Options for filtering all LessWrong content by category, e.g. Frontpage vs Personal blogposts, Curated, Questions, Events, and Meta (mostly deprecated).
  • Option to sort posts by Magic (New & Upvoted), Recent Comments, New, Old, and Top.
  • Options to group posts by time periods of All Time, Daily, Weekly, Monthly, or Yearly.
  • The All Posts page is also a way to view Shortform posts which show up under the post lists when a non-All Time time period is selected.
Comment by ruby on Noticing Frame Differences · 2019-10-01T00:30:44.138Z · score: 3 (2 votes) · LW · GW

These seems pretty good and I think your current approach might suffice for this.

Comment by ruby on Noticing Frame Differences · 2019-10-01T00:29:46.353Z · score: 2 (1 votes) · LW · GW
I think I'm mostly just not trying to do science at this point.

"Methodology of science" was in quotes because I meant more generally "process for figuring things out." I think we do have different leanings there, but they won't be quick to sort out, mostly because (at least in my case) they're more leanings and untested instincts. So mostly just flagging for now.

*I* don't necessarily know what I mean by frame at this point, so being more specific just gives people an inaccurate representation of my own understanding.

I think there are ways to be very "precise about your uncertainty" though.

I don't we have the luxury of doing this – each disagreement I've found was fairly unique, and required learning a new set of things to notice and pay attention to, and stretched my conception of what it meant to disagree productively.

Hmm, it does seem that naturally occurring debates will be over all the place in a way that will make want you to study them all simultaneously.

Comment by ruby on Noticing Frame Differences · 2019-09-30T20:33:24.101Z · score: 2 (1 votes) · LW · GW

I think we've got some significant disagreements about "methodology of science" here. I think that much of this approach feels off to me, or at least concerning. I'll try to write a bit more about this later (edit: I find myself tempted to write several things now . . .).

And I think trying to define "frame" more rigorously would be more likely to give people a new false sense that they understand what's going on (a new "frame" to be locked into without noticing that you're in a frame), than to do anything helpful.

E.g. defining "frame" more rigorously would let people know more specifically what you're trying to talk about and help both you and them more specific/testable predictions and more quickly notice where the model fails in practice, etc. More precise definitions should lead to sticking your neck out more properly. Not that I'm an expert at doing this, but this feels like a better way to proceed even at the risk that people get anchored on bad models (I imagine a way to avoid this is to create, abandon, and iterate on lots of models, many which are mistaken and get abandoned, so people learn the habit.)

This is actually a good example of the sort of failure I'm worried about if I defined frame too cleanly. If you focus on "within gears mismatch" I predict you'll do worse at resolving disagreements that aren't about that.

So it's only my sense of what that word refers to when people say it that has been shifted, not my models, expectations, or predictions of the world. That seems fine in this context (I dislike eroding meanings of words with precise meanings via imprecise usage, but "frame" isn't in that class). The second sentence here doesn't seem related to the first. To your concern though, I say "maybe?", but I have a greater concern that if you don't focus on how to address a single specific type of frame clash well enough, you won't learn to address any of them very well.

Comment by ruby on Noticing Frame Differences · 2019-09-30T16:48:54.570Z · score: 4 (3 votes) · LW · GW

Copied from a Facebook conversation about this post which started with a claim that Raemon's usage of "frame" was too narrow..

[1. I like this version of the post having seen some drafts, seems pretty good to me. Clean, I think it conveys the idea very neatly and will be useful to build upon/reference.

2. Talking in 3rd person about Ray 'cause it's just clearer who I'm talking to.]

Pretty sure I'm the disagreeing person in question. I've been exposed to Ray's usage of "frame" for so long now that it's muddied my sense of what I would have used "frame" to mean before Ray starting talking about it a lot, though I do recall my prior usage being more specific, like a within-gears mismatch rather than a between gears/feelings mismatch.

My feedback at this point would be less around what specifically frame gets used for, and more that there's still a lot of ambiguity. There's a very broad set of variables that can differ between people in conversation: perspectives/backgrounds/assumptions/goals/motives/techniques/methods/culture/terminology/ontology/etc./etc. and there are lots of similarities and differences. Based on conversation, my understanding is that Ray wants to talk about all of them.

I'd kind of like a post that tries to establish clear terminology for discussing these using both intensionsal/extensional definitions and both positive/negative examples. Also maybe how "frames" related to other concepts in conversation space. I think the OP only has positive extensional examples that gesture generally at the cluster of concepts but doesn't help me get a sense of the boundaries of what Ray has in mind.

I expect the above post I'm suggesting to be kind of boring/dry (at least as I would naively write it) but if it was written, it would enable deeper conversations and better insights into the whole topic of conversation. Conceivably it's boring to have to learn how velocity/force/energy/power/momentum all mean very similar but still different things, but modern physics is powerful because it can talk so precisely about things.

I think posts/sequences I would like would be 1) how to notice/handle/respond to mismatches to between different gears-frames, or just how to navigate gears/feelings clashes. Specific and concrete. (This is basically Bacon's inductivism, spirit of Virtue of Precision, Specificity Sequence. I guess just a lot of things nudging my intuitions towards specificity rather than talking about all in the instances of a thing.

I think that's my revised current feedback.

ETA: Taking another look, there is intensional definition in the post, "seeing, thinking, and communicating", which are a helpful and a good start, I'd probably like those fleshed out more and connected to the examples.

Comment by ruby on Follow-Up to Petrov Day, 2019 · 2019-09-29T14:50:33.792Z · score: 7 (3 votes) · LW · GW

I've had pretty much the same objection to making Petrov day about UC and overall haven't felt entirely satisfied with the counters. (Same with Petrov kind of being a unilateralist, though I moderately buy Habryka's view that he was unilateralist with respect to local command structure but non-unilateralist with respect to humanity.) I generally have felt averse to what feels like dilution of the meaning of UC to something much closer to simple defection.

However, there is a way in which there is kind of disagreement about public benefits in the recent LW button scenario if you can sell your ability to nuke the site to someone who is a troll or motivated by spite, and then use the funds for something you think is good that outweighs the harm. At last Jeff K considered this but was dissuaded by others thinking it was a bad idea.

I like your donation/anonymity suggestion. The public shaming seems like a strong disincentive that disanalogizes the scenario from real world scenarios without this element (or where action would at least draw obvious praise from many).

The other direction is having an "opposing" site that our button takes down and vice versa plus chance of false alarm to more accurately resemble the history (though that is still more what I'd call a standard game theory/cooperation/defection thing than UC).

Comment by ruby on Follow-Up to Petrov Day, 2019 · 2019-09-28T01:26:24.784Z · score: 3 (2 votes) · LW · GW

If you had a valid code you could have just created a new account or given it to someone else . . .

Comment by ruby on Idols of the Mind Pt. 2 (Novum Organum Book 1: 53-68) · 2019-09-28T00:41:05.727Z · score: 11 (3 votes) · LW · GW

Huh, I think we're getting caught between the people who want them more frequently and less. I agree that every 2-days might be a bit much, so yeah, let's go with less. For the enthusiastic, I can note that the full text is already online at

Comment by ruby on Rationality and Levels of Intervention · 2019-09-27T19:11:30.952Z · score: 2 (1 votes) · LW · GW
There's a tricky balance to maintain here.

Very much so.

With epistemic dangers, I think there is a choice between "confront" and "evade".

Not a bid for further explanation, just flagging that I'm not sure what you actually mean by this, as in which concrete moves which correspond to each.

If a person is unwilling to take hits to their morale, they are unlikely to be wisely managing their morale and epistemics, and instead trading off too hard against their epistemics.

To me the empirical question is whether a person ought to be willing to take all possible hits to their morale for the sake of their epistemics. I have a consequentialist fear—and I think consequentialist means we're necessarily talking empiricism—that any exceptions/compromizes may be catastrophic.

. . .

It's possible there's a kind of meta-debate here going on, with some people (including me) sometimes having underlying consequentialist/empirical beliefs that even engaging in consequentialist/empirical arguments about trading off against epistemics would have overall bad consequences and/or an empirical belief that anyone who would offer such arguments readily must not really care about epistemics because they're not [naively] treating them as sacred enough [1].

I hadn't formulated this in that way before, so I'm glad this post/discussion has helped me realize that arguably "it's consequentialism/empiricism all the way up", even if you ultimately claim that your epistemological consequentialism cashes out to some inviolable deontological rules.

[1] Not treating them as sacred enough therefore they don't really care, therefore can't be trusted - this is my instinctive reaction when encounter, say, post-rationalist arguments about needing to consider what's useful, not just what's true. Maybe it's not always fair.

. . .

I had a revealing exchange with someone a few months ago about conversation norms on LessWrong. I was stating the necessity of considering the consequences of your speech and how that should factor into how one speaks. In course of that debate, they said [paraphrasing]:

"You' re trying to get me to admit that I sometimes trade off things against truth, and once I've admitted that, we're just "haggling over price". Except, no.

I think this response was a mistake, not in least because their rigidity meant we couldn't discuss different consequences of different policies or even what tradeoffs I thought I was making (fewer than they did). That discussion felt different than this post because it was mostly about what you say to others and how, but I see the analogy to even when you're considering how people individually think.

So, I may maintain my suspicions, but I won't say "except, no."

Comment by ruby on Honoring Petrov Day on LessWrong, in 2019 · 2019-09-27T17:04:23.293Z · score: 24 (7 votes) · LW · GW

I think we can celebrate that Petrov didn't want to destroy the world and this was a good impulse on his part. I think if we think it's doubtful that he made the correct decision, or that it's complicated, then we should be very, very upfront about that (your comment is upfront, the OP didn't make this fact stick with me). The fact the holiday is named after him made me think (implicitly if not explicitly) that people (including you, Ben) generally endorsed Petrov's reasoning/actions/etc. and so I did take the whole celebration as a claim about his reasoning. I mean, if Petrov reasoned poorly but happened to get a good result, we should celebrate the result yet condemn Petrov (or at least his reasoning). If Petrov reasoned poorly and took actions there were poor in expectation, doesn't that mean something like in the majority of world's Petrov caused bad stuff to happen (or at the algorithm which is Petrov generally would)?

. . .

I think it is extremely extremely weird to make a holiday about avoiding unilateralist's curse and name it after who did exactly that. I hadn't thought about it, but if Quanticle is right, then Petrov was taking unilateralist action. (We could celebrate that he his unilateralist action was good, but then the way Petrov day is being themed is here is weird.)

As an aside for those at home, I actually objected to Ben about the "unilateralist"/"unilateralist's curse" framing separately because our button seemed a very non-obvious instance of Bostrom's original meaning* to apply this to Petrov. Unilateralist's curse (Bostrom, 2013) is about when a group of people all have the same goals but have different estimates whether an action which effects them all would be beneficial goal. The curse is that the more people in the group, the more likely someone is to have a bad estimate of the value of the goal that they act separate from everyone else. In the case of US vs Russia, this is an adversarial/enemy situation. If two people are enemies and one decides to attack the other, while it is perhaps correct to say they do so "unilaterally" but it's not the phenomenon Bostrom was trying to point out with his paper/introduction of that term and I'm the kind of person who dislike when people's introduced terminology gets eroded.

I was thinking this at the time I objected, but we could say Petrov had the same values as the rest of the military command but had different estimate of the value of a particular action (what to report), in which case we're back to the above where he was taking unilateralist action.

Our button scenario is another question. I initially thought that someone would only press it if they were a troll (typical mind fallacy, you know?) in which case I'd call it more "defection" than "unilateralist" action and so it wasn't a good fit for the framing either. If we consider that some people might actually believe the correct thing (by our true, joint underlying values) is to press the button, e.g. to save a life via donation, then that actually does fit the original intended meaning.

There other lessons of:

  • Practice not pressing big red buttons that would have bad effects, and
  • Isn't it terrible that the world is making it easier and easier to do great harm, let's point this out by doing the same . . . (ironically, I guess)

*I somewhat dislike that the OP has the section header "unilateralist action", a term taken from Bostrom in one place, but then quotes something he said elsewhere maybe implying that the "building technologies that could destroy the world" was part of the original context for unilateralist's curse.

. . .

Okay, those be the objections/comments I had brewing beneath the surface. Overall, I think our celebration of Petrov was pretty damn good with good effects and a lot of fun (although maybe it was supposed to be serious...). Ben did a tonne of work to make it happen (so did the rest of the team, especially Ray working hard to make the button).

Agree that it was a significant historical event and case study. My comments are limited to the "unilateralist" angle mostly and a bit the we should be clear which behavior/reasoning we're endorsing. I look forward to doing the overall thing again.

Comment by ruby on Honoring Petrov Day on LessWrong, in 2019 · 2019-09-27T00:40:57.162Z · score: 7 (4 votes) · LW · GW

Comment by ruby on Resetting Somebody Will · 2019-09-25T22:37:18.185Z · score: 2 (1 votes) · LW · GW

Yeah, but I have those beliefs.

Comment by ruby on Novum Organum: Introduction · 2019-09-25T21:24:58.529Z · score: 4 (2 votes) · LW · GW

This is really good! No, I didn't think to look for the preface for the entire work. Thanks for raising this. It's probably okay for us to quote some passages, though I'd be hesitant to post the whole thing from Libgen for copywrite reasons. (We have the license to post the version we're posting, but I'd be surprised if Cambridge press was as permissive.)

Comment by ruby on Rationality and Levels of Intervention · 2019-09-25T21:19:50.168Z · score: 6 (4 votes) · LW · GW

I think I'm willing to concede that there is something of an empirical question about what works best for truth-seeking, as much as that feels like a dangerous statement to acknowledge. Though seemingly true, it feels like it's something that people who try to get you commit bad epistemic moves like to raise [1]. I'm thinking here of post-rationalist lines of thought (though I can't claim overmuch familiarity with them) or the perennial debates over whether it's ever okay to deceive yourself. Whether or not they do so doesn't make it less true however.

Questions over how quickly to aim for consensus or how long to entertain new and strange ideas seem like very important questions. There've been recent debates about this kind of thing on LessWrong, particularly the entertaining as-yet-not-completely-justified-in-the-standard-frame things. It does seem like getting the correct balance between Babble vs Prune is an empirical question.

Allowing questions of motivation to factor into one's truth-seeking process feels most perilous to me, mostly as it seems too easy to claim one's motivation will be affected adversely to justify any desired behavior. I don't deny certain moves might destroy motivation, but it seems the risks of allowing such a fear to be a justification for changing behavior are much worse. Granted, that's an empirical claim I'm making.

[1] Or at least it feels that way, because it's so easy to assert that something is useful and therefore justified despite violating what seem like the correct rules. By insisting on usefulness, one can seemingly defend any belief or model. Crystals, astrology, who knows what. Though maybe I merely react poorly at what seem like heresies.

Comment by ruby on Resetting Somebody Will · 2019-09-25T20:51:52.100Z · score: 4 (2 votes) · LW · GW

Miranda wanted this song as part of our 3rd wedding anniversary ceremony and I had the same complaint Habryka does. I think we had it but changed it slightly?

The interpretations you mention seem reasonable but still feels like the straightforward interpretation is of someone saying they don't expect to have the result for themselves in a way that feels fatalist and false or something.

Comment by ruby on Habryka's Shortform Feed · 2019-09-25T05:37:08.003Z · score: 2 (1 votes) · LW · GW

This is really good and I missed it until now. I vote for you making this a full-on post. I think it's fine as is for that.

Comment by ruby on The Power to Teach Concepts Better · 2019-09-25T05:33:43.002Z · score: 2 (1 votes) · LW · GW

I suppose a "coat-hanger" is something which is hung upon and mind-hanger kept raising that concept in analogy to mind hanger, but the analogy to coat hanger feels weird still.

Comment by ruby on The Power to Teach Concepts Better · 2019-09-25T05:32:06.886Z · score: 2 (1 votes) · LW · GW

The etymology makes sense. Perhaps the issue is that mind-hanger makes it sound like it is the thing doing the hanging rather than the thing being hung upon. Perhaps "pre-existing mental hooks" is closer.

Even this still feels slightly off because the name feels like it implies those concepts exist for the purpose of being hooks rather than happening to be the most suitable concepts to build upon. So perhaps "hookable concepts" or something. "Hook points." Those don't sound great, but conceptually they feel like they fit better maybe.

When you go to teach someone something new, you should try to find suitable hookable points in their pre-existing knowledge. Or something.

Comment by ruby on This is a test post · 2019-09-25T02:01:23.700Z · score: 2 (1 votes) · LW · GW

Chris_Leong (1) Counterfactuals are an Answer, Not a Question; Davidmanheim (1) Hackable Rewards as a Safety Valve?; Donald Hobson (1) Logical Counterfactuals and Proposition graphs, Part 3; Grue_Slinky (1) What are concrete examples of potential "lock-in" in AI research?; (2) Dissatisfactions with Garrabrant Induction; Stuart_Armstrong (1) Best utility normalisation method to date?; (2) Could I survive an ontology change?; (3) Issues with conservative agency; (4) Toy model; (5) Simple and composite partial preferences; (6) Is my result wrong? Maths vs intuition vs evolution in learning human preferences; (7) Toy model piece #4: partial preferences, re-re-visited; (8) Toy model piece #5: combining partial preferences; TurnTrout (1) What You See Isn't Always What You Want; Wei_Dai (1) AI Safety "Success Stories"; (2) Counterfactual Oracles = online supervised learning with random selection of training episodes; abergal (1) Conversation with Paul Christiano; abramdemski (1) Do Sufficiently Advanced Agents Use Logic?; evhub (1) Concrete experiments in inner alignment; (2) Are minimal circuits deceptive?; (3) Relaxed adversarial training for inner alignment; johnswentworth (1) Probability as Minimal Map; (2) How to Throw Away Information; (3) Theory of Ideal Agents, or of Existing Agents?; michaelcohen (1) Utility uncertainty vs. expected information gain; paulfchristiano (1) The strategy-stealing assumption; rohinmshah (1) [AN #63] How architecture search, meta learning, and environment design could lead to general intelligence; (2) [AN #64]: Using Deep RL and Reward Uncertainty to Incentivize Preference Learning; vlad_m (1) Utility ≠ Reward; wdmacaskill (1) A Critique of Functional Decision Theory;

Comment by ruby on What are some of your "Crazy Ideas" that you're currently thinking about? · 2019-09-25T00:57:51.336Z · score: 2 (1 votes) · LW · GW

Definitely a question worth thinking about.

A difficulty with your proposal of textbooks being free and their authors getting paid once by the government is that now someone has to decide which textbook authors get paid. If you pay anyone who writes a textbook you'll find there are a lot of people writing quite crappy textbooks. Right now there's some process (probably at the level of teachers?) who decide which textbooks are worth buying and it wouldn't be easy to have all the teachers to all coordinate on which textbooks should be bought with an unlimited license by the government.

High school textbook are probably a crappy market at present, but it's still something of pricing mechanism I think you'd need to replicate.

Comment by ruby on The Power to Teach Concepts Better · 2019-09-25T00:31:37.452Z · score: 10 (5 votes) · LW · GW

Great post! Very clear, especially the Paul Graham/start-up pitch section.

Small feedback though "mind-hangers" continue to feels weird as terminology to me. Like I've managed to get my mind to attach to the concept you're pointing but it somehow still feels wrong, like each time you use it I need to run a patch to substitute in the actual thing.

Other than that, seems pretty spot on. I think there is a caveat to be made that even when leading with examples, you want to be clear upfront what the examples are going to be an example of, e.g. "I'm going to give you an example of Sunk Cost Fallacy." Not a mistake I think you're likely to make, but sometimes people start with particulars and you have no idea where they're going.

I guess generally the "start with examples" needs to be differentiated from different good advice (particularly for business contexts) which is "start with conclusions." The advice is actually compatible since conclusions can be very specific and conclusions often serve as very useful "mind-hangers." Once I know what you're arguing for, I can start to assemble each of the pieces you're giving me. So in short, start with examples doesn't necessarily mean start with nitty gritty details.

Comment by ruby on Therapy vs emotional support · 2019-09-25T00:04:10.169Z · score: 7 (4 votes) · LW · GW

An model which finds commonality between the emotional support of friends and therapists (though not in a way that really contradicts any of the OP):

A General Theory of Love by Lewis, Amini, and Lannon has the interesting model that the reason people share their problems with others is because we use other people for emotional regulation. They say that some answer like this is necessary because it's weird that we divulge things that make us look weak to others (as opposed to speech motivated by making ourselves look good) [1].

I read the book a while ago, but I do remember a motivating example of a small child who falls over and hurts themselves. They look to their caregiver to help interpret the significance of their pain. If the caregiver is not overly perturbed, the child then learns that their pain was not that big a deal. According to the authors of the book, adults sharing their woes are doing a similar thing. If I tell you about how times are tough at work and you maintain a visage that all will be okay in the end, I will find that reassuring.

They then further strongly assert that this how all successful psychotherapy works. They claim no matter the school of psychotherapy, what the successful psychotherapist is doing is being present with the patient while the patient experiences their emotions (the therapists not getting lost themselves in these emotions) and helping the patient shift their S1 reaction/emotionally regulate.

[1] I think there is something to be explained here, but we can come up with other hypotheses too. Like revealing my weaknesses/struggles to you might be valuable because it gives me reassurance that you'd still be my friend even if you found them out of my own accord. Perhaps also knowing all of each other's weaknesses results in some kind of mutually assured destruction so that you stay friends and don't go off badmouthing each other. Or something.

Comment by ruby on How can I reframe my study motivation? · 2019-09-24T23:49:03.053Z · score: 6 (3 votes) · LW · GW

Hi DivineMango, welcome!

Nice reading list. I'm not sure if this will quite help, but Nate Soares wrote a lot of good stuff on motivation on his blog, A couple of relevant posts (but there's many more there):

Though neither of those feel like they quite hit the thing on the head. Here are some further ideas which have helped me a little:

Seems like maybe you want to rediscover/get in touch with the inherent enjoyment of those activities separate from whether they lead to any state of completion. Extrinsic motivations seems kill intrinsic motivation (even when you otherwise had intrinsic motivation) so perhaps something to try is not worrying about how much you complete, just trying to enjoy the experience of doing however much if you're doing - and allowing yourself to stop at any point if you really don't want to continue (Soares-style).