From research I did on New Zealand: 14 days isn't long enough if people are isolating together- live infections get passed to housemates partway through, who are still contagious when the 14 days are up. So either you have to truly isolate everyone, or extend it by some number of days for each additional person.

Also, I assume you mean "14 days, unless they test positive or show symptoms", but then you have to figure out how to test and verify symptoms for everyone.

A bunch of people have alluded to Decision Tree being "non-representative" in ways I worried people might interpret as "unusually bad" rather than "different by design in ways that predictably made things harder". I also think there are some lessons from the Decision Tree experiment that aren't currently obvious and could save people a lot of heartache. So I got permission from the organizers to share more of the story.

There was a group house in the bay that had a history of taking in strays (more than one in fact, but we're only talking about one). They would get to know people, usually online, with job loss, needing to escape abuse, or just wanting to move to CA and needing some help to get some feet under them and give them a few months couch time and food. It worked in part because the hosts were really good at boundaries and so really could limit themselves to just providing couch, food, and an amount of emotional support they were truly comfortable with. I don't know the whole roster, but I know at least four people this house took in who were much better off for it, and no one who was made worse off.

But they only had one couch, and it took months to get people on their feet, and so many people needed help. So with the help of a benefactor they decided to create an entire house dedicated to giving people this launching pad. That was Decision Tree. The plan was to subsidize rent and provide various RA/RD types to help people get on track.

Unfortunately this meant that you went from a house full of fully functional people + one rescue person, to a house full of rescues (with the people who became functional fastest leaving soonest).  There are a million reasons someone might need a place to crash for a few months. An unfortunate number of them are also reasons someone might be a difficult roommate. People who have only ever lived in dysfunctional situations, or are coming off a major trauma, are on average either harder to live or find living with other people harder. The subsidized rent meant that some portion of people genuinely couldn't afford to move, which could force housemates to stay together where richer people would have moved apart. 

So that's why I think Decision Tree is not a great data point for group houses as a whole. 

John and I had a fantastic offline discussion and I'm currently revising this in light of that. We're also working on a postmortem on the whole thing that I expect to be very informative. I keep mission creeping on my edits and response and it's going to take a while so I'm writing the bare minimum comment to register that this is happening.

I looked into the success of different countries' quarantines. New Zealand had both the best implementation and the best data, so I draw most of my conclusions from them. With a 14 day quarantine (testing on day 3 and 12), New Zealand had a "barely visible on the graph" number of import-adjacent infections. But according to a statistical model, these are caused by infections caught during quarantine (i.e. a couple with one infected member quarantines together, the second member catches it on day 7,  leaves after an effective quarantine of only 7 days, then spreads it in the wild), so if people are completely isolated and occasionally tested, 14 days in indeed sufficient

I agree that Decision Tree was non-representative by design (in ways I'm not sure are public), in ways that will make it perform worse on average. I think that should have been noted more explicitly. I also think deluks is being really brave in naming something that made them a worse person, and I'm grateful they provided that data point.

I've been thinking a lot about this comment, and wanted to think more, but it seems useful to have something up as voting starts, so....

I think there's A Thing JW both agree is harmful (around assigning people moral responsibility when they're responding to incentives), and that I was trying to fight against. One thing I took from this comment is there's a good chance I had only a partial victory against Harmful Thing, and tried to pull down the master's house with the master's tools. I'd be very interested in exploring that further. (I also think it's possible JW is doing the same thing... it's a hard trap to escape)

I don't think giving up the question "Who should we blame?" entirely is a good idea. Possibly the benefits of the norm would outweigh the costs for LessWrong in particular, but I don't believe such a norm would be a pareto improvement. 

People often assume he's all about frugality. In reality, he thinks that some things are worth spending good money on.

From my limited reading of MMM, he's pretty intolerant/dismissive of people who have different beliefs about what is worth spending money on, which is a pet peeve of mine. 

I'm confused why this is a response to my comment, which was not an argument for or against the dodo bird verdict but about definitions.

Oh man, I wish you'd come in under the deadline.

For people who don't feel like clicking: it's a quantification of behavior predicted by different scores on Big-5.

Winner

Last week we announced a prize for the best example of an evaluation. The winner of the evaluations prize is David Manheim, for his detailed suggestions on quantitative measures in psychology.  I selected this answer because, although IAT was already on my list, David provided novel information about multiple tests that saved me a lot of work in evaluating them. David has had involvement with QURI (which funded this work) in the past and may again in the future, so this feels a little awkward, but ultimately it was the best suggestion so it didn’t feel right to take the prize away from him.

EDIT: David has elected to have the prize donated to GiveWell.

Honorable mentions to Orborde on financial stress tests, which was a very relevant suggestion that I was unfortunately already familiar with, and alexrjl on rock climbing route grades, which I would never have thought of in a million years but has less transferability to the kinds of things we want to evaluate.

Post-Mortem

How useful was this prize? I think running the contest was more useful than $50 of my time, however it was not as useful as it could have been because the target moved after we announced the contest. I went from writing about evaluations as a whole to specifically evaluations that worked, and I’m sure if I’d asked for examples of that they would have been provided. So possibly I should have waited to refine my question before asking for examples. On the other hand, the project was refined in part by looking at a wide array of examples (generated here and elsewhere), and it might have taken longer to hone in on a specific facet without the contest.

Other timeless but year-of-publication restricted anthologies like "Year's Best Dark Fantasy and Horror" and  "Year's Best Science Writing"  have either "Nth annual" or [year the entries were published] prominently on the title.  This is an established convention. The problems of "what the hell book did I read that in?", "Finding the books on Amazon" and "Have I read this already? Who's to say." seem much bigger to me than a fraction of the audience that hasn't picked up that convention AND will be blocked from reading by it.

Have moved to comments. Thank you both for the feedback.

My risk model treats all the available vaccines as "drug that was developed under political and financial pressure and whose trials ended much sooner than is normally the case".

I think we can do better than that, even in the current information climate. Drugs can be almost anything and have almost any goal. Vaccines are a pretty narrow class of treatment that are attempting to do one thing- give you an immune memory that will trigger if infected by a disease. They do that by exposing you to some part of the disease. The natural cap of badness of that attempt is giving you the disease itself- anything more needs an explanation.

Some examples off the top of my head where a vaccine caused effects you wouldn't get from the disease itself (not all of which rendered the vaccine net negative):

1. certain adjuvants encouraged cancer (e.g. some pet vaccine)
2. the vaccine triggered an immune overreaction that left people worse off if actually infected (there was an STD vaccine that did this, and I believe SARS-1)
3. Allergy to something else in the vaccine (e.g. eggs in the flu vaccine).

Of these, we'd expect #3 to show up nigh immediately upon immunization, and is not dependent on how many people were exposed to actual covid, so we have a fairly large sample size.  I vaguely recollect that where #2 was a factor, it was pretty universally true, not a rare reaction- so the sample size is probably large enough for that too. 

This is complicated by the fact that at least one of the covid vaccines is using an entirely new mechanism. This could leave us vulnerable to certain problems like the adjuvants, that take a long time and large sample size to catch. 

Certainly some people are very sensitive and medically excused from vaccines- but those people are pretty screwed if they catch the actual disease too. The only reason not getting the vaccine is viable for them is herd immunity.

I'm not an expert and I haven't looked into this very long. But there are mechanistic models that can help us predict the risk here, and I think it's a mistake to use the entire collection of FDA-monitored treatments as a reference class.

Eli was in fact checking in while I babbled, in retrospect he was putting out feelers for redirection but I was so happy with how generative I was being I wasn't that responsive, and I think he was reluctant to push back because what if strict separation of babble and prune was the best process? It also might not have been obvious how much transformation my babbled ideas needed to be usable until we did the next step.

Follow up: @elityre asked to discuss doublecrux ideas and I spent a little more than an hour spewing forth topics. There were 74 in the first 30 minutes and its slowed down from there. I don't think I would have done that nearly as easily without the babble challenges.

It turns out these were answers to the slightly wrong question, so we had to do a prune pass. I'm not sure if it would have been better to clarify first, or if the absolute babble freedom ended up  leading to better post-pruning ideas.

What's the model under which the vaccine is more dangerous than the virus it's based on? That's not unknown, but it would be quite weird.

I wonder what could be done to make commenting more rewarding for people.

My understanding is that flushable is not a legally protected term, and many things labeled flushable will clog your pipes or the sewage system. Any thoughts on this?

I can't say it's impossible, but you would have to constantly go back and forth whenever you got new information, while hiding from yourself that you were doing so, and at a certain point it becomes psychologically easier to work hard despite a lack of guaranteed success.

I agree the post strongly frames things as for potential winners who aren't considering other options, but I think that's a mistake for this exact reason. Even if a set that benefits from self-deceit exists, you can't know if you're in it while you're self-deceiving.

Yes, oxytocin and other hormones released during hugs/sex have a similar effect to drugs. Actually, most drugs don’t add anything into your body but will simply increase/activate or affect hormones naturally present, which will give you an altered state of mind

This doesn't seem surprising or compelling to me. People say similar things about morphine and sugar, but of course drugs take advantage of existing systems that evolved (imperfectly) to help you: why would you evolve a neurochemical system set up for the sole purpose of fucking up your life if you took the wrong chemical? The surprising thing would be if drugs used a system that didn't have a normal, survival-aiding purpose.

• Regardless of what the truth is, it seems like working hard is the best way forward, rather than getting lazy or seeking side-hustles.

This isn't obvious to me. If you are in, say, the bottom 90% and have no chance of getting an academic job no matter how hard you work, knowing as soon as possible and making exit plans seems like the best thing.

Filler text for preview

Alternate explanation: people who are capable of believing they are good at a thing will eventually stop trying to improve, while people who are stuck perceiving themselves as unacceptably bad at X will continue working to get better.

If the interventions have aggregate impact distinct from individual impact, they're by definition not identical

Why not? I don't see how that follows.

VALUE OF INFORMATION 

Say I have the option to purchase a lottery ticket with a 1/10m chance of winning, a $1m guaranteed jackpot, and a $1 ticket price. The EV of a ticket is $1m/10m - $1 = -$0.9, a bad deal. What is the value of information that narrows the odds?

If it narrowed the odds all the way to 1/1, the value would be $999,999, obviously a very good deal. I should be willing to pay up to almost $999,999 for it.

But if the odds are doubled? The EV of a purchase is now $1/5-$1 = -$0.80. In one sense I’ve gained $0.1. But since buying a lottery ticket is still EV < 0, I won’t purchase one, and so the information nets me no money. The information is worth $0 to me, as long as I have the option not to buy a ticket.

But… maybe there are multiple interventions that double my odds? Under the above rule, the first three are worth nothing and the fourth is worth $2^4/10 - $1 = $0.6. It seems wrong that 4 identical interventions would have such different values. I could just divide .6  by 4, but what if I’m not certain how many doublings will be available?

Filler text for preview purposes....

I reject your reality and substitute my own.

More constructively, I am confused as to the model behind rules. There are two main confusions:

• Rejecting variants requires pruning, and this is purely a babble challenge, so why don't variants count? I think coaxing variants out is pretty integral to getting my brain in a babble mode.
• Why do some things that clearly won't work (bird in a space suit) count while others (watching Tiger King) don't?

I’m Elizabeth.You may remember me from such series as Epistemic Spot Checks and the LessWrong Covid Effort, but for the last year my main focus has been developing a method for Knowledge Bootstrapping- going from 0 to 1 in an unfamiliar field without undue deference to credentialism. I’m at the stage where I have a system that works well for me, and I’ve gotten feedback from a few other people about what works and doesn’t work for them, but there’s a long way to go. A lot of my knowledge is implicit and not explained on the page, plus I am only one person; what works for me will not translate perfectly for every human. So I’m looking for test subjects.

One particular part of my method is breaking down one large question into many smaller questions. This has several purposes: it forces you to clarify what you actually care about, and makes it more obvious what information is relevant. I describe this process and the reasoning behind it here, but not very well.  I’m looking for test subjects that have a research question, and would like to practice breaking it down into smaller questions, with the goal of refining the technique and my teaching of it.

What This Looks Like

1. Come up with a question you might like to research.
2. You book a phone call with me via calendly, or email me at elizabeth -at- acesounderglass.com to set up a time. 
3. We discuss your question in an attempt to break it down into smaller parts.
4. I sure hope some people actually go off and research the new questions but there’s no commitment required to do so.

What Are the Expected Outcomes?

• You will have a better understanding of what you actually want to know and will be better positioned to find answers.
• You will be better able to break down your next research question, without me.
• I will make some of my metis on breaking down questions more explicit.
• I will become better at teaching the technique of breaking down questions.
• I learn techniques from you I couldn’t have learned on my own.

6. Draw ideas instead of writing them down in words, so there's more time to think between ideas, a higher barrier to committing them to paper, and you'll get bored with very similar ones faster.

Do you write in Roam using a phone?

Yes, but not as part of this workflow, because Roam doesn't support offline yet and not being online is pretty critical. I will sometimes move things I write as part of this workflow into Roam after the fact.

 Do you read literature sources on it as well?

Not sure what "it" refers to or what you mean by "literature sources". If you mean "do I do the kind of research I frequently talk about elsewhere using this workflow?" the answer is no.

Wikipedia defines the dodo bird verdict as "the claim that all empirically validated psychotherapies, regardless of their specific components, produce equivalent outcomes", which is noticeably different than "talking to a friend produces equivalent outcomes to therapy". They can both be true, but I don't think they're the same thing.

Similar definitions from study.com, Scientific American, some journal

I got nervous when I heard people were applying for forgiveness, so I looked into it. Here's what I found

• Some banks are accepting forgiveness applications right now, my bank isn't and is working on their own submission process. I discovered this by googling "[my bank] PPP loan forgiveness".
• The absolute drop dead deadline before you will be charged interest is 10 months after the period the loan covers (source)
• Sample application and instructions here

Space Chem is a work of art in the amount of gameplay it gets out of a ridiculously simple mechanic (it is also fun).

QUBE is a totally adequate member of the genre.

Additional factor: availability of compatible people who didn't move there for this group. This is important for several reasons, including:

• Inflow of new ideas
• Ease of joining and leaving. If rationalists take over a small town, the only thing there for them is other rationalists. That makes joining and leaving into very binary decisions. It doesn't let people slowly notice incompatibilities and amoeba into another social group.  

Is there any policy for a refund if someone drops out sufficiently early?

A link can be both affiliate and smile, they stack.

There's a plug in that will look for PDFs for you that match the page you're on or the text you have highlighted.

This sure seems like it should work. My experience is that there's either nothing, or whatever quality analyses exist are drowned out by pap reviews (it is possible I should tolerate reading more pap reviews in order to find the gems). However I think you're right that for issues that have an academic presence, google scholar will return good results.

My experience is that readability doesn't translate much to quality and might even be negatively correlated, because reality is messy and simplifications are easier to read. I do think works that make themselves easy to double check are probably higher quality on average, but haven't rigorously tested this.

I originally named the types of knowledge "Type 1", "Type 2", and "Type 3", but was encouraged by early reviewers to actually name them. In light of the conversation here, I think doing that was a mistake. Unless I was sure my names were out of the park correct (and maybe not even then), I should have left it generic so I could get input on more people for what the names, and for that matter definitions, should be.

A thing I really structured to capture was that "i did actual research and had actual models for why masks would help against covid, but it's still not type-3", which is why "know why" doesn't feel right to me.

I tentatively think that some of what you're calling engineering knowledge would fit into what I call scientific (which is a strike agains the names), and/or that I didn't do a good enough job explaining why engineering knowledge is useful.

Ignoring the labels I put on them, do you feel like you have a good sense of what I mean by each kind of knowledge? if so, what would you label them?

Welp, I did not make that deadline. Unfortunately the conditions that led me and the LW team to miss that deadline- high opportunity costs- are not likely to change soon, so instead of holding out for perfection I'm just going to share a couple of thoughts.

I was brought on to lead covid research efforts at LW as an experiment. The hope was that there was significant untapped research capacity, which could be unlocked by providing some structure (hence the research agenda). The structure was not only supposed to give people a sense of what would be useful to research, but reassurance that their research would actually be used, and social reinforcement. This mostly did not pan out- I think I did useful research during the time in question, I think other people produced useful research during that time, but questions I asked tended to be answered by only me.

The experiment was well worth running, and the team got a lot of information on infrastructure useful to support coordinated research (most notably it led to some reworks of Questions). But after 6 weeks it was not achieving its stated goal and had not found something clearly high value to pivot to, so I called it.

I think "prevent" is a strong word but broadly agree that my original approach was not conducive to deep model building. You can read more about how my thoughts have changed over time here.

Ah, I have only read Deep Work.

Cal Newport was specifically who I was thinking of when I said "people who don't understand how useful social media is to me." And I still think he doesn't, I'm just now more aware of the cost as well.