LessWrong 2.0 Reader

I'll let you know. They're working on open-sourcing their scaffold at the moment.

No, that is not the definition of a prior. There are priors which imply an expected number of buses, and priors that don't. If you select a prior that doesn't, you can still get a meaningful posterior distribution even if that posterior distribution doesn't have a real-valued mean.

But I claim that all such priors make assumptions about the distribution of the possible number of buses

I mean, yes, that's the definition of a prior. How to calculate a prior is an old question in bayesianism, with different approaches - kolmogorov complexity being one.

Sorry, I meant to add in an example where for simplicity you saw the bus numbered 1.

Agreed it's a terrible prior, it's just an easy one for a worked example.

“How should we operationalize the concept ‘sex worker who is actively being sex trafficked’ for the purpose of determining how many such people there are, taking care that our operationalization successfully captures our moral intuitions related to this subject” is certainly an interesting question, and your answer is not prima facie unreasonable (although we would surely want to consider it carefully and with a critical eye, before applying it).

But I want to note that it is an almost wholly irrelevant question unless we intend to do our own data collection from scratch. For the purpose of determining the answer to our question by looking at existing data, what use is an operationalization like this if the data we would need to answer that operationalized question is not in our data sets?

In other words—it’s not like the existing studies that Aella cites involved the subjects being asked the question you propose, or anything like it. Or maybe some of them did! But we don’t know, because in many cases the data is from sources like this:

They cite a 2006 GAO report which says depressing things like “the U.S. government’s estimate was developed by one person who did not document all his work” and “There is also a considerable discrepancy between the numbers of observed and estimated victims of human trafficking”

… probably not, though.

Text adventures do seem like a good eval right now, since they're the ONLY games that can be tested without either relying on vision (which is still very bad), or writing a custom harness for each game (in which case your results depend heavily on the harness).

Hello! Here is a practical use https://www.lesswrong.com/posts/iByJhF2cLQmtTGFht/estimat-8-identities

You've arrived to the same conclusion as I state. I say caring about simulated minds explodes in paradixes in my thought experiment, so we probably shouldn't. You came to the same conclusion that caring aboout digital minds shouldn't be a priority through your introduced infinitesmal measure of digital minds. We're not in disagreement here.

No, that's not the posterior distribution--clearly, the number of buses cannot be lower than 1546, but that distribution has material probability mass on low integers. I'm not quite sure how you got that equation.

But regardless, I think this shows where we disagree. That prior has mean 2... that's a pretty strong assumption about the distribution of n. If you want to avoid that kind of assumption, you can get posterior distributions but not a posterior expectation.

Thanks everyone for all the feedback and answers to my unending questions! The branching comments are starting to become too much to handle, so I'm going to take a breather and then write a followup post – hopefully by the end of the week but we'll see – in which I'll share some consolidated thoughts on the new (to me) ideas that surfaced here and also respond to some specific points.