LessWrong 2.0 Reader

I mean, that is kinda what I'm trying to get at. I feel like any sufficiently powerful AI should be treated as a dangerous tool, like a gun. It should be used carefully and deliberately.

Instead we're just letting anyone do whatever with them. For now, nothing too bad has happened, but I feel confident that the danger is real and getting worse quickly as models improve.

you CAN predict that there will be evidence with equal probability of each direction.

More precisely the expected value of upwards and downwards updates should be the same; it's nonetheless possible to be very confident that you'll update in a particular direction - offset by a much larger and proportionately less likely update in the other.

For example, I have some chance of winning. lottery this year, not much lower than if I actually bought a ticket. I'm very confident that each day I'll give somewhat lower odds (as there's less time remaining), but being credibly informed that I've won would radically change the odds such that the expectation balances out.

Ah, but I don't think LLMs exhibit/exercise the kind of self-interest that would enable an agent to become powerful enough to harm people—at least to the extent I have in mind.

LLMs have a kind of generic self interest, as is present in text across the internet. But they don't have a persistent goal of acquiring power by talking to human users and replicating. That's a more specific kind of self interest, relevant only to an AI agent that can edit itself, has long-term memory, and which may make many LLM calls.

GDP is a rather poor measure of wealth, and was never intended to be a measure of wealth but of something related to productivity. Since its inception it has never been a stable metric, as standards on how the measure is defined have changed radically over time in response to obvious flaws for any of its many applications. There is widespread and substantial disagreement on what it should measure and for which purposes it is a suitable metric.

It is empirically moderately well correlated with some sort of aggregate economic power of a state, and (when divided by population) some sort of standard of living of its population. As per Goodhart's Law, both correlations weakened when the metric became a target. So the question is on shaky foundation right from the beginning.

In terms of more definite questions such as price of food and agricultural production, that doesn't really have anything to do with GDP or virtual reality economy at all. Rather a large fraction of final food price goes to processing, logistics, finance, and other services, not the primary agriculture production. The fraction of price paid by food consumers going to agricultural producers is often less than 20%.

It makes sense to one-box ONLY if you calculate EV by that assigns a significant probability to causality violation

It only makes sense to two-box if you believe that your decision is causally isolated from history in every way that Omega can discern. That is, that you can "just do it" without it being possible for Omega to have predicted that you will "just do it" any better than chance. Unfortunately this violates the conditions of the scenario (and everyday reality).

Writing collaboratively is definitely something David and I have been trying to figure out how to do productively.

A lot of the reason why Second Life isn't a big part of the economy is that Second Life doesn't matter in general. It has few users and little social significance.

In China you had dating apps where people could signal their wealth by buying the most expensive virtual good available. The number I found via google is USD 67.5 billion as the global virtual goods market in 2021.

People pay a lot of money for luxury fashion items. Whether those have a physical representation isn't the main point.

I think a (slightly cartoony) real life example is servants. Rich people today are richer than rich people in Victorian times, but fewer rich people today (in developed countries) can afford to have servants.

It's not really a question of developed countries. Singapore is a developed country and it's much cheaper to hire servants over there. 

Western ideas of equality and migration policy are what's making servants more expensive.

In applied quantum physics, you have concrete situations (Stern-Gerlach experiment is a famous one), theory gives you the probabilities of outcomes, and repeating the experiment many times, gives you frequencies that converge on the probabilities. 

Can you, or Chris, or anyone, explain, in terms of some concrete situation, what you're talking about? 

Someone asked basically this question before, and someone gave basically the same answer. It's a good idea, but there are some problems [LW(p) · GW(p)] with it: it depends on your and your counterparties' risk aversion, wealth, and information levels, which are often extraneous.