Posts
Comments
Isn't the counterfactual trolley problem setup backwards? It should be counterfactual Omega giving you the better setup (not tying people to the tracks) if it predicts you'll take the locally "worse" option in the actual case, not the other way around, right?
Because with the current setup you just don't pull and Omega doesn't tie people to tracks.
As an example of differentiating different kinds of footnotes, waitbutwhy.com uses different appearances for “interesting extra info” notes vs “citation” notes.
Both kinds also appear as popups when interacted with (certainly an advantage of the digital format).
Somehow I missed that bit.
That makes the situation better, but there’s still some issue. The refund is not earning interest, but you liabilities are.
Take the situation with owing $25 million. Say that there’s a one year time between the tax being assessed and your asset going to $0 (at which time you claim the refund). In this time the $25 million loan you took is accruing interest. Let’s say it does so at a 4% rate per year, when you get your $25 million refund you therefore have $26 million in loans.
So you still end up $1 million in debt due to “gains” that you were never able to realize.
Scenario: you have equity worth (say) $100 million in expectation, but of no realized value at the moment.
You are forced to pay unrealized gains tax on that amount, and so are now $25 million in the hole. Even if you avoid this crashing you immediately (such as by getting a loan), if your equity goes to $0 you’re still out for the $25 million you paid, with no assets to back it.
The fact that this could be counted as a prepayment for a hypothetical later unrealized gain doesn’t help you, you can’t actually get your money back.
But if UDT starts with a broad prior, it will probably not learn, because it will have some weird stuff in its prior which causes it to obey random imperatives from imaginary Lizards.
I don’t think this necessarily follows? For there to be a systematic impact on UDT’s behavior there would need to be more Lizard-Worlds that reward X than Anti-Lizard-Worlds that penalize X, so this is only a concern if there is reason to believe that there are “more” worlds (in an abstract logical-probability sense) that favor a specific direction.
Clearly this could still potentially cause problems, but (at least to me) it doesn’t seem like the problem is as ubiquitous as the essay makes it out to be.
My benchmark for thinking about the experience machine: imagine a universe where only one person and the stuff they interact with exist (with any other “people” they interact with being non-sapient simulations) and said person lives a fulfilling life. I maintain that such a universe has notable positive value, and that a person in an experience machine is in a similarly valuable situation to the above person (both being sole-moral-patients in a universe not causally impacting any other moral patients).
This does not preclude the possibility of improving on that life by e.g. interacting with actual sapient others. This view is fully compatible with non-experience-machine lives having much more value than experience-machine ones, but it’s a far cry from the experience-machine lives having zero value.
Also related: Yudkowsky on making Solvable Mysteries:
If you have not called upon your readers explicitly to halt and pay attention, they are already reading the next sentence. Even if you do explicitly ask them to pay attention, they are already reading the next sentence. If you have your character think, “Hm… there’s something funny about that story, I should stop and think about that?” guess what your reader does next? That’s right, your reader goes on to read the next sentence immediately, to see what the character thinks about it.
You can’t just trivially scale up the angular resolution by bolting more sensors together (or similar methods). It gets more difficult to engineer the lenses and sensors to meet super-high specs.
And aside from that, the problem behaves nonlinearly with the amount of atmosphere between you and the plane. Each bit of distortion in the air along the way will combine, potentially pretty harshly limiting how far away you can get any useful image. This may be able to be worked around with AI to reconstruct from highly distorted images, but it’s far from trivial on the face of it.
My guess is the largest contributor is the cultural shift to expecting much more involved parenting (example: the various areas where parents had CPS called on them for letting their kids do what the parents were allowed to do independently as kids)
Another big thing is that you can’t get tone-of-voice information via text. The way that someone says something may convey more to you than what they said, especially for some types of journalism.
I’d imagine that once we see the axis it will probably (~70%) have a reasonably clear meaning. Likely not as obvious as the left-right axis on Twitter but probably still interpretable.
I think a lot of the value that I’d get out of something like that being implemented would be getting an answer to “what is the biggest axis along which LW users vary” according to the algorithm. I am highly unsure about what the axis would even end up being.
To lay out some of the foundation of public choice theory:
We can model the members of an organization (such as the government) as being subject to the dynamics of natural selection. In particular, in a democracy elected officials are subject to selection whereby those who are better at getting votes can displace those who are worse at it, through elections.
This creates a selection dynamic where over time the elected officials will become better at vote-gathering, whether through conscious or unconscious adaptation by the officials to their circumstances, or simply through those who are naturally better at vote-gathering replacing those worse at it.
This is certainly not a bad thing per se. After all, coupling elected officials’ success to what the electorate wants is one of the major purposes of democracy, but “what gets votes” is not identical to “what’s good for the electorate”, and Goodhart’s law can bite us through that gap.
One of the classic examples of this is “doling out pork”, where concentrated benefits (such as construction contracts) can be distributed to a favored sub-group (thus ensuring their loyalty in upcoming elections) while the loss in efficiency from that favoritism is only indirectly and diffusely suffered by the rest of the electorate (making it much less likely that any of them get outraged about it enough to not vote for the pork-doler).
The application of this to market failures is that you can look at a market under government regulation as two systems (the market and the government), each with different incentives that imperfectly bind their constituent actors to the public good. The market generally encourages positive-sum trades to happen, but has various imperfections, especially regarding externalities and transaction costs, and the government generally encourages laws/regulations that benefit the public, but has its own imperfections, such as pork-doling and encouraging actions which look better to the public than their actual results would merit.
The result of this is that it is not necessarily clear whether whether changing how much influence market vs government dynamics have on a specific domain will improve it or not. Moving something to more government control may fix market failures, or it may just encourage good-looking-but-ineffective political posturing, and moving something to the market may cut down on corruption, or may just hit you with a bunch of not-properly-accounted-for externalities.
In the particular case of “government action to solve market failures”, the incentives may be against the government actors solving them, as in the case of the coal industry providing a loyal voting bloc, thereby encouraging coal subsidies that make the externality problem worse.
Therefore, my presentation of the market-failure-idea-skeptic’s position would be something like “we should be wary of moving the locus of control in such-and-such domains away from the market toward the government, because we expect that likely the situation will be made worse by doing so, whether due to government action exacerbating existing market failures more than it solves them, or due to other public-choice problems arising”.
Just because the US government contains agents that care about market failures, does not mean that it can be accurately modeled as itself being agentic and caring about market failures.
The more detailed argument would be public choice theory 101, about how the incentives that people in various parts of the government are faced with may or may not encourage market-failure-correcting behavior.
For chess in particular the piece-trading nature of the game also makes piece handicaps pretty huge in impact. Compare to shogi: in shogi having multiple non-pawn pieces handicapped can still be a moderate handicap, whereas multiple non-pawns in chess is basically a predestined loss unless there is a truly gargantuan skill difference.
I haven’t played many handicapped chess games, but my rough feel for it is that each successive “step” of handicap in chess is something like 3 times as impactful as the comparable shogi handicap. This makes chess handicaps harder to use as there’s much more risk of over- or under-shooting the appropriate handicap level and ending up with one side being highly likely to win.
Also note that socks with sandals being uncool is not a universal thing. For example, in Japan it is reasonably common to wear (often split-toed) socks with sandals, though it’s more associated with traditional garb than modern fashion.
A way of implementing the serving-vs-kitchen separation that avoids that problem (and actually the way of doing it I initially envisioned after reading the post) would be that within each workplace there is a separation, but different workplaces are split between the polarities of separation. That way any individual’s available options of workplace are, at worst, ~half of what they could be with mixed workplaces, regardless of their preference.
(Caveat that an individual’s options could end up being less than half the total if there is a workplace-gender correlation overall (creating an imbalance of how many workplaces of each polarity there are), and an individual has a workplace-gender matchup which is opposite to the trend, but in this case at least that individual’s lesser amount of choices is counterbalanced by the majority of people having more than 50% of the max choices of workplace fitting them.)
It kind of passed without much note in the post, but isn’t the passport non-renewal one of the biggest limiters here? $59,000 divided by 10 years is $5,900 per year, so unless you’re willing to forgo having a passport that’s the upper limit of how much you could benefit from non-payment (exclusive of the tax liability reduction strategies). That seems like a pretty low amount per year in exchange for having to research and plan this, then having your available income and saving methods limited (which could easily lower your income by more than $5,900 just by limiting the jobs available to you).
One other way of putting the reverse order, though it sounds a bit stilted in English: “beagles have Fido”. I don’t think it’s used commonly at all but it came to mind as a form in the reverse order without looping.
I would be interested in this, probably in role A (but depending on the pool of other players possibly one of the other roles; I have no opposition to any of them). I play chess casually with friends, and am probably at somewhere around 1300 elo (based on my winrate against one friend who plays online).
To add to this, if the ranked choice voting is implemented with a “no confidence” option (as it should to prevent the vote-in vote-out cycle described above), then you could easily end up in the same situation as the house currently is in, where no candidate manages to beat out “no confidence”.
SIA can be considered (IMO more naturally) as randomly sampling you from “observers in your epistemic situation”, so it’s not so much “increasing the prior” but rather “caring about the absolute number of observers in your epistemic situation” rather than “caring about the proportion of observers in your epistemic situation” as SSA does.
This has the same end result as “up-weighting the prior then using the proportion of observers in your epistemic situation”, but I find it to be much more intuitive than that, as the latter seems to me to be overly circuitous by multiplying by population then dividing by population (as part of taking the proportion of the reference class that you comprise), rather than just taking the number we care about (number of observers in your epistemic situation) in the first place.
Related: https://everythingstudies.com/2017/09/12/the-big-list-of-existing-things/
I think the point being made in the post is that there’s a ground-truth-of-the-matter as to what comprises Art-Following Discourse.
To move into a different frame which I feel may capture the distinction more clearly, the True Laws of Discourse are not socially constructed, but our norms (though they attempt to approximate the True Laws) are definitely socially constructed.
From the SIA viewpoint the anthropic update process is essentially just a prior and an update. You start with a prior on each hypothesis (possible universe) and then update by weighting each by how many observers in your epistemic situation each universe has.
This perspective sees the equalization of “anthropic probability mass” between possible universes prior to apportionment as an unnecessary distortion of the process: after all, “why would you give a hypothesis an artificial boost in likelihood just because it posits fewer observers than other hypotheses”.
Of course, this is just the flip side of what SSA sees as an unnecessary distortion in the other direction. “Why would you give a hypothesis an artificial boost due to positing more observers” it says. And here we get back to deep-seated differences in what people consider the intuitive way of doing things that underlie the whole disagreement over different anthropic methods.
On the question of how to modify your prior over possible universe+index combinations based on observer counts, the way that I like to think of the SSA vs SIA methods is that with SSA you are first apportioning probability mass to each possible universe, then dividing that up among possible observers within each universe, while with SIA you are directly apportioning among possible observers, irrespective of which possible universes they are in.
The numbers come out the same as considering it in the way you write in the post, but this way feels more intuitive to me (as a natural way of doing things, rather than “and then we add an arbitrary weighing to make the numbers come out right”) and maybe to others.
If you’re adding the salt after you turn on the burner then it doesn’t actually add to the heating+cooking time.
To steelman the anti-sex-for-rent case, it could be considered that after the tenant has entered into that arrangement, the tenant could feel pressure to keep having sex with the landlord (even if they would prefer not to and would not at that later point choose to enter the contract) due to the transfer cost of moving to a new home. (Though this also applies to monetary rent, the potential for threatening the boundaries of consent is generally seen as more harmful than threatening the boundaries of one’s budget)
This could also be used as a point of leverage by the landlord to e.g. pressure the tenant to engage in sex acts they would otherwise not want to or else be evicted (unless the contract specifies from the beginning exactly what kind of sex the payment will entail). I think many people would see such actions by the landlord as more of an infringement upon the tenant than e.g. raising the amount of monetary rent (sacredness of sex/consent).
Additionally, this could be seen as a specific manifestation of the modern trend of more general opposition to sexual relationships with a power imbalance between the participants.
(Parenthetically, I also want to thank you for writing this post, as it’s a good expression of a principle I generally agree with)
In terms of similarity between telling the truth and lying, think about how much of a change you would have to make to the mindset of a person at each level to get them to level 1 (truth)
Level 2: they’re already thinking about world models, you just need to get them cooperate with you in seeking the truth rather than trying to manipulate you.
Level 3: you need to get them the idea of words as having some sort of correspondence with the actual world, rather than just as floating tribal signifiers. After doing that, you still have to make sure that they are focusing on the truth of those words, like the level 2 case.
Level 4: the hardest of them all; you need to get them the idea of words having any sort of meaning in the first place, rather than just being certain patterns of mouth movements that one does when it feels like the right time to do so. After doing that, you again still have the whole problem of making sure that they focus on truth instead of manipulation or tribal identity.
For a more detailed treatment of this, see Zvi’s https://thezvi.wordpress.com/2020/09/07/the-four-children-of-the-seder-as-the-simulacra-levels/
Re: “best vs better”: claiming that something is the best can be a weaker claim than claiming that it it better than something else. Specifically, if two things are of equal quality (and not surpassed) then both are the best, but neither is better than the other.
Apocryphally, I’ve heard that certain types of goods are regarded by regulatory agencies as being of uniform quality, such that there’s not considered to be an objective basis for claiming that your brand is better than another. However, you can freely claim that yours is the best, as there is similarly no objective basis on which to prove that your product is inferior to another (as would be needed to show that it is not the best).
One other mechanism that would lead to the persistence of e.g. antibiotic resistance would be when the mutation that confers the resistance is not costly (e.g. a mutation which changes the shape of a protein targeted by an antibiotic to a different shape that, while equally functional, is not disrupted by the antibiotic). Note that I don’t actually know whether this mechanism is common in practice.
Thanks for writing this nice article. Also thanks for the “Qualia the Purple” recommendation. I’ve read it now and it really is great.
In the spirit of paying it forward, I can recommend https://imagakblog.wordpress.com/2018/07/18/suspended-in-dreams-on-the-mitakihara-loopline-a-nietzschean-reading-of-madoka-magica-rebellion-story/ as a nice analysis of themes in PMMM.
It seems like this might be double-counting uncertainty? Normal EV-type decision calculations already (should, at least) account for uncertainty about how our actions affect the future.
Adding explicit time-discounting seems like it would over-adjust in that regard, with the extra adjustment (time) just being an imperfect proxy for the first (uncertainty), when we only really care about the uncertainty to begin with.
Indeed humans are significantly non-aligned. In order for an ASI to be non-catastrophic, it would likely have to be substantially more aligned than humans are. This is probably less-than-impossible due to the fact that the AI can be built from the get-go to be aligned, rather than being a bunch of barely-coherent odds and ends thrown together by natural selection.
Of course, reaching that level of alignedness remains a very hard task, hence the whole AI alignment problem.
I had another thing planned for this week, but turned out I’d already written a version of it back in 2010
What is the post that this is referring to, and what prompted thinking of those particular ideas now?
I see it in a similar light to “would you rather have more or fewer cells in your body?”. If you made me choose I probably would rather have more, but only insofar as having fewer might be associated with certain bad things (e.g. losing a limb).
Correspondingly, I don’t care intrinsically about e.g. how much algae exists except insofar as that amount being too high or low might cause problems in things I actually care about (such as human lives).
Seeing the relative lack of pickup in terms of upvotes, I just want to thank you for putting this together. I’ve only read a couple of Dath Ilan posts, and this provided a nice coverage of the AI-in-Dath-Ilan concepts, many of the specifics of which I had not read previously.
My understanding of it is that there is conflict between different “types” of the mixed population based on e.g. skin lightness and which particular blend of ethnic groups makes up a person’s ancestry.
EDIT: my knowledge on this topic mostly concerns Mexico, but should still generally apply to Brazil.
That PDF seems like it is a part of a spoken presentation (it’s rather abbreviated for a standalone thing). Does there exist such a presentation? If so, I was not successful in funding it, and would appreciate it if you could point it out.
I similarly offer myself as an author, in either the dungeon master or player role. I could possibly get involved in the management or technical side of things, but would likely not be effective in heading a project (for similar reasons to Brangus), and do not have practical experience in machine learning.
I am best reached through direct message or comment reply here on Lesswrong, and can provide other contact information if someone wants to work with me.
The main post of what amounts of evidence different tests give is this one: https://www.lesswrong.com/posts/cEohkb9mqbc3JwSLW/how-much-should-you-update-on-a-covid-test-result
Also related is part of this post from Zvi (specifically the section starting “Michael Mena”): https://www.lesswrong.com/posts/CoZitvxi2ru9ehypC/covid-9-9-passing-the-peak
Combining the information from the two, it seems like insofar as you care about infectivity rather than the person having dead virus RNA still in their body, the actual amount of evidence from rapid antigen tests will be higher than the amounts given in the first post. There’s a good case to be made that the sensitivity with respect to infectiousness would be 95% or more, though I am not aware of any research directly addressing this question.
This is a good piece of writing. It reminds me of another piece of fiction (somewhat happier in tone) which I cannot find again. The plot involves a woman trying to rescue her boyfriend from a nemesis in a similar AI-managed world. I think it involves her jumping out of a plane, and landing in the garden of someone who eschews AI-protection for his garden, rendering it vulnerable to destruction without his consent. Does anyone recall the name/location of this story?
Copyediting: “Miriam removed off her cornea too” should probably not have the “off”.
The part about hiring proofreading brought a question to mind: where does the operating budget for the lesswrong website come from, both for stuff like that and standard server costs?
Do you have any recommendations of such stories?
If you also consider the indirect deaths due to the collapse of civilization, I would say that 95% lies within the realm of reason. You don’t need anywhere close to 95% of the population to be fully affected by the scissor to bring about 95% destruction.
Sorry if I was ambiguous in my remark. The comparison that I’m musing about is between “fierce” vs “not fierce” nerds, with no particular consideration of those who are not nerds in the first place.
It’s interesting to read posts like this and “Fierce Nerds” while myself being much less ambitious/fierce/driven than the objects of said essays. I wonder what other psychological traits are associated with the difference between those who are more vs less ambitious/fierce/driven, other things being equal.
Nice poem! It’s cool to see philosophical and mathematical concepts expressed through elegant language, though it it somewhat less common, due to the divergence of interests and skills.