Traversable wormholes, were they to exist for any length of time, would act as electric and gravitational Faraday cages, i.e. attenuate non-normal electric and gravitational field exponentially inside their throats with the scale parameter of the mouth size/throat circumference. Consequently, the electric/gravitational field around them is non-conservative. This follows straightforwardly from solving the Laplace equation, but never discussed in the literature as far as I can find.
Smoking lesion is an interesting problem in that it's really not that well defined. If an FDT agent is making the decision, then its reference class should be other FDT agents, so all agents in the same class make the same decision, contrary to the lesion.
Wha...? Isn't like saying that Newcomb's is not well defined? In the smoking lesion problem there is only one decision that gives you highest expected utility, no?
Right, sorry, I let my frustration get the best of me. I possibly am misinterpreting the FDT paper, though I am not sure where and how.
To answer your question, yes, obviously desire to smoke is correlated with the increased chance of cancer, through the common cause. If those without the lesion got utility from smoking (contrary to what the FDT paper stipulates), then the columns 3,4 and 7,8 would become relevant, definitely. We can then assign the probabilities and utilities as appropriate. What is the formulation of the problem that you have in mind?
I... do not follow. Unlike the FDT paper, I try to write out every assumption. I certainly may have missed something, but it is not clear to me what. Can you point out something specific? I have explained the missing $1000 checkup cost: it has no bearing on decision making because the cosmic ray strike making one somehow do the opposite of what they intended and hence go and get examined can happen with equal (if small) probability whether they take $1 or $100. If the cosmic ray strikes only those who take $100, or if those who take $100 while intending to take $1 do not bother with the checkup, this can certainly be included in the calculations.
The problem is usually set up so that they gain utility from smoking, but choose not to smoke.
Well, I went by the setup presented in the FDT paper (which is terrifyingly vague in most of the examples while purporting to be mathematically precise), and it clearly says that only those with the lesion love smoking. Again, if the setup is different, the numbers would be different.
In any case, you seem to have ignored the part of the problem where smoking increases chance of the lesion and hence cancer. So there seems to be some implicit normalisation? What's your exact process there?
Smoking does not increase the chances of the lesion in this setup! From the FDT paper:
an arterial lesion that causes those afflicted with it to love smoking and also (99% of the time) causes them to develop lung cancer. There is no direct causal link between smoking and lung cancer.
do the opposite of what she would have done otherwise
might mean, and came up with
Deciding and attempting to do X, but ending up doing the opposite of X and realizing it after the fact.
which does not depend on which decision is made, and so the checkup cost has no bearing on the decision. Again, if you want to specify the problem differently but still precisely (as in, where it is possible to write an algorithm that would unambiguously calculate expected utilities given the inputs), by all means, do, we can apply the same approach to your favorite setup.
I agree that some of the rows are there just for completeness (they are possible worlds after all), not because they are interesting in the problem setup. How is the problem normally interpreted? The description in your link is underspecified.
But you're also giving everyone without the lesion 0 utility for not having cancer?
Good point. Should have put the missing million there. Wouldn't have made a difference in this setup, since the agent would not consider taking up smoking if they have no lesion, but in a different setup, where smoking brings pleasure to those without the lesion, and the probability of the lesion is specified, the probabilities and utilities for each possible world are to be evaluated accordingly.
Still super confused why you are trying this convoluted and complicated approach, or mixing this problem with Newcomb's, while the useful calculation is very straightforward. To quote from my old post here:
An agent is debating whether or not to smoke. She knows that smoking is correlated with an invariably fatal variety of lung cancer, but the correlation is (in this imaginary world) entirely due to a common cause: an arterial lesion that causes those afflicted with it to love smoking and also (99% of the time) causes them to develop lung cancer. There is no direct causal link between smoking and lung cancer. Agents without this lesion contract lung cancer only 1% of the time, and an agent can neither directly observe, nor control whether she suffers from the lesion. The agent gains utility equivalent to $1,000 by smoking (regardless of whether she dies soon), and gains utility equivalent to $1,000,000 if she doesn’t die of cancer. Should she smoke, or refrain?
There are 8 possible worlds here, with different utilities and probabilities:
An agent who "decides" to smoke has higher expected utility than the one who decides not to, and this "decision" lets us learn which of the 4 possible worlds could be actual, and eventually when she gets the test results we learn which one is the actual world.
Note that the analysis would be exactly the same if there was a “direct causal link between desire for smoking and lung cancer”, without any “arterial lesion”. In the problem as stated there is no way to distinguish between the two, since there are no other observable consequences of the lesion. There is 99% correlation between the desire to smoke and and cancer, and that’s the only thing that matters. Whether there is a “common cause” or cancer causes the desire to smoke, or desire to smoke causes cancer is irrelevant in this setup. It may become relevant if there were a way to affect this correlation, say, by curing the lesion, but it is not in the problem as stated.
To repeat the intuitive idea: an abstract model throws away or ignores information from the concrete model, but in such a way that we can still make reliable predictions about some aspects of the underlying system.
That's not quite my understanding what an abstraction is. What you have described is basic modeling. An abstraction is a model that works well in a large class of disparate domains, like polymorphism in programming. The idea of addition is such an abstraction, it works equally well for numbers, strings, sheep, etc. What you call a natural abstraction is closer to my intuitive understanding of the concept. I do not subscribe to your assertion that this is a "property of the territory". Sheep are not like bit strings. Anyhow, ideas are in the mind, and different sets of ideas can be useful for predicting different sets of observations of seemingly unrelated parts of the territory. Also, good luck with your research, whatever definition of abstraction you use, as long as it is useful to you.
Looks like we are mostly on the same page. Your example is worth looking into a bit.
Someone argues with you that graffiti is harmful for the community. You fight vehemently to defend graffiti, because deep down you know that without graffiti, your group of friends wouldn't exist, and then you'd be alone
Right, if you identify as belonging to that group, you would feel that you have no choice but to defend it. If you were to consider the graffiti group as a place to socialize instead, then your reaction to someone attacking graffiti artists would still make you think that they are ignorant of the subculture and you would engage them without feeling endangered and getting defensive, which tends to be more productive. In this case your personal identity could be as in your description
"Be funny, act unflappable, be competent at the basic stuff"
or something similar that lets you pick and choose what groups you meet your social needs in without getting stuck in their world and having to bend your identity to fit into theirs.
To me KYIS is about group identity, not so much about personal identity. So the actionable advice would be "Does this particular part of your identity come from belonging to a group, wanting to belong to one? Then discard it."
The development of advanced AI increases the risk of human extinction (by a non-trivial amount, e.g. 1%)
This is where I call BS. Even the best calibrated people are not accurate at the margins. They probably cannot tell 1% from 0.1%. The rest of us can't reliably tell 1% from 0.00001% or from 10%. If you are in doubt, ask those who self-calibrate all the time and are good at it (Eliezer? Scott? Anna? gwern?) how accurate their 1% predictions are.
Also notice your motivated cognition. You are not trying to figure out whether your views are justified, but how to convince those ignorant others that your views are correct.
think we can construct a meaningful (but non-unique) notion of counterfactuals in the map
Quite likely, and you have been working on it for some time. But is it a useful direction to work in? Every time I read what people write about it, I get the impression that they end up more confused than when they had started.
Counterfactuals are in the mind, so of course they depend on the mental models, including "social conventions". They are also a bad model of the actual world, because they tempt you to argue with reality. (Here I assume a realist position.) There is only one world. There is no what could have been, only what was, is and may yet be. And that "may" is in the mind, not in the territory. Being an embedded agent, you cannot change the world, only learn more about which of your maps, if any, are more accurate. That's why a's and b's in your examples are identical, only some sound more confused than others. There is no difference between an opaque and a transparent Newcomb's.
This post came across to me as mostly speculative but trying to be academic, I may well be wrong. Habryka in the other comment suggested that your claims have some grounding that I was not aware of. Additionally, I do not subscribe to the local lore of Eliezer's contrarianism and extreme Bayesianism. The metaphor of "reality joints", or "reality fluid", falls flat for me, as well. If you perspective is different, then feel free to disregard my comment, it's not like you and I can square our epistemic views in a comment thread.
I'm confused, you seem to describe a very elaborate model of cognition, yet I can find no literature review, no testable predictions and no experimental results to ground this model in something observable. What is this model based on?
I wonder if the claim of extreme bright light alleviating the symptoms of SAD has been tested by more than one person. While efficacy is not correlated with popularity, given that this is a rational-thinking based forum, it would be at least nice to know if the device works.
Your post starts with a questionable statement and is heavy on poorly motivated emotional shoulds and light on anything that can be charitably called "research". How do you define a food crisis? Which food crises in the past are you comparing it to? How were they resolved? Have you looked at the food availability issues around the world and the various ways they have been or are being (mis-)handled?
Social justice does not need steelmanning. The wikipedia definition makes perfect sense as is: "Social justice is a concept of fair and just relations between the individual and society." I am not sure what you are trying to steelman (or, from the tone of your post, dismiss), but it is something else, unrelated to social justice except maybe in some loose way.
Of course, if your chipped card is stolen and pinless tap is supported for small purchases, no transaction verification helps you to avoid fraudulent charges. And if someone has your card and knows your pin (which still can be skimmed with card skimmers), then you are likely on the hook for all transactions up until you report your card stolen.
I no longer own a car, just use a ride share like car2go when I need to get somewhere. Also, I hate owning things. Renting what I need for when I need it and letting someone else deal with the nitty-gritty of maintenance frees my time and my mind. And I am not alone. Most corporate compute is rented. Most commercial space is rented. Most offices are rented. That most houses (in the US) are owned is an artifact of the way thing came to be. If land had no intrinsic value, and renting was a universally more economical option, then real estate ownership would drop dramatically over time.
that a single product, a single vehicle needs to do both the moving and the accommodation of the user
That's a good point. Short trips only need moving, and longer travel needs to include accommodation, though not necessarily of a house-type kind. So, there is some room for what you propose, but probably not for the daily commute, where splitting the tug from the cabin is not necessarily economical. Still, I can see that standardized tugs might be economically viable in some niche cases.
How likely is it that your question has not already been asked, answered, indexed and on the first page of google search with a reasonable query? My guess is very unlikely. So a better question might be, and it's something you could ask here, "I'm interested in understanding X, but not sure how to find reliable information about it, what search terms should I try?" The relevant part of stack overflow could also be a good candidate for such a question. Even at their unfriendliest, they might close your question as a duplicate or off-topic, but still provide a link to the similar questions asked.
Note that cognition comes with no goals attached. Emotions is where the drive to do anything comes from. You allude to it in "contemplation of a possible choice or outcome". My suspicion is that having a component not available to accurate introspection is essential to functioning. An agent (well, an algorithm) that can completely model itself (or, rather, model its identical copy, to mitigate the usual decision theory paradoxes), may end up in a state where there is no intrinsic meaning to anything, a nirvana of sorts, and no reason to act.
First, really like your post! I've done online emotional support for some years, and I could clearly see how people end up in these self-blame and self-punishment loops with little to no improvement. Someone called it "shoulding oneself in the foot." The common denominator is people trying to argue with the past, and to change the past, or to punish themselves (or others) for the past transgressions, whether "real" or not. What gets lost is trying to affect the present with the goal to improve the potential future. Arguably, self-blame is an easy and tempting way out, as it lets one avoid acting in the present, and just keep the self-punishment going. It also saps the energy out of you, creating a vicious circle hard to break out of. If you try to point out the loop, and ask if they are interesting in working on improving the future, they (we) tend to find/invent/create a lot of reasons why self-judgment, self-blame and self-punishment is the only way to go.
Note that for a simulation to be useful, it has to be as faithful as possible, so SimDave would not be given any clues that he is simulated.
Comment by shminux on [deleted post]
There are many many ways something bad can happen. It is natural to focus on something you find especially scary and that your mind insists is a real possibility. Most of the time, it is you privileging a hypothesis through motivated emotional cognition. When this thought pattern goes into overdrive, one can end up with a real mental illness, not an imagined one, an anxiety disorder based on a specific if unfounded fear. Consider examining your motivations, emotions and thought patterns and figure out why you single out this specific scenario over all others.
If PhotoFeeler or similar apps drastically improve your chances, why wouldn't dating apps offer to improve your picture? Surely they are interested in more interactions between users? Snapchat automatic filters show that it is not hard or resource-intensive to touch up one's picture to make it look much better.
So, has this attempt failed? From the comment thread it looks like the OP has made an honest attempt and read up and probably learned a fair bit, but I did not see anything like "aha, this is where we disagree" and, to quote the link in the OP
If B, then A. Furthermore, if ¬B, then ¬A. You've both agreed that the states of B are crucial for the states of A, and in this way your continuing "agreement to disagree" isn't just "well, you take your truth and I'll take mine," but rather "okay, well, let's see what the evidence shows."
Economically it would be better if humans were sort of like salmon and died after a predetermined event, without suffering through physical and cognitive decline while draining the lion share of societal resources. Some of those potential events could be: age 60, birth of a great grandchild, completing a pilgrimage, writing an autobiography... Sadly, we are stuck with the evolutionary leftovers like waiting to die of disease and old age.
I suspect the real underlying issue is that of free will: all decision theories assume we can make different decisions in EXACT SAME circumstances, whereas from what we understand about the physical world, there is no such thing, and the only non-dualist proposal on the table is that of Scott Aaronson's freebits. I have written a related post last year. We certainly do have a very realistic illusion of free will, to the degree where any argument to the contrary tends to be rejected, ignored, strawmanned or misinterpreted. If you read through the philosophical writings on compatibilism, people keep talking past each other all the time, never getting to the crux of their disagreement. Not that it (or anything else) matters in the universe where there is no freedom of choice, anyway.
The issue of so-called logical counterfactuals has been discussed here and on the alignment forum quite a few times, including a bunch of posts by Chris_Leong, and at least one by yours truly. Consider browsing through them before embarking on original research:
Comment by shminux on [deleted post]
This situation, if real, seems self-correcting. If you are sure you have a superior technology, and can show you do by monetizing it (otherwise, how is it superior?), then you will gain a lot of attention and change a lot of minds pretty quick.
Maybe it is worth starting with a core definition of a lie that most people would agree with, something like "an utterance that is consciously considered by the person uttering it to misrepresent reality as they know it at that moment, with the intention to force the recipient to adjust their map of reality to be less accurate". Well, that's unwieldy. Maybe "an attempt to deceive through conscious misrepresentation of one's models"? Still not great.
I still intermittently run into people who claim that there's no such thing as reality or truth;
This sounds... strawmanny. "Reality and truth are not always the most useful concepts and it pays to think in other ways at times" would be a somewhat more charitable representation of non-realist ideas.
If I understood correctly, your objection to Three Worlds Collide is (mostly?) descriptive rather than prescriptive: you think the story is unrealistic, rather than dispute some normative position that you believe it defends.
I am not a moral realist, so I cannot dispute someone else's morals, even if I don't relate to them, as long as they leave me alone. So, yes, descriptive, and yes, I find the story a great read, but that particular element, moral expansionism, does not match the implied cohesiveness of the multi-world human species.
Do you believe real world humans are "slow to act against the morals it finds abhorrent"?
how do you explain all (often extremely violent) conflicts over religion and political ideology over the course of human history?
Generally, economic or some other interests in disguise. Like distracting the populous from the internal issues. You can read up on the reasons behind the Crusades, the Holocaust, etc. You can also notice that when the morals lead the way, extreme religious zealotry leads to internal instability, like the fractures inside Christianity and Islam. So, my model that you call "factually wrong" seems to fit the observations rather well, though I'm sure not perfectly.
Whatever explanation you provide to this survival, what prevents it from explaining the continued survival of the human species until the imaginary future in the story?
My point is that humans are behaviorally both much more and much less tolerant of the morals they find deviant than they profess. In the story I would have expected humans to express extreme indignation over babyeaters' way of life, but do nothing about it beyond condemnation.
It's frustrating where an honest exchange fails to achieve any noticeable convergence... Might try once more and if not, well, Aumann does not apply here, anyhow.
My main point: "to survive, a species has to be slow to act against the morals it finds abhorrent". I am not sure if this is the disagreement, maybe you think that it's not a valid implication (and by implication I mean the converse, "intolerant => stunted").
I had a pair programming experience at my first job back in the late 80s, before it was a thing, and my coworker and I clicked well, so it was fun while it lasted. Never had a chance to do it again, but miss it a lot. Wish I could work at a place where this is practiced.
I still don't understand, is your claim descriptive or prescriptive?
Neither... Or maybe descriptive? I am simply stating the implication, not prescribing what to do.
I don't understand what you're saying here at all.
Yes, we do have plenty of laws, but no one goes out of their way to find and hunt down the violators. If anything, the more horrific something is, the more we try to pretend it does not exist. You can argue and point at the law enforcement, whose job it is, but it doesn't change the simple fact that you can sleep soundly at night ignoring what is going on somewhere not far from you, let alone in the babyeaters' world.
"Universal we!right" is a contradiction in terms.
We may have not agreed on the meaning. I meant "human universal" not some species-independent morality.
in a given debate about ethics there might be hope that the participants can come to a consensus
I find it too optimistic a statement for a large "we". The best one can hope for is that logical people can agree with an implication like "given this set of values, this is the course of action someone holding these values ought to take to stay consistent", without necessarily agreeing with the set of values themselves. In that sense, again, it's describes self-consistent behaviors without privileging a specific one.
In general, it feels like this comment thread has failed to get to the crux of the disagreement, and I am not sure if anything can be done about it, at least without using a more interactive medium.
If interpreted in the logical sense, I don't think your argument makes sense: it seems like trying to derive an "ought" from an "is".
Hmm, in my reply to OP I expressed what the moral of the story is for me, and in my reply to you I tried to justify it by appealing to the expected stability of the species as a whole. The "ought", if any, is purely utilitarian: to survive, a species has to be slow to act against the morals it finds abhorrent.
Also, the actual distance between those diverging morals matters, and baby eating surely seems like an extreme example.
Uh. If you live in a city, there is a 99% chance that there is little girl within a mile from you being raped and tortured by her father/older brother daily for their own pleasure, yet no effort is made to find and save her. I don't find the babyeaters' morals all that divergent from human, at least the babyeaters had a justification for their actions based on the need for the whole species to survive.
I don't claim claim that leaving the Baby-eaters alone is necessarily we!wrong, but it is not obvious to me that it is we!right
My point is that there is no universal we!right and we!wrong in the first place, yet the story was constructed on this premise, which led to the whole species being hoisted on its own petard.
it is supposed to be a "weird" culture by modern standards), much less an alien culture like the Super-Happies
Oh. It never struck me as weird, let alone alien. The babyeaters are basically Spartans and the super-happies are hedonists.
The near-universal reaction of the crew to the baby-eaters customs is not just horror and disgust, but also the moral imperative to act to change them. It's as if there existed a species-wide objective "species!wrong", which is an untenable position, and, even less believably than that, as if there existed a "universal!meta-wrong" where anyone not adhering to your moral norms must be changed in some way to make it palatable (the super-happies are willing to go an extra mile to change themselves in their haste to fix things that are "wrong" with others).
This position is untenable because it would lead to constant internal infighting, as customs and morals naturally drift apart for a diverse enough society. Unless you impose a central moral authority and ruthlessly weed out all deviants.
I am not sure how much of the anti-prime-directive morality is endorsed by Eliezer personally, as opposed to merely being described by Eliezer the fiction writer.
I liked the story, but could never relate to its Eliezer-imposed "universal morality" of forcing others to conform to your own norms. To me the message of the story is "expansive metaethics leads to trouble, stick to your own and let others live the way they are used to, while being open to learning about each other's ways non-judgmentally".
I've tried to learn the basics of the category theory some years ago, already having some background in algebraic topology, mathematical physics and programming. And, presumably, in rationality. I got the glimpses of how interesting it is, how it could be useful, but was never quite able to make use of it. Very curious if your series of posts can change that for me. Keep going!