Comment by shminux on Graceful Shutdown · 2019-02-16T18:48:30.677Z · score: 4 (2 votes) · LW · GW

I'm against graceful shutdowns. The reason is that the client has to be able to handle a hard disconnect in any case, so might as well design it to handle it well, so no graceful shutdown is needed. Short-lived connections, always-consistent state, journaling, crash-only-software... There are plenty of techniques for handling hard shutdowns, use them instead of adding server code to mitigate client's failures of error handling.

Comment by shminux on Short story: An AGI's Repugnant Physics Experiment · 2019-02-14T15:47:24.614Z · score: 3 (3 votes) · LW · GW

Or a low-impact AI. "Don't break what you can't fix given your current level of knowledge and technology."

Comment by shminux on Some Thoughts on Metaphilosophy · 2019-02-14T15:44:26.825Z · score: 2 (1 votes) · LW · GW

I don't see this as anything related to containment. Just interaction. Good philosophy provides a well-defined problem to investigate for a given science, and, once in a blue moon, an outline of methodology, like Popper did. In turn, the scientific investigation in question can give philosophy some new "big" problems to ponder.

Comment by shminux on Probability space has 2 metrics · 2019-02-14T15:39:20.234Z · score: 2 (1 votes) · LW · GW

Could you explain your point further?

Comment by shminux on Learning preferences by looking at the world · 2019-02-14T06:42:23.026Z · score: 3 (2 votes) · LW · GW

That is indeed an interesting question, what constitutes uniqueness? Maybe simulating the past gives a hint in which circumstances the difference between snowflakes matter. Snow shoveling might be different from snowflake photography.

Comment by shminux on Learning preferences by looking at the world · 2019-02-13T15:36:28.715Z · score: 2 (1 votes) · LW · GW

Yes. Best not break those, either, unless explicitly instructed.

Comment by shminux on Learning preferences by looking at the world · 2019-02-13T08:42:54.881Z · score: 9 (2 votes) · LW · GW
The world state is “surprisingly” ordered and low-entropy. Anywhere you see such order, you can bet that a human was responsible for it, and that the human cared about it.

Indeed, the state of the world optimized by humans for humans tends to be rather ordered, with low entropy. An unstable equilibrium. Which means that small random deviations from a given human-optimized environment are nearly universally exothermic and entropy-increasing. A non-intrusive AI trying to follow its reward function, e.g. serve people at the dining table efficiently, would consider multiple ways to achieve its goal and evaluate

  • The change in entropy of the environment after the task is accomplished.
  • The extra effort/energy/cost required to restore the state of the world to the previous one.

In your examples, breaking the vase is very costly, first because it increases the entropy and releases energy, and second because restoring the state of the world means reassembling the vase from the shards, a very costly undertaking in general. So a non-intrusive robot might prefer to go around the vase, or maybe pick it up, move it out of the way, then put it back where it was, rather than break it. But if the vase is a cheap plastic one, and it knows that it is replaceable by an identical item from the store room, the robot might not care as much and allow for the possibility of knocking it over.

Whether it is necessary to simulate the past to figure out the cost of deviating from the present state, I am not sure. Entropy is often in the eye of the beholder (a good example is the clutter on someone's desk: it might seem like a mess to you, but they know exactly where everything is, and any externally imposed change, like arranging everything neatly, actually decreases the order for the desk's inhabitant), so maybe an AI would have trouble figuring out the cost of restoration in many cases. Maybe swapping chairs is OK, maybe not. But at least it is not likely to go around breaking things. Unless it has unlimited supply of replacements, in which case it might be acceptable. Unless a particular broken item has sentimental value to a particular human. Which would require digging quite far into the past.

Comment by shminux on Functional silence: communication that minimizes change of receiver's beliefs · 2019-02-13T08:08:25.421Z · score: 5 (3 votes) · LW · GW

Some of these techniques are known as phatic communication, I think.

Comment by shminux on Some Thoughts on Metaphilosophy · 2019-02-13T04:16:39.069Z · score: 2 (1 votes) · LW · GW
Many people doing philosophy, myself included, think of it more as the "mother" discipline from which we might specialize into other disciplines once we have the ground well understood enough to cleave off a part of reality for a time being while we work with that small part so as to avoid constantly facing the complete, overwhelming complexity of facing all of reality at once.

That's a great summary, yeah. I don't see any contradiction with what I said.

What is today philosophy is perhaps tomorrow a more narrow field of study, except it seems in those cases where we touch so closely upon fundamental uncertainty that we cannot hope to create a useful abstraction, like physics or chemistry, to let us manipulate some small part of the world accurately without worrying about the rest of it.

You have a way with words :) Yes, specific sciences study small slivers of what we experience, and philosophy ponders the big picture, helping to spawn another sliver to study. Still don't see how it provides answers, just helps crystallize questions.

Comment by shminux on How important is it that LW has an unlimited supply of karma? · 2019-02-11T15:34:23.372Z · score: 1 (2 votes) · LW · GW

I'd leave the current system mostly as is, but let people use their own karma to up/down-vote a post or comment they like/dislike, beyond the free level.

Comment by shminux on The Argument from Philosophical Difficulty · 2019-02-11T07:04:07.375Z · score: 2 (1 votes) · LW · GW

I may have misunderstood, sorry. I thought you gave it near 100% certainty that there could be only 3 ways, not the more reasonable "my knowledge of this problem is so marginal, I can't give it a good estimate of probability, since it would be drowned in error bars".

would you agree that people trying to solve a problem over some time only to find that all the plausible approaches they could come up with seem quite difficult is useful evidence for that problem being intrinsically difficult?

Certainly it's an indicator, especially if a group of smart people who have been able to successfully solve a number of related problems get stumped by something that appears to be in the same reference class. In my area it was the attempts to quantize gravity some time in the 50s and 60s, after successes with electromagnetism and weak interactions. After all, gravity is the weakest of them all. No one expected that there would be very little progress, half a century later, despite the signs being there.

I am not sure what "intrinsically difficult" means. My best guess is that it requires a Kuhnian paradigm change. Though sometimes it's not enough, and there are also the issues of just having to grind through a lot of calculations, like with the Fermat's last theorem, and the Poincaré conjecture. Special relativity, on the other hand, only required a "paradigm shift", the underlying math is trivial.

It might be interesting to consider this argument from an outside view perspective. Can you give a sample of arguments that you think is comparable to this one so we can check how valid they tend to be in retrospect?

One off-hand example that springs to mind is the Landau pole, inevitable and unavoidable in gauge theories. That resulted in the whole approach having been rejected in the Soviet Union for years, yet the the development of the renormalization formalism made QED the most precise physical theory, while still being mathematically inconsistent to this day. I strongly suspect that similarly adequate progress in AI alignment, for example, can be made without resolving all the mathematical, philosophical or meta-philosophical difficulties. The scenario 5 hints at something like that.

Comment by shminux on Probability space has 2 metrics · 2019-02-10T03:13:49.985Z · score: 5 (5 votes) · LW · GW

Note that the closer the probability of something to 0 or to 1, the harder it is evaluate accurately. A simple example: starting with a fair coin and observing a sequence of N heads in a row, what is an unbiased estimate of the coin's bias? Log odds of N heads are -N when starting with a point estimate of a fair coin, which matches the Bayesian updates, so it is reasonable to conclude that the probability of heads is 1-2^(-N), but at the level small enough there are so many other factors that can interfere, the calculation ceases being accurate. Maybe the coin has heads on both sides? Maybe your brain makes you see heads when the coin flip outcome is actually tails? Maybe you are only hallucinating the coin flips? So, if you finally get a tail, reducing the estimated probability of heads, you are able to reject multiple other unlikely possibilities, as well, and it makes sense that one would need less evidence when moving from -N to -N+1 for large N than for small N.

Comment by shminux on Some Thoughts on Metaphilosophy · 2019-02-10T02:01:27.024Z · score: 0 (7 votes) · LW · GW

As I said here countless times before, answering questions is not what philosophy is good at. It's good at asking questions, and figuring out how to slice a small manageable piece of a big question for some other science to work on. Sadly, most philosophers misunderstand what their job is. They absolutely suck at finding answers, even as they excel as debating the questions. The debate is important as it crystallizes how to slice the big question into smaller ones, but it does not provide answers. Sometimes it's the philosophers themselves that are polymaths enough to be able to both slice a question and to answer it, like Pierce/Russell/Wittgenstein with truth tables. Most of the time a good question is posed, or a non-obvious perspective is highlighted, like the oft-discussed here Searle's Chinese room argument, or Jackson's Mary's room setup, but the proposed solution itself is nowhere close to satisfactory.

Philosophy is NOT a general purpose problem solver, and NOT a meta problem solver, it is a (meta) problem problem asker and slicer.

Comment by shminux on The Argument from Philosophical Difficulty · 2019-02-10T01:43:10.607Z · score: 12 (9 votes) · LW · GW

Whenever someone says "there are only N ways that X is possible" outside of a mathematical proof, my immediate reaction is "Oh, great, here is another argument from lack of imagination". This seems like a typical case.

Comment by shminux on When should we expect the education bubble to pop? How can we short it? · 2019-02-09T23:03:23.140Z · score: 15 (7 votes) · LW · GW

Always good to look for a reference class example. Have any cost disease bubbles burst before? If not, why not, if so, what happened?

Comment by shminux on The Hamming Question · 2019-02-09T05:10:04.048Z · score: 6 (3 votes) · LW · GW

Yes, the comparative advantage answer is a compelling one, when it's not an excuse based on motivated cognition.

Comment by shminux on Boundaries - A map and territory experiment. [post-rationality] · 2019-02-06T16:19:37.393Z · score: 2 (1 votes) · LW · GW

I am not sure what you are referring to, and I'd be interested if you can link/PM me more details about what those people read.

Comment by shminux on Boundaries - A map and territory experiment. [post-rationality] · 2019-02-06T16:17:28.573Z · score: 2 (1 votes) · LW · GW

If you find yourself getting overly emotional over a reply on a rationality forum post, a prudent thing to do is to step away and chill for a bit before replying.

Comment by shminux on How to notice being mind-hacked · 2019-02-05T16:16:56.715Z · score: 2 (1 votes) · LW · GW

Seems we are talking past each other. You cannot imagine that a way of thinking where "reality" and the map/territory distinction is not primal, and I am tired of explaining why in my view this way of thinking is nice to start, but eventually harmful, I've been doing it on this blog for over 5 years. Take care.

Comment by shminux on Rationality: What's the point? · 2019-02-05T08:30:58.379Z · score: 2 (1 votes) · LW · GW

I found the concepts of true and false to be quite harmful for rational discourse. People argue about what is true or not all the time without coming to an agreement. So I avoid those terms as much as possible. Usefulness is easier to determine and it is subjective, so there is less urge to argue about it.

Comment by shminux on How to notice being mind-hacked · 2019-02-04T16:06:29.279Z · score: 2 (1 votes) · LW · GW

A useful model "pays rent", in Eliezer's words. Its predictions match future observations. It does not need to be related to any deep underlying truth, assuming one existed. It is also contextual, not absolute. It varies between people and cultures. Epicycles worked for astronomy and astrology for some centuries. Belief in God among religious people gets you to socialize and be accepted by the community, with all the associated perks, and so is useful, and therefore "good", if thriving in your community is what you value. If, on the other hand, self-consistency is what you are after, faith would not pay rent and you need to find a "better" way to make sense of the world.

Comment by shminux on Rationality: What's the point? · 2019-02-03T18:58:39.183Z · score: -1 (3 votes) · LW · GW
Things I claim you can get better at

Believing true things and not believing false things.
Arrive at true beliefs faster.
"Failing mysteriously" less often
Understanding how your own mind works.

The first two are not even a real thing. There are no absolutely true or false beliefs, only useful and harmful. The last two have more merit. Certainly spending time on analyzing something, including your own mind, tends to increase one's knowledge of that something. I have also heard anecdotal evidence of people "failing mysteriously" less often, but I am not convinced that better understanding how your mind works makes you feel less, as opposed to fail less mysteriously. If anything, people who I see as succeeding more tend to talk about "post-rationality" instead of rationality.

Comment by shminux on How to notice being mind-hacked · 2019-02-03T18:27:17.832Z · score: 2 (1 votes) · LW · GW

Right, certainly there are models that go viral without being accurate/predictive in scientific terms. They are nonetheless useful, at least in the context they proliferate. They often attain the status of "truth" and "fact", and in that way they are indeed "good models"

Comment by shminux on How to notice being mind-hacked · 2019-02-03T18:21:30.413Z · score: 2 (1 votes) · LW · GW

By analogy to the actual hacking, where the hacked system behaves significantly differently from the original in behavior, and this behavior is not intended or desired by/from the original system, and often benefits the hacker.

Comment by shminux on How to notice being mind-hacked · 2019-02-03T09:47:51.603Z · score: 2 (3 votes) · LW · GW
truth is just an effective hack in which you start to believe.

Exactly. That's one reason I dislike using the terms "true" and "fact" and instead prefer something like "a good model of.."

Comment by shminux on Depression philosophizing · 2019-02-03T05:04:59.823Z · score: 4 (3 votes) · LW · GW

It's useful to remember that general intelligence does not imply emotional intelligence, and it's natural for smarter people to realize their insignificance in the vast and uncaring universe, and run into a number of basilisks the rest of us never notice. As humans, we create our own meaning. And yes, CBT can be very useful to get out of depressive philosophizing.

How to notice being mind-hacked

2019-02-02T23:13:48.812Z · score: 16 (8 votes)
Comment by shminux on Boundaries - A map and territory experiment. [post-rationality] · 2019-02-01T15:39:17.696Z · score: 2 (1 votes) · LW · GW

Sometimes "where" is a wrong question to ask. Sort of like "where does the Universe expand to?"

Comment by shminux on Boundaries - A map and territory experiment. [post-rationality] · 2019-02-01T02:57:06.920Z · score: 2 (1 votes) · LW · GW

Seems you are only a step removed from what I have been saying here for years, it's maps all the way down. The idea of the territory is just another map.

Comment by shminux on The Question Of Perception · 2019-01-30T02:26:58.296Z · score: 3 (3 votes) · LW · GW

Super extra short mini summary: "It's models all the way down".

Comment by shminux on Confessions of an Abstraction Hater · 2019-01-27T06:31:52.606Z · score: 6 (4 votes) · LW · GW

Upvoted for

some codebases are sadder than Romeo and Juliet and King Lear combined

Also, readability vs boilerplating/code duplication is not an obvious trade-off

Comment by shminux on From Personal to Prison Gangs: Enforcing Prosocial Behavior · 2019-01-26T06:53:28.114Z · score: 2 (1 votes) · LW · GW

The click leads to an iterated prisoner dilemma game, not sure what this has to do with the Dunbar instability.

Comment by shminux on For what do we need Superintelligent AI? · 2019-01-26T06:49:47.521Z · score: 5 (3 votes) · LW · GW

Not even a superintelligent AI, but an Alpha* level AI could do a lot of good now, if it learned to understand humans without falling prey to the human biases. For example, an AI friend who knows just the right words to say in a given situation, never losing patience and never having own agenda would make the world a much better place almost instantly.

Comment by shminux on The human side of interaction · 2019-01-26T06:46:30.929Z · score: 2 (1 votes) · LW · GW

I'd expect a more promising approach toward capability amplification to be focused on actual human behavior, not on explicit human feedback. Humans are notoriously bad at explaining real reasons for why we do what we do, so accepting their words as quality feedback seems counterproductive. The feedback need not be ignored, but treated as just another source of information, just like lies and misguided ideas are a source of information about the person expressing them. A reward function would not be anything explicit, but a sort of a Turing test, (Pinocchio test?): fitting in and being implicitly recognized as a fellow human. That's how real humans learn, and seems like a promising way to start, at least in some constrained environment with reasonably clear behavioral boundaries and expectations.

Comment by shminux on From Personal to Prison Gangs: Enforcing Prosocial Behavior · 2019-01-26T05:18:28.337Z · score: 2 (1 votes) · LW · GW

Are you aware of any research in the area that numerical modeled this Dunbar instability?

Comment by shminux on Is Agent Simulates Predictor a "fair" problem? · 2019-01-26T02:14:50.201Z · score: 2 (1 votes) · LW · GW

Say you have certain information about the world and calculate the odds of different outcomes and their utilities. For example, in the twin prisoners dilemma the odds of DC and CD are zero, so the choice is between DD and CC. In the Newcomb's problem the odds of getting $1001000 are zero, so the choice is between $1000000 (one-box) and $1000 (two-box). In the Death in Damascus problem the odds of escaping Death are zero, so the choice is to spend money on travel or not. What would be a concrete example of an unfair problem against this approach?

Comment by shminux on Is Agent Simulates Predictor a "fair" problem? · 2019-01-25T07:35:50.019Z · score: 1 (2 votes) · LW · GW

How can a predictor be unfair to an algorithm that enumerates possible worlds and picks the best one, without any "decision theory" whatsoever? Unless by "unfair" you mean something like "you will get a coin that always lands tails, but the heads win, while everyone else gets a fair coin"

Comment by shminux on Should questions be called "questions" or "confusions" (or "other")? · 2019-01-23T03:58:51.283Z · score: 2 (1 votes) · LW · GW

Maybe expressing confusions can be another feature of LW2.0

Comment by shminux on Life can be better than you think · 2019-01-21T06:17:21.705Z · score: 3 (2 votes) · LW · GW

I asked because odds are that your insights only work for a brain with a certain neurochemistry. I have seen this in those with bipolar. Many have all these amazing insights when (hypo)manic, but none of them have any effect when depressed.

Comment by shminux on Life can be better than you think · 2019-01-20T19:36:06.288Z · score: 5 (4 votes) · LW · GW

Glad this med is working well for you. Are you still on Wellbutrin? If so, what do you think might happen if you stopped taking it, do you expect to be able to retain some of your insights if your brain chemistry reverts back to its pre-medicated state?

Comment by shminux on Curing the World of Men · 2019-01-20T01:02:40.168Z · score: 7 (5 votes) · LW · GW

I see your post and your condemnation as somewhat one-sided. I am an older cis white male who is not a fan of SJW, radfem and double standards in any areas, I cannot help but notice that your reading of the guidelines is skewed by your personal views and prejudices. Let's look at the analysis of the guidelines that you linked to, by Stephanie Pappas, rather than at your interpretation of them. Here are some quotes and comments. The guidelines themselves are here.

[Men] are the demographic group most at risk of being victimized by violent crime. They are 3.5 times more likely than women to die by suicide, and their life expectancy is 4.9 years shorter than women’s. Boys are far more likely to be diagnosed with attention-deficit hyperactivity disorder than girls, and they face harsher punishments in school—especially boys of color.

This seems like a statement of facts not at all unsympathetic to men's struggles.

“Though men benefit from patriarchy, they are also impinged upon by patriarchy,”

I hate when people use the notion of patriarchy to invalidate personal experiences of men, and there are plenty of other forces, at least in the West, that make life harder for men that feminism tends to discount, the above statement, if corrected to read “Though many men benefit from patriarchy, many are also impinged upon by patriarchy,” would match the experiences of most people. One ought to avoid the sweeping statements like "you are a cis white male, therefore you are automatically the beneficial of "patriarchy" and are complicit in the structural oppression of women", but treat each person's experiences individually, while being aware of the general trends, of course.

Prior to the second-wave feminist movement in the 1960s, all psychology was the psychology of men. Most major studies were done only on white men and boys, who stood in as proxies for humans as a whole. Researchers assumed that masculinity and femininity were opposite ends of a spectrum, and “healthy” psychology entailed identifying strongly with the gender roles conferred by a person’s biological sex.

This is not an unreasonable assessment of the pre-feminist state of affairs.

Once psychologists began studying the experiences of women through a gender lens, it became increasingly clear that the study of men needed the same gender-aware approach

Not sure why anyone would argue with that.

The main thrust of the subsequent research is that traditional masculinity—marked by stoicism, competitiveness, dominance and aggression—is, on the whole, harmful. Men socialized in this way are less likely to engage in healthy behaviors. [...] the more men conformed to masculine norms, the more likely they were to consider as normal risky health behaviors [...] This masculine reluctance toward self-care extends to psychological help.[...] men who bought into traditional notions of masculinity were more negative about seeking mental health services than those with more flexible gender attitudes [...] mental health professionals need to be aware that men are often reluctant to admit vulnerability [...] “Because of the way many men have been brought up—to be self-sufficient and able to take care of themselves—any sense that things aren’t OK needs to be kept secret,” [...] Though men report less depression than women, they complete suicide at far higher rates than women, and the numbers are moving in the wrong direction.

All of the above matches my personal experiences and has no hint of "science denial."

questionnaires on depression and other mental health problems are missing something when they garner answers suggesting men don’t struggle with these issues as much as women

I see this as raising alarms about men's psychological health, which is quite timely, especially in the face of pervasive invalidation of men's struggles in the media.

“Boys and men of color [are] dealing with all their hurts and their struggles in ways that are consistent with masculinity,” Liang says. “So, ‘be tough,’ and ‘don’t show your hurts.’ And they have to do this in a system where their behaviors are looked upon more negatively than boys and men from different groups.”

Again, seems like a reasonable description of the reality, doesn't it?

The post also acknowledge that the male/female dichotomy is outdated and recognize the rich spectrum of gender identification:

Today, transgender issues are at the forefront of the cultural conversation, and there is increased awareness of the diversity of gender identity.

Pappas then talks about the struggles of gender-nonconforming people, especially those that the society pigeonholes as males, and concludes the topic with

“There is a lot of diversity in the experience of men and masculinity, between groups, within groups and even within an individual,” Liang says. “What’s important is to understand that despite all of this diversity, boys and men may experience incredible pressure to live up to these rules around masculinity that they may have learned within their own cultural context.”

Again, the emphasis is on better understanding, something I can only cheer for.

Next the l talk about how to better help those who "would never dream of seeking mental health treatment". Sadly, this is where they are the weakest:

First, clinicians must be aware of dominant masculine ideals, and cognizant of their own potential biases. Second, they must recognize the integrated nature of masculinity, and how factors ranging from spirituality to ability status to age and ethnicity interact. Mental health professionals must also understand how power, privilege and sexism work both by conferring benefits to men and by trapping them in narrow roles. They should consider how stoicism and a reluctance to admit vulnerability hamstring men in personal relationships, and they should combat these forces, in part, by encouraging fathers to engage more fully with their children.

This is painting men with an overly wide brush and succumbing to the stereotypes. While some men certainly fit the above description, the situation is far from universal, at least in the modern Western societies. Some statements are more reasonable:

They should encourage men to protect their own health. And they should offer services sensitive to the socialization that men have undergone, while fighting against homophobia, transphobia, racial bias and other types of discrimination in institutions such as the criminal justice system.

They encourage the mental health professionals to look past the veneer:

When men do seek help, clinicians need to be aware that aggression and other externalizing symptoms can mask internalizing problems

The "Supporting the positive" section is a bit of a mixed bag. On the one hand it starts with "important to encourage pro-social aspects of masculinity," on the other, they attribute the largely gender- neutral traits like self-sacrifice and stoicism to mainly men. Men should "discard the harmful ideologies of traditional masculinity (violence, sexism) and find flexibility in the potentially positive aspects (courage, leadership)" seems a bit overly gender normative, and not very reflective of the realities.

Pappas then backtracks and says that

there’s less daylight between what’s expected of men and what’s expected of women than a glimpse at media and culture might reveal. About a third of the traits that people consider to be positive aspects of masculinity, such as sacrificing for others and having strong morals, are actually expected more from women than men when researchers ask both men and women [...] Indeed, when researchers strip away stereotypes and expectations, there isn’t much difference in the basic behaviors of men and women.

This is as moderate and reasonable a position as one can hope. I don't like the concluding sentence, though,

And if psychologists can focus on supporting men in breaking free of masculinity rules that don’t help them, the effects could spread beyond just mental health for men, McDermott says. “If we can change men,” he says, “we can change the world.”

mostly because it specifically targets men. If phrased as "supporting everyone, including men, in breaking free of gender rules that don't help them..." I would have no problem with it.

Comment by shminux on Debate AI and the Decision to Release an AI · 2019-01-18T03:00:27.985Z · score: 0 (2 votes) · LW · GW

If an AI has superhuman intelligence, it will make all the decisions, since human mind and tech is full of loopholes and exploits. There is just no way we could contain it if it wanted to be free. If it is not significantly smarter than humans, then there is little danger in releasing it. Using an extra AI as a judge of safety can only work if the judge is at least as smart as the prisoner, in which case you need a judge for the judge, ad infinitum. Maybe the judge can be only, say, 90% as smart as the intelligence it needs to decide on, then it might be possible to have a finite number of judges originating from an actual human, depending on how the probability of an error in judgment stacks up against the intelligence ratio at each step. Sort of like iterated amplification, or a blockchain.

Comment by shminux on Anthropics is pretty normal · 2019-01-18T02:46:22.047Z · score: 10 (2 votes) · LW · GW

Again, anthropics is basically generalizing from one example. Yes, humans have dodged an x-risk bullet a few times so far. There was no nuclear war. The atmosphere didn't explode when the first nuclear bomb was detonated (something that happens to white dwarfs in binary systems, leading to some supernovae explosion). The black plague pandemic did not wipe out nearly everyone, etc. If we have a reference class of x-risks and assign the probability of a close call p to each member of the class, then all we know is that after observing n close calls the probability of no extinction would be p^n. If the number is vanishingly small, we might want to reconsider our estimate of p ("the world is safer than we thought"). Or maybe the reference class is not constructed correctly. Or maybe we truly got luckier than other hypothetical observable civilizations who didn't make it. Or maybe quantum immortality is a thing. Or maybe something else. After all, there is only one example, and until we observe some other civilizations actually not making it through, anthropics is groundless theorizing. Maybe we can gain more insights into the reference classes and the probabilities of a close call, and of surviving an even from studying near extinction events roughly fitting into the same reference class (past asteroid strikes, plagues, climate changes, ...). However, none of the useful information comes from guessing the size of the universe, of whether we are in a simulation, of "updating based on the fact that we exist" beyond accounting for the close calls and x-risk events.

That said, I certainly agree with your point 4. That only the observed data need to be accounted for.

Comment by shminux on Solving the Doomsday argument · 2019-01-17T16:30:41.210Z · score: 9 (5 votes) · LW · GW

The Doomsday argument is utter BS because one cannot reliably evaluate probabilities without fixing a probability distribution first. Without knowing more than just the number of humans existing so far, the argument devolves into arguing which probability distribution to pick out of uncountable number of possibilities. An honest attempt to address this question would start with modeling human population fluctuations including various extinction events. In such a model there are multiple free parameters, such as rate of growth, distribution of odds of various extinction-level events, distribution of odds of surviving each type of events, event clustering and so on. The the minimum number of humans does not constrain the models in any interesting way, i.e. to privilege a certain class of models over others, or a certain set of free parameters over others to the degree where we could evaluate a model-independent upper bound for the total number of humans with any degree of confidence.

If you want to productively talk about Doomsday, you have to get your hands dirty and deal with specific x-risks and their effects, not armchair-theorize based on a single number and a few so-called selection/indication principles that have nothing to do with the actual human population dynamics.

Comment by shminux on Some Thoughts on My Psychiatry Practice · 2019-01-17T02:57:57.457Z · score: 5 (3 votes) · LW · GW

A spelling note: It's Goodhart's law. A question: do you come across patients with DID, not "just" C-PTSD?

Comment by shminux on Human-AI Interaction · 2019-01-15T03:32:24.152Z · score: 2 (1 votes) · LW · GW

When thinking about how a smarter-than-human AI would treat human input to close the control loop, it pays to consider the cases where humans are that smart intelligence. How do we close the loop when dealing with young children? primates/dolphins/magpies? dogs/cats? fish? insects? bacteria? In all these cases the apparent values/preferences of the "environment" are basically adversarial, something that must be taken into account, but definitely not obeyed. In the original setup a super-intelligent aligned AI's actions would be incomprehensible to us, no matter how much it would try to explain them to us (go explain to a baby that eating all the chocolate it wants is not a good idea, or to a cat that their favorite window must remain closed). Again, in the original setup it can be as drastic as an AI culling the human population, to help save us from a worse fate, etc. Sadly, this is not far from the "God works in mysterious ways" excuse one hears as a universal answer to the questions of theodicy.

Comment by shminux on Towards formalizing universality · 2019-01-14T03:12:30.292Z · score: 4 (2 votes) · LW · GW

Those are all pretty opaque, as in, their inner workings are not immediately obvious, so it's natural to take the intentional stance toward them. I had in mind something much simpler. For example, does an algorithm that adds two numbers have a belief about the rules of addition? Does a GIF to JPEG converter have a belief about which image format is "better"?

Comment by shminux on Towards formalizing universality · 2019-01-13T21:41:04.601Z · score: 2 (1 votes) · LW · GW

Can you describe a simple non-opaque algorithm to which you can meaningfully ascribe a belief?

Comment by shminux on Electrons don’t think (or suffer) · 2019-01-04T16:35:36.391Z · score: 10 (2 votes) · LW · GW

Seems people are really confused about the statements made in the linked post.

1. Individual elementary particles have no internal states beyond mass/charge/spin etc. and therefore are incapable of thinking, feeling or suffering, as this would require, at a minimum, processes changing those states.

2. Human-shaped collections of underlying physical constituents clearly have the sensations of feeling something.

3. We do not currently know where the boundary is, as in, what kind of physical or logical structures are needed to support qualia.

Electrons don’t think (or suffer)

2019-01-02T16:27:13.159Z · score: 5 (7 votes)
Comment by shminux on Revealed preferences vs. misaligned incentives · 2018-12-30T03:50:22.292Z · score: 3 (2 votes) · LW · GW

I suspect the situation is vastly more complicated than that. Revealed preferences show the contradiction between stated preferences and actions. Misaligned incentives models (a part of) a person as a separate agent with distinct short-term goals. But humans are not modeled well as a collection of agents. We are a messy result of billions of years of evolution, with some random mutations becoming meta-stable through sheer chance. All human behavior is a side effect of that. Certainly both RPT and MIT can be a rough starting point, and if someone actually numerically simulates human behavior, the two could be some of the algorithms to use. But I am skeptical they together would explain/predict a significant fraction of what we do.

Comment by shminux on Why I expect successful (narrow) alignment · 2018-12-29T19:55:09.268Z · score: 9 (6 votes) · LW · GW

Number two is sometimes known as the Adams Law of Slow-Moving Disasters:

whenever humanity can see a slow-moving disaster coming, we find a way to avoid it

In general, however, it seems that you believe something you want to believe and find justifications for this belief, because it is more comfortable to think that things will magically work out. Eliezer wrote at length about this failure mode.

Sabine "Bee" Hossenfelder (and Robin Hanson) on How to fix Academia with Prediction Markets

2018-12-16T06:37:13.623Z · score: 11 (3 votes)

Aligned AI, The Scientist

2018-11-12T06:36:30.972Z · score: 12 (3 votes)

Logical Counterfactuals are low-res

2018-10-15T03:36:32.380Z · score: 22 (8 votes)

Decisions are not about changing the world, they are about learning what world you live in

2018-07-28T08:41:26.465Z · score: 31 (16 votes)

Probability is a model, frequency is an observation: Why both halfers and thirders are correct in the Sleeping Beauty problem.

2018-07-12T06:52:19.440Z · score: 24 (12 votes)

The Fermi Paradox: What did Sandberg, Drexler and Ord Really Dissolve?

2018-07-08T21:18:20.358Z · score: 47 (20 votes)

Wirehead your Chickens

2018-06-20T05:49:29.344Z · score: 72 (44 votes)

Order from Randomness: Ordering the Universe of Random Numbers

2018-06-19T05:37:42.404Z · score: 14 (4 votes)

Physics has laws, the Universe might not

2018-06-09T05:33:29.122Z · score: 28 (14 votes)

[LINK] The Bayesian Second Law of Thermodynamics

2015-08-12T16:52:48.556Z · score: 8 (9 votes)

Philosophy professors fail on basic philosophy problems

2015-07-15T18:41:06.473Z · score: 16 (21 votes)

Agency is bugs and uncertainty

2015-06-06T04:53:19.307Z · score: 10 (17 votes)

A simple exercise in rationality: rephrase an objective statement as subjective and explore the caveats

2015-04-18T23:46:49.750Z · score: 19 (21 votes)

[LINK] Scott Adam's "Rationality Engine". Part III: Assisted Dying

2015-04-02T16:55:29.684Z · score: 7 (8 votes)

In memory of Leonard Nimoy, most famous for playing the (straw) rationalist Spock, what are your top 3 ST:TOS episodes with him?

2015-02-27T20:57:19.777Z · score: 10 (15 votes)

We live in an unbreakable simulation: a mathematical proof.

2015-02-09T04:01:48.531Z · score: -31 (42 votes)

Calibrating your probability estimates of world events: Russia vs Ukraine, 6 months later.

2014-08-28T23:37:06.430Z · score: 19 (19 votes)

[LINK] Could a Quantum Computer Have Subjective Experience?

2014-08-26T18:55:43.420Z · score: 16 (17 votes)

[LINK] Physicist Carlo Rovelli on Modern Physics Research

2014-08-22T21:46:01.254Z · score: 6 (11 votes)

[LINK] "Harry Potter And The Cryptocurrency of Stars"

2014-08-05T20:57:27.644Z · score: 2 (4 votes)

[LINK] Claustrum Stimulation Temporarily Turns Off Consciousness in an otherwise Awake Patient

2014-07-04T20:00:48.176Z · score: 37 (37 votes)

[LINK] Why Talk to Philosophers: Physicist Sean Carroll Discusses "Common Misunderstandings" about Philosophy

2014-06-23T19:09:54.047Z · score: 10 (12 votes)

[LINK] Scott Aaronson on Google, Breaking Circularity and Eigenmorality

2014-06-19T20:17:14.063Z · score: 20 (20 votes)

List a few posts in Main and/or Discussion which actually made you change your mind

2014-06-13T02:42:59.433Z · score: 16 (16 votes)

Mathematics as a lossy compression algorithm gone wild

2014-06-06T23:53:46.887Z · score: 39 (41 votes)

Reflective Mini-Tasking against Procrastination

2014-06-06T00:20:30.692Z · score: 17 (17 votes)

[LINK] No Boltzmann Brains in an Empty Expanding Universe

2014-05-08T00:37:38.525Z · score: 9 (11 votes)

[LINK] Sean Carroll Against Afterlife

2014-05-07T21:47:37.752Z · score: 5 (9 votes)

[LINK] Sean Carrol's reflections on his debate with WL Craig on "God and Cosmology"

2014-02-25T00:56:34.368Z · score: 8 (8 votes)

Are you a virtue ethicist at heart?

2014-01-27T22:20:25.189Z · score: 11 (13 votes)

LINK: AI Researcher Yann LeCun on AI function

2013-12-11T00:29:52.608Z · score: 2 (12 votes)

As an upload, would you join the society of full telepaths/empaths?

2013-10-15T20:59:30.879Z · score: 7 (17 votes)

[LINK] Larry = Harry sans magic? Google vs. Death

2013-09-18T16:49:17.876Z · score: 25 (31 votes)

[Link] AI advances: computers can be almost as funny as people

2013-08-02T18:41:08.410Z · score: 7 (9 votes)

How would not having free will feel to you?

2013-06-20T20:51:33.213Z · score: 6 (14 votes)

Quotes and Notes on Scott Aaronson’s "The Ghost in the Quantum Turing Machine"

2013-06-17T05:11:29.160Z · score: 18 (22 votes)

Applied art of rationality: Richard Feynman steelmanning his mother's concerns

2013-06-04T17:31:24.675Z · score: 8 (17 votes)

[LINK] SMBC on human and alien values

2013-05-29T15:14:45.362Z · score: 3 (10 votes)

[LINK]s: Who says Watson is only a narrow AI?

2013-05-21T18:04:12.240Z · score: 4 (11 votes)

LINK: Google research chief: 'Emergent artificial intelligence? Hogwash!'

2013-05-17T19:45:45.739Z · score: 7 (16 votes)

[LINK] The Unbelievers: Lawrence Krauss and Richard Dawkins Team Up Against Religion

2013-04-30T18:11:13.901Z · score: 1 (15 votes)

Litany of a Bright Dilettante

2013-04-18T05:06:05.490Z · score: 55 (66 votes)

Time turners, Energy Conservation and General Relativity

2013-04-16T07:23:13.411Z · score: 7 (30 votes)

Litany of Instrumentarski

2013-04-09T15:07:10.565Z · score: 3 (17 votes)

Noticing the 5-second mindkill

2013-03-28T05:58:50.447Z · score: 14 (19 votes)

[LINK] Transcendence (2014) -- A movie about "technological singularity"

2013-03-21T16:51:00.007Z · score: 8 (11 votes)

Exponent of Desire

2013-02-26T18:01:40.545Z · score: 8 (19 votes)

"What-the-hell" Cognitive Failure Mode: a Separate Bias or a Combination of Other Biases?

2013-02-22T21:44:01.437Z · score: 18 (19 votes)