Posts

Write down your basic realizations as a matter of habit. Share this file with the world. 2014-01-17T12:57:04.455Z · score: 14 (17 votes)
Forked Russian Roulette and Anticipation of Survival 2012-04-06T03:57:08.683Z · score: 7 (12 votes)

Comments

Comment by feepingcreature on Philosophy in the Darkest Timeline: Basics of the Evolution of Meaning · 2020-06-22T07:00:11.053Z · score: 1 (1 votes) · LW · GW

Fair enough - it's probably good to have it in writing. But this seems to me like the sort of explanation that is "the only possible way it could conceivably work." How could we bootstrap language learning if not for our existing, probably-inherent faculty for correlating classifiers over the the environment? Once you say "I want to teach something the meaning of a word, but the only means I have to transmit information to them is present them with situations and have them make inferences"… there almost isn't anything to add to this. The question already seems to contain the only possible answer.

Maybe you need to have read Through the Looking Glass?

Comment by feepingcreature on Philosophy in the Darkest Timeline: Basics of the Evolution of Meaning · 2020-06-08T08:25:38.552Z · score: -2 (11 votes) · LW · GW

I thought this was the standard theory of meaning that everyone already believed.

Is there anyone who doesn't know this?

Comment by feepingcreature on The One Mistake Rule · 2020-04-11T10:57:55.370Z · score: 1 (1 votes) · LW · GW

But to be fair, if you then fixed the model to output errors once you exceeded the speed of light, as the post recommends, you would have come up with a model that actually communicated a deep truth. There's no reason a model has to be continuous, after all.

Comment by feepingcreature on Predictors exist: CDT going bonkers... forever · 2020-01-15T07:40:40.249Z · score: 10 (2 votes) · LW · GW

Since when does CDT include backtracking on noticing other people's predictive inconsistency? And, I'm not sure that any such explicitly iterative algorithm would be stable.

  1. The CDT agent considers making the decision to say “one” but notices that Omega’s prediction aligns with its actions.

This is the key. You're not playing CDT here, you're playing "human-style hacky decision theory." CDT cannot notice that Omega's prediction aligns with its hypothetical decision because Omega's prediction is causally "before" CDT's decision, so any causal decision graph cannot condition on it. This is why post-TDT decision theories are also called "acausal."

Comment by feepingcreature on The "Commitment Races" problem · 2020-01-05T15:38:53.135Z · score: 2 (2 votes) · LW · GW

True, sorry, I forgot the whole set of paradoxes that led up to FDT/UDT. I mean something like... "this is equivalent to the problem that FDT/UDT already has to solve anyways." Allowing you to make exceptions doesn't make your job harder.

Comment by feepingcreature on The "Commitment Races" problem · 2020-01-04T02:05:52.698Z · score: 1 (1 votes) · LW · GW

I concur in general, but:

you might accidentally realize that such-and-such type of agent will threaten you regardless of what you commit to and then if you are a coward you will “give in” by making an exception for that agent.

this seems like a problem for humans and badly-built AIs. Nothing that reliably one-boxes should ever do this.

Comment by feepingcreature on Meta-discussion from "Circling as Cousin to Rationality" · 2020-01-04T02:03:42.652Z · score: 10 (2 votes) · LW · GW

I don't think it's so implausible for some people to be significantly more baffled by some things that we must interpret it as an attack. An unusually large imposition of costs is not inherently an attack! May as well blame the disabled for dastardly forcing us to waste money on wheelchair ramps.

Comment by feepingcreature on Meta-discussion from "Circling as Cousin to Rationality" · 2020-01-03T01:44:54.296Z · score: 19 (5 votes) · LW · GW

I think this once again presupposes a lot of unestablished consensus: for one, that it's trivial for people to generate hypotheses for undefined words, that this is a worthwhile skill to begin with, and that this is a proper approach to begin with. I don't think that a post author should get to impose this level of ideological conformance onto a commenter, and it weirds me out how much the people on this site now seem to be agreeing that Said deserves censure for (verbosely and repeatedly) disagreeing with this position.

And then it seems to be doing a lot of high-distance inference from presuming a "typical" mindset on Said's part and figuring out a lot of implications as to what they were doing, which is exactly the thing that Said wanted to avoid by not guessing a definition? Thus kind of proving their point?

More importantly, I at least consider providing hypotheses as to a definition as obviously supererogatory. If you don't know the meaning of a word in a text, then the meaning may be either obvious or obscured; the risk you take by asking is wasting somebody's time for no reason. But I consider it far from shown that giving a hypothesis shortens this time at all, and more importantly, there is none such Schelling point established and thus it seems a stretch of propriety to demand it as if it was an agreed upon convention. Certainly the work to establish it as a convention should be done before the readership breaks out the mass downvotes; I mean seriously- what the fuck, LessWrong?

Comment by feepingcreature on Meta-discussion from "Circling as Cousin to Rationality" · 2020-01-02T23:00:18.735Z · score: 21 (8 votes) · LW · GW

I find myself thinking: if you’re so consistently unable to guess what people might mean, or why people might think something, maybe the problem is (at least some of the time) with your imagination.

Who cares who "the problem" is with? Text is supposed to be understood. The thing that attracted me to the Sequences to begin with was sensible, comprehensible and coherent explanations of complex concepts. Are we giving up on this? Or are people who value clear language and want to avoid misunderstandings (and may even be, dare I say, neuroatypical) no longer part of the target group, but instead someone to be suspicious of?

The Sequences exist to provide a canon of shared information and terminology to reference. If you can't explain something without referencing a term that is evidently not shared by everyone, and that you don't just not bother to define but react with hostility when pressed on, then ... frankly, I don't think that behavior is in keeping with the spirit of this blog.

Comment by feepingcreature on Does GPT-2 Understand Anything? · 2020-01-02T22:27:36.420Z · score: 6 (5 votes) · LW · GW

Sentences 1 and 4 should have higher probability than sentences 2 and 3. What they find is that GPT-2 does worse than chance on these kinds of problems. If a sentence is likely, a variation on the sentence with opposite meaning tends to have similar likelihood.

I can anecdotally confirm this; I've been personally calling this the "GPT swerve", ie. sentences of the form "We are in favor of recycling, because recycling doesn't actually improve the environment, and that's why we are against recycling."

The proposed explanation makes sense as well. Is anyone trying to pre-train a GPT-2 with unlikelihood avoidance?

Comment by feepingcreature on Circling as Cousin to Rationality · 2020-01-01T05:02:08.735Z · score: 4 (3 votes) · LW · GW

I think the Litany of Gendlin sorta bridges between those sentiments - anything that can be destroyed by the truth should be, because it cannot be a load-bearing belief since it doesn't do any work.

Of course, the amount of effort you have to put in to (re)construct a properly working belief may be significant and the interval in between may be quite unsettling.

Comment by feepingcreature on The "Commitment Races" problem · 2019-12-30T03:00:25.208Z · score: 2 (2 votes) · LW · GW

I think this undervalues conditional commitments. The problem of "early commitment" depends entirely on you possibly having a wrong image of the state of the world. So if you just condition your commitment on the information you have available, you avoid premature commitments made in ignorance and give other agents an incentive to improve your world model. Likewise, this would protect you from learning about other agents' commitments "too late" - you can always just condition on things like "unless I find an agent with commitment X". You can do this whether or not you even know to think of an agent with commitment X, as long as other agents who care about X can predict your reaction to learning about X.

Commitments aren't inescapable shackles, they're just another term for "predictable behavior." The usefulness of commitments doesn't require you to bind yourself regardless of learning any new information about reality. Oaths are highly binding for humans because we "look for excuses", our behavior is hard to predict, and we can't reliably predict and evaluate complex rule systems. None of those should pose serious problems for trading superintelligences.

Comment by feepingcreature on Minicamps on Rationality and Awesomeness: May 11-13, June 22-24, and July 21-28 · 2019-12-13T10:15:17.527Z · score: 1 (1 votes) · LW · GW

Probabilities can be empirically wrong, sure, but I find it weird to say that they're "not probabilities" until they're calibrated. If you imagine 20 scenarios in this class, and your brain says "I expect to be wrong in one of those", that just is a probability straight up.

(This may come down to frequency vs belief interpretations of probability, but I think saying that beliefs aren't probabilistic at all needs defending separately.)

Comment by feepingcreature on Multiple Moralities · 2019-11-03T18:46:33.929Z · score: 8 (6 votes) · LW · GW

Most morality is cashed out game theory. In humans this usually takes the form of "rule utilitarianism", because first of all humans don't have the cognitive capability to evaluate a utilitarian criterion over their entire worldstate, and second of all some if not all humans would defect if given the chance, so everyone agrees it's best we pick a system where they're not given the chance, so we can all benefit from the cooperate/cooperate outcomes. Now, when evaluating the quality of a rule, we use something called the "Rawlsian veil", which avoids temporary imbalances in power from skewing the outcomes - for instance, if in any exchange, only one player gets to decide the rules, but which player it is varies randomly, then both players will want the exchange to be judged as if both players have equal worth because otherwise you can't get the cooperate/cooperate outcomes yada yada. (Power imbalances are common in human society.) So the way our morality works, or at least this is the way we'll claim it works when asked¹, is whenever we face a moral challenge we first set the moral value of all participants to equal, and then compute the rule that maximizes outcomes across the board. Understand that this is not arbitrary - so far everything we're saying is for the purpose of maximizing payoffs and somewhat insensitive to the actual distribution of rewards, and we should expect this system - call it "rule of law" - to be somewhat convergent in alien species.

Transitioning society to this shape was not an instant process. As such, we also have a bunch of metamorality bound up; we see people who defect against weaker partners as exploiters, and people who cooperate even when stronger as heroic and good. However, because we needed to transition to this morality (it wasn't inherently inborn in our species) we also have standards for when defecting is acceptable, and it's that if defecting is ever acceptable it's against people who themselves defected, denying them the benefits of the social compact of rule of law. Again, we should expect this to be convergent.

Now, it's true that if you're really on top of things you can defect to your heart's content. However, you just met aliens. So what you're actually espousing, behaviorally, is that more powerful aliens can eat you if it makes them really really happy - you're saying that you're happy being in the defector category. If you ever meet more powerful aliens, that may be risky - in fact, you should preemptively try to guess what kinds of rules these aliens would expect arbitrary aliens to apply and live by them.

Probably "don't eat weaker participants" is not a rule you want to be exempt from...

¹ In our current society, we don't always live by these principles, but also we're in the tail end of a transition away from belief in a higher power, so we're in an unusually defecty stage. It's unclear how representative that is.

Comment by feepingcreature on When is pair-programming superior to regular programming? · 2019-10-09T15:13:15.237Z · score: 9 (6 votes) · LW · GW

By personal subjective experience, the usecase where pair programming shines most is onboarding and skill transfer. Basically, you have a person who needs to learn a skill and a person who has the skill, so you sit the person who has the skill next to the other person and get the other person to do a task that needs the skill by instruction of the person who has the skill. This is way more efficient than lectures and slightly more efficient than reading documentation, because all parts of the skill are taught, only relevant information is taught, and the instruction is necessarily in actionable form. Important conditions are that the learning person is open to instruction and correction, and the teaching person avoids meandering into irrelevant information. Pair programming done like this relies on both parties being effective communicators.

(A related technique is mob programming, which is useful for collaborative design assuming your code environment is highlevel enough to keep pace with the discussion.)

Comment by feepingcreature on A Critique of Functional Decision Theory · 2019-09-13T20:30:39.056Z · score: 11 (8 votes) · LW · GW

Not a decision theorist, but my intuition on the first example with the bomb also says "take the bomb". I don't think it's obvious or universal that one should choose to avoid burning slowly to death; the example may make more sense if one optimizes over "agents like me who encounter the box", instead of "the specific agent who sees a bomb"; ie. acting under a Rawlsian veil. The standard argument is if you could commit yourself in advance to slowly burning to death if you see a bomb, you would certainly do so; the commitment all but guarantees it does not happen. For another example, "maximize payoff for any situation you find yourself in" fails to second-strike in global thermonuclear warfare (MAD), leading to the extinction of humanity. (This is not dissimilar to slowly burning to death.) So I think your "guaranteed payoff" rule is contradicted in practice; one may argue it does little more than judge FDT by CDT.

Comment by feepingcreature on Reasonable Explanations · 2019-06-16T08:23:13.718Z · score: 8 (4 votes) · LW · GW

I had a similar one to that, where I completely overwrote my actual memory of what happened with what habit said should have happened, where I went to get my bike from the garage and it was not there. But I clearly remembered having stored it in the garage the day prior.

Spoiler: I hadn't. I'd gone to the store on the way back, left the bike locked in front of the store, then (since I almost always go to the store on foot) walked home. My brain internally rewrote this as "rode home, [stored bike, went to the store], went home." (The [] part did not happen.)

Memory is weird, especially if your experience is normally highly compressible.

Comment by feepingcreature on Drowning children are rare · 2019-05-28T20:58:41.716Z · score: 3 (2 votes) · LW · GW

Doesn't this only hold if you abdicate all moral judgment to Gates/Good Ventures, such that if Gates Foundation/Good Ventures pass up an opportunity to save lives, it follows necessarily that the offer was fraudulent?

Comment by feepingcreature on Minimize Use of Standard Internet Food Delivery · 2019-02-13T19:33:02.551Z · score: 4 (3 votes) · LW · GW

Non-profits are "profitable" in the limit sense of a profit of zero. Non-profits with negative profit cannot exist and, in fact, generally quickly cease to.

There are problems with the equation "profit=worth", but it holds to a first approximation. The free market is vulnerable to collusion, fraud and outright value hijacking, but those are all manipulations and divergences of the baseline, which is "1 money = 1 unit of caring."

I usually tend to assume that the markets, being the dominant optimization power in society, are the "authority" on value, because they generally function to model society's revealed preferences. A thought that often comes to mind is "If you didn't want X, why did you allow your markets to fall into an X attractor?" I suspect people tend to model markets as cosmic laws, whereas I think of them more as highly powerful mechanical contraptions that require maintenance. Or maybe as a dev I just model everything as software.

Comment by feepingcreature on Minimize Use of Standard Internet Food Delivery · 2019-02-11T20:49:06.051Z · score: 12 (8 votes) · LW · GW

I bought a service at an excessive price, so I'm defecting? What?

If the delivery services are taking too much, they'll be outcompeted. Given your description, Slice is not some sort of hero, it's a vertically integrated competitor doing exactly what the free market would expect to happen.

In any case, I reject the notion that "giving a company too much money" could in any sense be mapped to the game theoretic notion of defection. That's not how game theory works, and it's not how markets work.

If GrubHub was misadvertising their costs or declaring that 90% of the price went to the restaurant, and it didn't, then sure. But that's not my impression.

GrubHub offers a service for a price. In the back, it negotiates with the restaurant to push down prices. This is good and proper. The free market drives margins to zero; the point of it is to drive margins to zero. If you don't want that, then don't sell on GrubHub?

"But all my competitors are on GrubHub and I can't compete!" Yes? Were you under the impression that you were owed a business model? A business is either profitable or it isn't. If it is not profitable, then it morally shouldn't ought to exist; the market is indicating that the business is a waste of society's limited resources. Propping up a business that is unprofitable is more defection to me than anything GrubHub does.

Comment by feepingcreature on Bottle Caps Aren't Optimisers · 2018-12-26T20:41:16.262Z · score: 1 (1 votes) · LW · GW

The model of the bank account compresses the target function of the brain, even when expressed in terms of specific physical effects on the world. Further, the model of health compresses the target function of the liver better than the bank account.

Comment by feepingcreature on Is Clickbait Destroying Our General Intelligence? · 2018-11-17T11:17:18.215Z · score: 3 (2 votes) · LW · GW

Eh, the gig economy will fix it.

I can't think of any economic model that would more select for the ability to take pieces of cognitive architecture and put them together into novel ways. Weren't you the one who said science was going too slow, and that a true Bayesian should be able to solve shallow problems in, let's say a quarter hour and more complex ones like unified physics in a week? That does not sound "old style of work" to me, but it does moreso sound, amusingly, "glib memetics" - and "startups". Similarly, the Agile model of development is to accept doing damage but be aware of it as you do so - make the feature usable, then move on but put a cleanup task in the backlog. At least where I work, modern-style seems more reactive and demanding of fluidity, and startup/gig work can only increase that. I think we're kind of in a transition phase where the human mind is being effectively operationalized as a target platform, but large parts of the population haven't fully evolved the software to actually manage being treated as a target, and social systems are taking full advantage. But society is also taking advantage of the increased flexibility on offer here, and in the medium run self-awareness will have to catch up to be able to keep up and frontrun the rapidly-evolving memetic environment. At least that's my expectation.

Comment by feepingcreature on Rationality Is Not Systematized Winning · 2018-11-12T00:11:03.783Z · score: 22 (11 votes) · LW · GW

“Look, if I go to college and get my degree, and I go start a traditional family with 4 kids, and I make 120k a year and vote for my favorite political party, and the decades pass and I get old but I'm doing pretty damn well by historical human standards; just by doing everything society would like me to, what use do I have for your 'rationality'? Why should I change any of my actions from the societal default?”

You must have an answer for them. Saying rationality is systematized winning is ridiculous. It ignores that systematized winning is the default, you need to do more than that to be attractive. I think the strongest frame you can use to start really exploring the benefits of rationality is to ask yourself what advantage it has over societal defaults. When you give yourself permission to move away from the "systematized winning" definition, without the fear that you'll tie yourself in knots of paradox; it's then that you can really start to think about the subject concretely.

I mean, isn't the answer to that, as laid out in the Sequences, that Rationality really doesn't have anything to offer them? Tsuyoku Naritai, Something to Protect, etc. - Eliezer made the Sequences because he needed people to be considering the evidence that AI was dangerous and was gonna kill everyone by default, so short-term give money to MIRI and/or long-term join up as a researcher. "No one truly searches for the Way until their parents have failed them, their Gods are dead and their tools have shattered in their hands." I think it's fair that the majority of people don't have problems with that sort of magnitude of impact in their lives; and in any case, anyone who cared that much would already have gone off to join an EA project. I'm not sure that Eliezer-style rationality needs to struggle for some way to justify its existence when the explicit goal of its existence has already largely been fulfilled. Most people don't have one or two questions in their life that they absolutely, pass-or-die need to get right, and the answer is nontrivial. The societal default is a time-tested satisficing path.

When you are struggling to explain why something is true, make sure that it actually is true.

Comment by feepingcreature on Update on Structured Concurrency · 2018-10-22T05:21:29.201Z · score: 5 (3 votes) · LW · GW

I've read the relevant articles on structured concurrency, and I don't see what it buys you.

To be honest, in all my time writing threaded code, I have never once worried about the lifetime of threads. If I wanted to shut threads down in a certain order, I'd just use condition variables to have the latter threads wait on the former threads' shutdown sequences. Is this just a more readable way of implementing that sort of thing? Which, well, great, but with a name like "structured concurrency" I'd expect a larger paradigm shift than that.

Comment by feepingcreature on Probability is a model, frequency is an observation: Why both halfers and thirders are correct in the Sleeping Beauty problem. · 2018-07-12T18:18:16.003Z · score: 3 (2 votes) · LW · GW

Say that the second time you wake her on Monday, you just outright ignore everything she says. Suddenly, because you changed your behavior, her objectively correct belief is 0.5 / 0.5?

The real problem is that the question is undefined - Sleeping Beauty has no goal function. If she had a goal function, she could just choose the probability assignment that maximized the payoff under it. All the handwaving at "standard frameworks" is just another way to say "assignment that maximizes payoff under a broad spread of goals".

Alternate scenario: all three wakings of Sleeping Beauty are actually copies. Monday 2 is deleted after a few minutes. What's the "true" probability then?

Comment by feepingcreature on Are ethical asymmetries from property rights? · 2018-07-02T05:26:51.797Z · score: 7 (4 votes) · LW · GW

At least where I live, two out of three of those property rights are wrong.

Property rights explicitly give way to more basic human rights. For instance, you are allowed to steal a car if it's the only way that you can get an injured person to a hospital. And of course, you're allowed to steal bread if that's the only way you can get to eat.

Comment by feepingcreature on Duncan Sabien: "In Defense of Punch Bug" · 2018-05-16T17:45:57.196Z · score: 31 (9 votes) · LW · GW

I would actually not be able to think in common or public spaces with that ambient level of physical threat.

As I understand the argument, the claim is that you (or rather, the reference class of people who feel like you) only react like that because you are hypersensitized to harmless (ie. with a short physiological return to baseline) threats due to lack of exposure.

If I am around a spider, I am in a state of mild to severe panic, depending on size. But this is not a fact about the inherent horribleness of spiders, but about my phobic mindset, and attempting to make the world spider-free would be a completely undue cost compared to treating my phobia.

To me, the question with Punch Bug is whether the suffering imposed on people who are naturally ill-suited to mild violence outweighs the suffering caused by lack of physicality and possible neuroticism (?) due to underexposure in people well-suited to mild violence.

Comment by feepingcreature on The Craft & The Community - A Post-Mortem & Resurrection · 2018-04-26T17:36:41.676Z · score: 3 (2 votes) · LW · GW

That's really shitty.

Comment by FeepingCreature on [deleted post] 2018-03-21T18:49:27.819Z

Yeah but you can't derive fault from property, because by your own admission your model makes no claim of fault. At most you can say that Alex is the immediate causal source of the problem.

Comment by feepingcreature on Hero Licensing · 2017-11-25T00:45:51.979Z · score: 5 (3 votes) · LW · GW

One thing I think is probably true: unless you're unusually competent or unusually driven, you should look for areas where you can "charge ahead with reckless abandon", and you should try to cultivate a sense of quality in those areas, possibly by transfer learning; then, having done so, you can exploit your ability to charge ahead in that direction to effectively build skill. In this model, directions to charge in are not something you can reasonably choose, but rather become available to you through chance. If this holds, not everyone can be "Eliezer writing books on rationality", but I think a lot more people can be "Eliezer writing HPMoR". I don't think those things are unknowable from the outside view; I think you just look for "has a good sense of quality" and "is charging recklessly ahead". At least for learnable skills, I believe that consumptive volume plus self-criticism plus productive volume = quality.

(Forcing yourself to charge in an arbitrary direction is, I suspect, a field that's genuinely inexhaustible because if you could repeatably do that, you'd own the self-help sector.)

Comment by FeepingCreature on [deleted post] 2017-05-31T11:41:37.155Z

Five years ago, or even two, my opinion would have been quite different. By this point, however, I have undergone a fairly massive update in the direction of thinking people are far, far too sensitive about matters of "tone" and the like.

Yeah but exposure therapy doesn't work like that though. If people are too sensitive, you can't just rub their faces in the thing they're sensitive about and expect them to change. In fact, what you'd want to desensitize people is the exact opposite - really tight conversation norms that still let people push slightly outside their comfort zone.

Comment by feepingcreature on Allegory On AI Risk, Game Theory, and Mithril · 2017-02-16T01:44:21.955Z · score: 0 (0 votes) · LW · GW

Azathoth, check.

Comment by feepingcreature on CFAR’s new focus, and AI Safety · 2016-12-03T13:58:39.397Z · score: 6 (6 votes) · LW · GW

I shall preface by saying that I am neither a rationalist nor an aspiring rationalist. Instead, I would classify myself as a "rationality consumer" - I enjoy debating philosophy and reading good competence/insight porn. My life is good enough that I don't anticipate much subjective value from optimizing my decisionmaking.

I don't know how representative I am. But I think if you want to reach "people who have something to protect" you need to use different approaches from "people who like competence porn", and I think while a site like LW can serve both groups we are to some extent running into issues where we may have a population that is largely the latter instead of the former - people admire Gwern, but who wants to be Gwern? Who wants to be like Eliezer or lukeprog? We may not want leaders, but we don't even have heroes.

I think possibly what's missing, and this is especially relevant in the case of CFAR, is a solid, empirical, visceral case for the benefit of putting the techniques into action. At the risk of being branded outreach, and at the very real risk of significantly skewing their post-workshop stats gathering, CFAR should possibly put more effort into documenting stories of success through applying the techniques. I think the main focus of research should be full System-1 integration, not just for the techniques themselves but also for CFAR's advertisement. I believe it's possible to do this responsibly if one combines it with transparency and System-2 relevant statistics. Contingent, of course, on CFAR delivering the proportionate value.

I realize that there is a chicken-and-egg problem here where for reasons of honesty, you want to use System-1-appealing techniques that only work if the case is solid, which is exactly the thing that System-1 is traditionally bad at! I'm not sure how to solve that, but I think it needs to be solved. To my intuition, rationality won't take off until it's value-positive for S1 as well as S2. If you have something to protect you can push against S1 in the short-term, but the default engagement must be one of playful ease if you want to capture people in a state of idle interest.

Comment by feepingcreature on On the importance of Less Wrong, or another single conversational locus · 2016-11-29T12:28:47.062Z · score: 1 (1 votes) · LW · GW

Would you use the LW comments section if it was embeddable, like Disqus is?

Comment by feepingcreature on Identity map · 2016-09-09T09:39:40.120Z · score: 0 (0 votes) · LW · GW

The past doesn't exist either.

Comment by feepingcreature on If MWI is correct, should we expect to experience Quantum Torment? · 2016-07-25T08:04:41.352Z · score: 0 (0 votes) · LW · GW

"Hell of a scary afterlife you got here, missy."

! ! !

Be honest. Are you prescient? And are you using your eldritch powers to troll us?

Comment by feepingcreature on [LINK] Why Cryonics Makes Sense - Wait But Why · 2016-03-27T22:42:07.928Z · score: 1 (1 votes) · LW · GW

Tegmark 4 is not related to quantum physics. Quantum physics does not give an avenue for rescue simulations; in fact, it makes them harder.

As a simulationist, you can somewhat salvage traditional notions of fear if you retreat into a full-on absurdist framework where the point of your existence is to give a good showing to the simulating universes; alternately, risk avoidance is a good Schelling point for a high score. Furthermore, no matter how much utility you will be able to attain in Simulationist Heaven, this is your single shot to attain utility on Earth, and you shouldn't waste it.

It does take the sting off death though, and may well be maladaptive in that sense. That said - it seems plausible a lot of simulating universes would end up with a "don't rescue suicides" policy, purely out of a TDT desire to avoid the infinite-suicidal-regress loop.

I am continuously amused how catholic this cosmology ends up by sheer logic.

Comment by feepingcreature on Why CFAR? The view from 2015 · 2015-12-25T00:55:13.243Z · score: 0 (2 votes) · LW · GW

book

Basically, systems that can improve from damage.

Comment by feepingcreature on Leaving LessWrong for a more rational life · 2015-05-22T14:49:03.996Z · score: 0 (0 votes) · LW · GW

Not sure. Suspect nobody knows, but seems possible?

I think the most instructive post on this is actually Three Worlds Collide, for making a strong case for the arbitrary nature of our own "universal" values.

Comment by feepingcreature on Leaving LessWrong for a more rational life · 2015-05-22T11:06:03.373Z · score: 9 (9 votes) · LW · GW

Intelligence to what purpose?

Nobody's saying AI will be human without humor, joy, etc. The point is AI will be dangerous, because it'll have those aspects of intelligence that make us powerful, without those that make us nice. Like, that's basically the point of worrying about UFAI.

Comment by feepingcreature on The Hardcore AI Box Experiment · 2015-04-03T22:37:11.517Z · score: 1 (1 votes) · LW · GW

I suspect that seeing the logs would have made Eliezer seem like a horrible human being. Most people who hear of AI Box imagine a convincing argument, when to me it seems more plausible to exploit issues in people's sense of narrative or emotion.

Comment by feepingcreature on Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 104 · 2015-02-17T08:12:37.803Z · score: 0 (0 votes) · LW · GW

Unrelated conclusion: fursonas are HP canon.

Comment by feepingcreature on Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 104 · 2015-02-17T08:09:14.288Z · score: 2 (2 votes) · LW · GW

I'm still really curious how the Deathly Hallows are going to tie into this.

Okay. Hm. I think maybe you can't transfigure Hermione into Hermione if you don't have a true image of what Hermione was like. But if you had the Resurrection Stone, maybe you could use it to create a true image to work from?

No idea about the wand/cloak.

Comment by feepingcreature on Rationality Quotes Thread February 2015 · 2015-02-07T20:00:20.120Z · score: 2 (2 votes) · LW · GW

Haha. True!

Comment by feepingcreature on [LINK] Wait But Why - The AI Revolution Part 2 · 2015-02-07T17:18:18.193Z · score: 0 (0 votes) · LW · GW

I think the idea is, you need to solve the wireheading for any sort of self-improving AI. You don't have an AI catastrophe without that, because you don't have an AI without that (at least not for long).

Comment by feepingcreature on [LINK] Wait But Why - The AI Revolution Part 2 · 2015-02-06T13:37:24.037Z · score: 0 (0 votes) · LW · GW

It wouldn't.

But I think this is such a basic failure mechanism that I don't believe an AI could get to superintelligence without somehow valuing the accuracy and completeness of its model.

Solving this problem - somehow! - is part of the "normal" development of any self-improving AI.

Though note that a reward maximizing AI could still be an existential risk by virtue of turning the entire universe into a busy-beaver counter for its reward. Though this presumes it can't just set reward to float.infinity.

Comment by feepingcreature on [LINK] Wait But Why - The AI Revolution Part 2 · 2015-02-06T13:30:13.679Z · score: 1 (1 votes) · LW · GW

whatever terminal goal you've given it isn't actually terminal.

This is a contradiction in terms.

If you have given it a terminal goal, that goal is now a terminal goal for the AI.

You may not have intended it to be a terminal goal for the AI, but the AI cares about that less than it does about its terminal goal. Because it's a terminal goal.

If the AI could realize that its terminal goal wasn't actually a terminal goal, all it'd mean would be that you failed to make it a terminal goal for the AI.

And yeah, reinforcement based AIs have flexible goals. That doesn't mean they have flexible terminal goals, but that they have a single terminal goal, that being "maximize reward". A reinforcement AI changing its terminal goal would be like a reinforcement AI learning to seek out the absence of reward.

Comment by feepingcreature on Rationality Quotes Thread February 2015 · 2015-02-06T13:25:13.236Z · score: 3 (5 votes) · LW · GW

Yeah but it's also easy to falsely label a genuine problem as "practically already solved". The proof is in the pudding.

The next day, the novice approached Ougi and related the events, and said, "Master, I am constantly consumed by worry that this is all really a cult, and that your teachings are only dogma." Ougi replied, "If you find a hammer lying in the road and sell it, you may ask a low price or a high one. But if you keep the hammer and use it to drive nails, who can doubt its worth?"

--Two Cult Koans

Conversely, to show the worth of clarity you actually have to go drive some nails with it.

Comment by feepingcreature on Rationality Quotes Thread February 2015 · 2015-02-01T20:22:02.364Z · score: 6 (6 votes) · LW · GW

I need to stop being surprised at how many problems can be solved with clarity alone.

Note to Scott: a problem only counts as solved when it's actually gone.

Comment by feepingcreature on [LINK] The P + epsilon Attack (Precommitment in cryptoeconomics) · 2015-01-29T11:10:02.946Z · score: 1 (1 votes) · LW · GW

Weird question: superrationally speaking, wouldn't the "correct" strategy be to switch to B with 0.49 probability? (Or with however much is needed to ensure that if everybody does this, A probably still wins)

[edit] Hm. If B wins, this strategy halves the expected payoff. So you'd have to account for the possibility of B winning accidentally. Seems to depend on the size of the player base - the larger it is, the closer you can drive your probability to 0.5? (at the limit, 0.5-e?) Not sure. I guess it depends on the size of the attacker's epsilon as well.

I'm sure there's some elegant formula here, but I have no idea what it is.