strangepoop's Shortform 2019-09-10T17:04:58.375Z · score: 3 (1 votes)


Comment by strangepoop on Open & Welcome Thread – October 2020 · 2020-10-04T05:14:08.063Z · score: 3 (2 votes) · LW · GW

As a meta-example, even to this I want to add:

  • There's this other economy to keep in mind of readers scrolling past walls of text. Often, I can and want to make what I'm saying cater to multiple attention spans (a la arbital?), and collapsed-by-default comments allow the reader to explore at will.
    • A strange worry (that may not be true for other people) is attempting to contribute to someone else's long thread or list feels a little uncomfortable/rude without reading it all/carefully. With collapsed-by-default, you could set up norms that it's okay to reply without engaging deeply.
  • It would be nice to have collapsing as part of the formatting
  • With this I already feel like I'm setting up a large-ish personal garden that would inhibit people from engaging in this conversation even if they want to, because there's so much going on.
    • And I can't edit this into my previous comment without cluttering it.
  • There's obviously no need for having norms of "talking too much" when it's decoupled from the rest of the control system
    • I do remember Eliezer saying in a small comment somewhere long ago that "the thumb rule is to not occupy more than three places in the Recent Comments page" (paraphrased).
Comment by strangepoop on Open & Welcome Thread – October 2020 · 2020-10-04T04:45:11.028Z · score: 3 (2 votes) · LW · GW
  • I noticed a thing that might hinder the goals of longevity as described here ("build on what was already said previously"): it feels like a huge cost to add a tiny/incremental comment to something because of all the zero-sum attention games it participates in. 

    It would be nice to do a silent comment, which:

    • Doesn't show up in Recent Comments
    • Collapsed by default
    • (less confident) Doesn't show up in author's notifications (unless "Notify on Silent" is enabled in personal settings)
    • (kinda weird) Comment gets appended automatically to previous comment (if yours) in a nice, standard format.
  • The operating metaphor is to allow the equivalent of bulleted lists to span across time, which I suppose would mostly be replies to yourself.
    • It feels strange to keep editing one comment, and too silent. Also disrupts flow for readers.
  • I don't see often that people have added several comments (via edit or otherwise) across months, or even days. Yet people seem to use a lot of nested lists here. Hard to believe that those list-erious ways go away if spread out in time.
Comment by strangepoop on Some thoughts on criticism · 2020-09-18T14:21:33.090Z · score: 2 (2 votes) · LW · GW
Often, people like that will respond well to criticism about X and Y but not about Z.

One (dark-artsy) aspect to add here is that the first time you ask somebody for criticism, you're managing more than your general identity, you're also managing your interaction norms with that person. You're giving them permission to criticize you (or sometimes, even think critically about you for the first time), creating common knowledge that there does exist a perspective from which it's okay/expected for them to do that. This is playing with the charity they normally extend to you, which might mean that your words and plans will be given less attention than before, even though there might not be any specific criticism in their head. This is especially relevant for low-legibility/fluid hierarchies, which might collapse and impede functioning from the resulting misalignment, perhaps not unlike your own fears of being "crushed", but at the org level.

Although it's usually clear that you'd want to get feedback rather than manage this (at least, I think so), it's important to notice as one kind of anxiety surrounding criticism. This is separate from any narcissistic worries about status, it can be a real systemic worry when you're acting prosocially.

Comment by strangepoop on Invisible Frameworks · 2020-09-16T13:51:53.207Z · score: 1 (1 votes) · LW · GW
Incidentally Eliezer, is this really worth your time?

This comment might have caused a tremendous loss of value, if Eliezer took Marcello's words seriously here and so stopped talking about his metaethics. As Luke points out here, despite all the ink spilled, very few seemed to have gotten the point (at least, from only reading him).

I've personally had to re-read it many times over, years apart even, and I'm still not sure I fully understand it. It's also been the most personally valuable sequence, the sole cause of significant fundamental updates. (The other sequences seemed mostly obvious --- which made them more suitable as just incredibly clear references, sometimes if only to send to others.)

I'm sad that there isn't more.

Comment by strangepoop on Sunday August 23rd, 12pm (PDT) – Double Crux with Buck Shlegeris and Oliver Habryka on Slow vs. Fast AI Takeoff · 2020-09-10T12:05:32.462Z · score: 2 (2 votes) · LW · GW


I've read/heard a lot about double crux but never had the opportunity to witness it.

EDIT: I did find one extensive example, but this would still be valuable since it was a live debate.

Comment by strangepoop on Toolbox-thinking and Law-thinking · 2020-09-05T20:41:26.815Z · score: 3 (2 votes) · LW · GW

This one? From the CT-thesis section in A first lesson in meta-rationality.

the objection turns partly on the ambiguity of the terms “system” and “rationality.” These are necessarily vague, and I am not going to give precise definitions. However, by “system” I mean, roughly, a set of rules that can be printed in a book weighing less than ten kilograms, and which a person can consciously follow.11 If a person is an algorithm, it is probably an incomprehensibly vast one, which could not written concisely. It is probably also an incomprehensibly weird one, which one could not consciously follow accurately. I say “probably” because we don’t know much about how minds work, so we can’t be certain.
What we can be certain is that, because we don’t know how minds work, we can’t treat them as systems now. That is the case even if, when neuroscience progresses sufficiently, they might eventually be described that way. Even if God told us that “a human, reasoning meta-systematically, is just a system,” it would be useless in practice. Since we can’t now write out rules for meta-systematic reasoning in less than ten kilograms, we have to act, for now, as if meta-systematic reasoning is non-systematic.
Comment by strangepoop on strangepoop's Shortform · 2020-07-25T20:30:18.902Z · score: 1 (1 votes) · LW · GW

Ideally, I'd make another ninja-edit that would retain the content in my post and the joke in your comment in a reflexive manner, but I am crap at strange loops.

Comment by strangepoop on strangepoop's Shortform · 2020-07-25T20:02:42.734Z · score: 1 (1 votes) · LW · GW

Cold Hands Fallacy/Fake Momentum/Null-Affective Death Stall

Although Hot Hands has been the subject of enough controversy to perhaps no longer be termed a fallacy, there is a sense in which I've fooled myself before with a fake momentum. I mean when you change your strategy using a faulty bottomline: incorrectly updating on your current dynamic.

As a somewhat extreme but actual example from my own life: when filling out answersheets to multiple-choice questions (with negative marks for incorrect responses) as a kid, I'd sometimes get excited about having marked almost all of the questions near the end, and then completely, obviously, irrationally decide to mark them all. This was out of some completion urge, and the positive affect around having filled in most of them. This involved a fair bit of self-deception to carry out, since I was aware at some level that I left some of them previously unanswered because I was in fact unsure, and to mark them I had to feel sure.

Now, for sure you could make the case that maybe there are times when you're thinking clearer and when you know the subject or whatever, where you can additionally infer this about yourself correctly and then rationally ramp up the confidence (even if slight) in yourself. But this wasn't one of those cases, it was the simple fact that I felt great about myself.

Anyway the real point of this post is that there's a flipside (or straightforward generalization) of this: we can talk about this fake inertia for subjects at rest or at motion. What I mean is there's this similar tendency to not feel like doing something because you don't have that dynamic right now, hence all the clichés of the form "first blow is half the battle". In a sense, that's all I'm communicating here, but seeing it as a simple irrational mistake (as in the example above) really helped me get over this without drama: just remind yourself of the bottomline and start moving in the correct flow, ignoring the uncalibrated halo (or lack thereof) of emotion.

Comment by strangepoop on (answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)? · 2020-07-07T15:48:22.056Z · score: 11 (3 votes) · LW · GW

There's a whole section on voting in the LDT For Economists page on Arbital. Also see the one for analytic philosophers, which has a few other angles on voting.

From what I can tell from your other comments on this page, you might already have internalized all the relevant intuitions, but it might be useful anyway. Superrationality is also discussed.

Sidenote: I'm a little surprised no one else mentioned it already. Somehow arbital posts by Eliezer aren't considered as canon as the sequences, maybe it's the structure (rather than just the content)?

Comment by strangepoop on Conversation Halters · 2020-06-28T12:20:32.981Z · score: 1 (1 votes) · LW · GW

I usually call this lampshading, and I'll link this comment to explain what I mean. Thanks!

Comment by strangepoop on How to evaluate (50%) predictions · 2020-05-13T01:30:15.680Z · score: 5 (3 votes) · LW · GW

Thank you for this comment. I went through almost exactly the same thing, and might have possibly tabled it at the "I am really confused by this post" stage had I not seen someone well-known in the community struggle with and get through it.

My brain especially refused to read past the line that said "pushing it to 50% is like throwing away information": Why would throwing away information correspond to the magic number 50%?! Throwing away information brings you closer to maxent, so if true, what is it about the setup that makes 50% the unique solution, independent of the baseline and your estimate? That is, what is the question?

I think it's this: in a world where people can report the probability for a claim or the negation of it, what is the distribution of probability-reports you'd see?

By banning one side of it as Rafael does, you get it to tend informative. Anyway, this kind of thinking makes it seem like it's a fact about this flipping trick and not fundamental to probability theory. I wonder if there are more such tricks/actual psychology to adjust for to get a different answer.

Comment by strangepoop on Is Rationalist Self-Improvement Real? · 2019-12-12T12:42:10.952Z · score: 12 (5 votes) · LW · GW

While you're technically correct, I'd say it's still a little unfair (in the sense of connoting "haha you call yourself a rationalist how come you're failing at akrasia").

Two assumptions that can, I think you'll agree, take away from the force of "akrasia is epistemic failure":

  • if modeling and solving akrasia is, like diet, a hard problem that even "experts" barely have an edge on, and importantly, things that do work seem to be very individual-specific making it quite hard to stand on the shoulders of giants
  • if a large percentage of people who've found and read through the sequences etc have done so only because they had very important deadlines to procrastinate

...then on average you'd see akrasia over-represented in rationalists. Add to this the fact that akrasia itself makes manually aiming your rationality skills at what you want harder. That can leave it stable even under very persistent efforts.

Comment by strangepoop on Sayan's Braindump · 2019-11-24T07:39:50.491Z · score: 1 (1 votes) · LW · GW

I'm interested in this. The problem is that if people consider the value provided by the different currencies at all fungible, side markets will pop up that allow their exchange.

An idea I haven't thought about enough (mainly because I lack expertise) is to mark a token as Contaminated if its history indicates that it has passed through "illegal" channels, ie has benefited someone in an exchange not considered a true exchange of value, and so purists can refuse to accept those. Purist communities, if large, would allow stability of such non-contaminated tokens.

Maybe a better question to ask is "do we have utility functions that are partial orders and thus would benefit from many isolated markets?", because if so, you wouldn't have to worry about enforcing anything, many different currencies will automatically come into existence and be stable.

Of course, more generally, you wouldn't quite have complete isolation, but different valuations of goods in different currencies, without "true" fungibility. I think it is quite possibe that our preference orderings are in fact partial and the current one-currency valuation of everything might be improved.

Comment by strangepoop on strangepoop's Shortform · 2019-11-20T17:52:35.633Z · score: 6 (4 votes) · LW · GW

The expectations you do not know you have control your happiness more than you know. High expectations that you currently have don't look like high expectations from the inside, they just look like how the world is/would be.

But "lower your expectations" can often be almost useless advice, kind of like "do the right thing".

Trying to incorporate "lower expectations" often amounts to "be sad". How low should you go? It's not clear at all if you're using territory-free un-asymmetric simple rules like "lower". Like any other attempt at truth-finding, it is not magic. It requires thermodynamic work.

The thing is, the payoff is rather amazing. You can just get down to work. As soon as you're free of a constant stream of abuse from beliefs previously housed in your head, you can Choose without Suffering.

The problem is, I'm not sure how to strategically go about doing this, other than using my full brain with Constant Vigilance.

Coda: A large portion of the LW project (or at least, more than a few offshoots) is about noticing you have beliefs that respond to incentives other than pure epistemic ones, and trying not to reload when shooting your foot off with those. So unsurprisingly, there's a failure mode here: when you publicly declare really low expectations (eg "everyone's an asshole"), it works to challenge people, urges them to prove you wrong. It's a cool trick to win games of Chicken but as usual, it works by handicapping you. So make sure you at least understand the costs and the contexts it works in.

Comment by strangepoop on [deleted post] 2019-09-26T09:34:39.969Z

I think a counterexample to "you should not devote cognition to achieving things that have already happened" is being angry at someone who has revealed they've betrayed you, which might acause them to not have betrayed you.

Comment by strangepoop on strangepoop's Shortform · 2019-09-10T17:04:58.545Z · score: 5 (3 votes) · LW · GW

Is metarationality about (really tearing open) the twelfth virtue?

It seems like it says "the map you have of map-making is not the territory of map-making", and gets into how to respond to it fluidly, with a necessarily nebulous strategy of applying the virtue of the Void.

(this is also why it always felt like metarationality seems to only provide comments where Eliezer would've just given you the code)

The parts that don't quite seem to follow is where meaning-making and epistemology collide. I can try to see it as a "all models are false, some models are useful" but I'm not sure if that's the right perspective.

Comment by strangepoop on If physics is many-worlds, does ethics matter? · 2019-07-28T09:05:50.460Z · score: 1 (1 votes) · LW · GW

I want to ask this because I think I missed it the first few times I read Living in Many Worlds: Are you similarly unsatisfied with our response to suffering that's already happened, like how Eliezer asks, about the twelfth century? It's boldface "just as real" too. Do you feel the same "deflation" and "incongruity"?

I expect that you might think (as I once did) that the notion of "generalized past" is a contrived but well-intentioned analogy to manage your feelings.

But that's not so at all: once you've redone your ontology, where the naive idea of time isn't necessarily a fundamental thing and thinking in terms of causal links comes a lot closer to how reality is arranged, it's not a stretch at all. If anything, it follows that you must try and think and feel correctly about the generalized past after being given this information.

Of course, you might modus tollens here.

Comment by strangepoop on Go Do Something · 2019-05-21T17:45:46.082Z · score: 5 (4 votes) · LW · GW

Soares also did a good job of impressing this in Dive In:

In my experience, the way you end up doing good in the world has very little to do with how good your initial plan was. Most of your outcome will depend on luck, timing, and your ability to actually get out of your own way and start somewhere. The way to end up with a good plan is not to start with a good plan, it's to start with some plan, and then slam that plan against reality until reality hands you a better plan.

The idea doesn't have to be good, and it doesn't have to be feasible, it just needs to be the best incredibly concrete plan that you can come up with at the moment. Don't worry, it will change rapidly when you start slamming it into reality. The important thing is to come up with a concrete plan, and then start executing it as hard as you can — while retaining a reflective state of mind updating in the face of evidence.
Comment by strangepoop on The concept of evidence as humanity currently uses it is a bit of a crutch. · 2019-05-21T17:33:41.603Z · score: 2 (2 votes) · LW · GW

I don't think the "idea of scientific thinking and evidence" has so much to do with throwing away information as adding reflection, post which you might excise the cruft.

Being able to describe what you're doing, ie usefully compress existing strategies-in-use, is probably going to be helpful regardless of level of intelligence because it allows you to cheaply tweak your strategies when either the situation or the goal is perturbed.

Comment by strangepoop on The Cacophony Hypothesis: Simulation (If It is Possible At All) Cannot Call New Consciousnesses Into Existence · 2019-04-15T21:19:13.437Z · score: 1 (1 votes) · LW · GW

To further elaborate 4: your example of the string "1" being a conscious agent because you can "unpack" it into an agent really feels like it shouldn't count: you're just throwing away the "1" and replaying a separate recording of something that was conscious. This sounds about as much of a non-sequitur as "I am next to this pen, so this pen is conscious".

We could, however, make it more interesting by making the computation depend "crucially" on the input. But what counts?

Suppose I have a program that turns noise into a conscious agent (much like generative models can turn a noise vector into a face, say). If we now seed this with a waterfall, is the waterfall now a part of the computation, enough to be granted some sentience/moral patienthood? I think the usual answer is "all the non-trivial work is being done by the program, not the random seed", as Scott Aaronson seems to say here. (He also makes the interesting claim of "has to participate fully in the arrow of time to be conscious", which would disqualify caching and replaying.)

But this can be made a little more confusing, because it's hard to tell which bit is non-trivial from the outside: suppose I save and encrypt the conscious-generating-program. This looks like random noise from the outside, and will pass all randomness tests. Now I have another program with the stored key decrypt it and run it. From the outside, you might disregard the random-seed-looking-thingy and instead try to analyze the decryption program, thinking that's where the magic is.

I'd love to hear about ideas to pin down the difference between Seeding and Decrypting in general, for arbitrary interpretations. It seems within reach, and like a good first step, since the two lie on roughly opposite ends of a spectrum of "cruciality" when the system breaks down into two or more modules.

Comment by strangepoop on The Cacophony Hypothesis: Simulation (If It is Possible At All) Cannot Call New Consciousnesses Into Existence · 2019-04-15T21:13:08.842Z · score: 1 (1 votes) · LW · GW

Responses to your four final notes:

1. This is, as has been remarked in another comment, pretty much Dust theory. See also Moravec's concise take on the topic, referenced in the Dust theory FAQ. Doing a search for it on LW might also prove helpful for previous discussions.

2. "that was already there"? What do you mean by this? Would you prefer to use the term 'magical reality fluid' instead of "exists"/"extant"/"real"/"there" etc, to mark your confusion about this? If you instead feel like you aren't confused about these terms, please provide (a link to) a solution. You can find the problem statement in The Anthropic Trilemma.

3. Eliezer deals with this using average utilitarianism, depending on whether or not you agree with rescuability (see below).

4. GAZP vs GLUT talks about the difference between a cellphone transmitting information of consciousness vs the actual conscious brain on the other end, and generalizes it to arbitrary "interpretations". That is, there are parts of the computation that are merely "interpreting", informing you about consciousness and others that are "actually" instantiating. It may not be clear what exactly the crucial difference is yet, but I think it might be possible to rescue the difference, even if you can construct continuums to mess with the notion. This is of course deeply tied to 2.


It may seem that my takeaway from your post is mostly negative, this is not the case. I appreciate this post, it was very well organized despite tackling some very hairy issues, which made it easier to respond to. I do feel like LW could solve this somewhat satisfactorily, perhaps some people already have and don't bother pointing the rest of us/are lost in the noise?

Comment by strangepoop on Epistemic Tenure · 2019-03-07T11:29:57.774Z · score: 1 (1 votes) · LW · GW

it is not as though rationality consisted of some singular epistemesis score that can be raised or lowered

I feel like this is fighting the hypothesis. As Garrabrant says:

Attention is a conserved resource, and attention that I give to Bob is being taken away from attention that could be directed toward GOOD ideas.

It doesn't matter whether or not you think it is possible to track rationality through some singular epistemesis score. The question is: you have limited attentional resources and the problem OP outlined; "rationality" is probably complicated; what do you do anyway?

How you divvy them is the score. Or, to replace the symbol with the substance: if you're in charge of divvying those resources, then your particular algorithm will decide what your underlings consider status/currency, and can backpropagate into their minds.

Comment by strangepoop on Out to Get You · 2019-03-05T22:04:04.207Z · score: 1 (1 votes) · LW · GW

Maybe you meant "cutting corners" rather than cutting corners? ie you did understand the distinction between the thing and the appearance of the thing, you just forgot to add the quotes.

Comment by strangepoop on Epistemic Tenure · 2019-03-04T11:00:05.776Z · score: 1 (1 votes) · LW · GW

I think your "attentional resources" are just being Counterfactually Mugged here, so if you're okay with that, you ought to be okay with some attention being diverted away from "real" ideas, if you're reasonably confident in your construction of the counterfactual "Bob’s idea might HAVE BEEN good".

This way of looking at it also says that tenure is a bad metaphor: your confidence in the counterfactual being true can change over time.

(If you then insist that this confidence in your counterfactual is also something that affects Bob, which it kinda does, then I'm afraid we're encountering an instance of unfair problem class in the wild and I don't know what to do)

As an aside, this makes me think: What happens when all consumers in the market are willing to get counterfactually mugged? Where I'm not able to return my defected phone because prediction markets said it would have worked? I suppose this is not very different from the concept of force majeure, only systematized.

Comment by strangepoop on Unconscious Economics · 2019-02-27T15:10:28.938Z · score: 29 (14 votes) · LW · GW

It's worth noting that David Friedman's Price Theory clearly states this in the very first chapter, just three paragraphs down:

The second half of the assumption, that people tend to find the correct way to achieve their objectives, is called rationality. This term is somewhat deceptive, since it suggests that the way in which people find the correct way to achieve their objectives is by rational analysis--analyzing evidence, using formal logic to deduce conclusions from assumptions, and so forth. No such assumption about how people find the correct means to achieve their ends is necessary.

One can imagine a variety of other explanations for rational behavior. To take a trivial example, most of our objectives require that we eat occasionally, so as not to die of hunger (exception--if my objective is to be fertilizer). Whether or not people have deduced this fact by logical analysis, those who do not choose to eat are not around to have their behavior analyzed by economists. More generally, evolution may produce people (and other animals) who behave rationally without knowing why. The same result may be produced by a process of trial and error; if you walk to work every day, you may by experiment find the shortest route even if you do not know enough geometry to calculate it. Rationality in this sense does not necessarily require thought. In the final section of this chapter, I give two examples of things that have no minds and yet exhibit rationality.

I don't think it counts as a standard textbook, but it is meant to be a textbook.

On the whole, I think it's perfectly okay for economists to mostly ignore how the equilibrium is achieved, since like you pointed out, there are so many juicy results popping out from just the fact that they are achieved on average.

Also, I enjoyed the examples in your post!

Comment by strangepoop on Humans Who Are Not Concentrating Are Not General Intelligences · 2019-02-25T21:35:18.842Z · score: 1 (1 votes) · LW · GW

effortless pattern-recognition is what machine learning can do today, while effortful attention, and explicit reasoning (which seems to be a subset of effortful attention) is generally beyond ML’s current capabilities.

Just to be clear, are you or aren't you (or neither) saying that this is only a matter of scale?

It seems to me like you're saying it could indeed only be a matter of scale, we're just in the stage of figuring out what the right dimension to amp up is ("be coherent for longer").

Comment by strangepoop on Double-Dipping in Dunning--Kruger · 2018-11-28T08:58:06.957Z · score: 4 (4 votes) · LW · GW

See, the problem is now that I've also internalized this (seemingly true) lesson, the +15% might double-boost my ass numbers.

But maybe if we accumulate enough lessons we can get increasingly close to the truth by adding these "higher order terms"?

I don't think so - the error bars do not necessarily diminish. For example:

  • Ass number for drawing ability percentile: ~70%
  • Dunning Kruger correction: ~50%
  • Double-dip correction: ~65%

Did I do it right? I have no idea. Every step might have already been taken into account in the first asstimate. Every system-2 patch that we discover might have immediately patched system-1.

One (admittedly lazy) way out is to chuck all context-sensitive formal rules like 'add/subtract X%' and leave it entirely to system-1: play calibration games for skill-percentiles.

Comment by strangepoop on If You Want to Win, Stop Conceding · 2018-11-26T12:33:46.275Z · score: 9 (2 votes) · LW · GW

I hope we haven't forgotten Stuck in the Middle With Bruce and Soares' Have No Excuses, which starts with a quote from Bonds That Make Us Free.

I think one reason people end up using a minimax strategy is that it's just easier to compute than EV-maximization.

But more importantly, it just feels like there's no downsides - it's free insurance!

If you want to have a convincing excuse however, you might actively impede your chances. (It might be possible to distance yourself from the excuse so the insurance is actually free, but I think this is unlikely/hard.)

If you've already hedged so hard that you've bet a lot against yourself, you might have sufficiently changed the payoffs to make losing rational, especially if you also add a penalty to Being Mediocre. This is why post like this are needed.

Comment by strangepoop on Unrolling social metacognition: Three levels of meta are not enough. · 2018-08-30T07:39:30.115Z · score: 5 (5 votes) · LW · GW

Somehow this comment was really inspiring! I'm glad this exchange happened, so maybe I should upvote grandparent too? :P



[not] acknowledging that your first concern had been addressed

We have terms for these! They are, respectively, stonewalling and logical rudeness.

I'm still split on how I feel about jargon, and of course it's good that you didn't use any here, but it does give the concepts you describe some legitimacy (for better or worse). Legitimacy helps especially in cases where such expressions are dismissed as over-reactions unique to you, and are thus assumed to be your responsibility to fix, by some implicit jargon-efficiency argument ("if this were a thing to be concerned about, we'd have a name for it!").

Comment by strangepoop on Simplicio and Sophisticus · 2018-07-22T21:53:52.909Z · score: 3 (3 votes) · LW · GW

"... natural science has shown a curious mixture of rationalism and irrationalism. Its prevalent tone of thought has been ardently rationalistic within its own borders, and dogmatically irrational beyond those borders. In practice such an attitude tends to become a dogmatic denial that there are any factors in the world not fully expressible in terms of its own primary notions devoid of further generalization. Such a denial is the self-denial of thought."

- A.N. Whitehead, Process and Reality

I can't really tell yet, but David Chapman's work seems to be trying to hint at this phenomenon all the time. See his How to Think Real Good, for example, even if you don't agree with his characterization of Bayesian rationality. There's also Fixation and Denial, where he goes into some failure modes when dealing with hard-to-fully-formalize things. Meta-rationality seems to be mostly about this, AFAICT.

I have to say, most of Chapman's stuff feels like pure lampshading, ie acknowledging that there is a problem and then simply moving on. I suppose he's building up to more practical advice.

If you're getting frustrated (I certainly am) that all everyone seems to be doing about this is offering loose and largely unhelpful tips, I think that's something Alan Perlis anticipated: "One can't proceed from the informal to the formal by formal means."

(of course, that's just another restatement of the fact that there is a problem.)

Comment by strangepoop on Osmosis learning: a crucial consideration for the craft · 2018-07-18T06:19:50.485Z · score: 3 (2 votes) · LW · GW

See also: "show, don't tell"/the iceberg theory in writing and the monad tutorial fallacy in functional programming. These are weakish evidence for the existence of this phenomenon, although they still reside in the lingual realm.

[posting a double comment because it is sufficiently different and the previous one is already too long]

Comment by strangepoop on Sleeping Beauty Resolved? · 2018-07-16T11:04:16.458Z · score: 1 (1 votes) · LW · GW

I'd say your reply is at least a little bit of logical rudeness, but I'll take the "Sure, ...".

I was pointing specifically at the flaw* in bringing up Everett branches into the discussion at all, not about whether the context happened to be changing here.

I wouldn't really mind the logical rudeness (if it is so), except for the missed opportunity of engaging more fully with your fascinating comment! (see also *)

It's also nice to see that the followup to OP starts with a discussion of why it's a good/easy first rule to, like I said, just ban non-timeless propositions, even if we can eventually come with a workable system that deals with it well.

(*) As noted in GP, it's still not clear to me that this is a flaw, only that I couldn't come up with anything in five minutes! Part of the reason I replied was in the hopes that you'd have a strong defense of "everettian-indexicals", because I'd never thought of it that way before!

Comment by strangepoop on Bayesian Probability is for things that are Space-like Separated from You · 2018-07-13T17:58:29.799Z · score: 8 (4 votes) · LW · GW

What rossry said, but also, why do you expect to be "winning" all arms races here? Genes in other people may have led to development of meme-hacks that you don't know are actually giving someone else an edge in a zero sum game.

In particular, they might call you fat or stupid or incompetent and you might end up believing it.

Comment by strangepoop on Mathematical Mindset · 2018-07-13T13:25:30.030Z · score: 4 (3 votes) · LW · GW
For mathematics is not about proofs; it is about definitions. The essence of great mathematics is coming up with a powerful definition that results in short proofs.

Or in software terms, coming up with a powerful and elegant exposed API/top level functions that don't require peeking into the abstraction (which would imply a "longer" "proof", following Curry-Howard).

Comment by strangepoop on An Agent is a Worldline in Tegmark V · 2018-07-13T13:11:35.049Z · score: 4 (3 votes) · LW · GW

I'm a little confused.

It seemed to me that the way Tegmark had put it, level IV is meta-closed: any consistent map of (even possibly eventually inconsistent) maps is still just a consistent map in level IV; it doesn't have to model any particular territory, it just has to be a mathematical "structure". Maybe you're saying that this is actually in level V and my view of level IV is too inclusive (but I think Tegmark would disagree with you, see esp. appendix A of his original paper), or maybe I missed your point altogether.

It's not even clear that there would be a notion of an "agent" in every level IV universe (in fact I'd say it's clear that this is NOT the case), so I think the idea of a worldline between them would not be well-defined. Nevertheless, I'm fine with non-standard uses of terms if it helps communicate the idea you have after you've clarified your usage of them (but don't canonize independently in an already nebulous territory!), but I'm having some trouble with that.

So can you clarify what you mean, particularly by level IV? (reasonably precise english is fine :P)

ETA: Okay, given your Mathematical Mindset post, I'm doubly fine with your redefinition, but I still want it :3

Comment by strangepoop on Bayesian Probability is for things that are Space-like Separated from You · 2018-07-13T12:08:32.568Z · score: 2 (2 votes) · LW · GW

I suppose you mean the fallibility of memory. I think Garrabrant meant it tautologically though (ie, as the definition of "past").

Comment by strangepoop on Bayesian Probability is for things that are Space-like Separated from You · 2018-07-13T11:54:17.605Z · score: 19 (6 votes) · LW · GW
I think the LW zeitgeist doesn't really engage with this.

Really? I feel quite the opposite, unless you're saying we could do still more. I think LW is actually one of the few communities that take this sort of non-dualism/naturalism in arriving at a probabilistic judgement (and all its meta levels) seriously. We've been repeatedly exposed to the fact that Newcomblike problems are everywhere since a long time ago, and then relatively recently, with Simler's wonderful post on crony beliefs (and now, his even more delightful book with Hanson, of course).

ETA: I'm missing quite a few posts that were even older (Wei Dai's? Drescher's? yvain had something too IIRC), it'd be nice if someone else who does remember posted them here.

Comment by strangepoop on A Sarno-Hanson Synthesis · 2018-07-13T10:57:03.638Z · score: 4 (4 votes) · LW · GW

why private? :(

Comment by strangepoop on A Sarno-Hanson Synthesis · 2018-07-13T10:56:18.526Z · score: 8 (2 votes) · LW · GW

Related to the sleep example, because you didn't say exactly this, and it makes a stronger case:

I noticed some time ago that my misery when waking up was a negotiation tactic from when my parents would wake me up. They were nice enough to let me sleep a little longer if I looked sufficiently upset at being woken up. It became obvious recently that I was pattern-matching my alarm to a parent. How do I know this? Because I knew that if I started loudly singing a cheery tune with a smile on my face I'd automatically become less miserable, but I never did, because I didn't want to be less miserable, even though my parents weren't around anymore. I started doing it when I realized this, and it works pretty well!

(There was one more problem, of feeling like I'm manipulating myself, which seems at first to be at odds with building self-loyalty. I think this went away as I got more comfortable with the idea of sometimes "wanting to be manipulated" for my own success, of desiring less freedom (which would be sacrilege to my younger self). Reading about Kegan's model of adult development and experimenting with BDSM helped me get there somehow.)

One problem with applying this thesis (did I mention I wholeheartedly agree with it?) is that it's hard to refrain from inadvertently reinforcing such negotiation tactics when someone else looks miserable (like my parents did), ie ferberization is painful (not to mention patronizing when done to adults). I think it's possible to be honest about it with someone reasonable and smart enough to grasp the subtleties, and then, usually only after they're done having their episode, but there's no good solution to this AFAIK. Else we wouldn't have hard problems of income redistribution either - the problem of helping those who need it without inducing weakness/dependence.

BTW, Is there an economic term for this specific problem?

Comment by strangepoop on Osmosis learning: a crucial consideration for the craft · 2018-07-13T09:51:58.586Z · score: 3 (3 votes) · LW · GW

I love this post! If you've ever noticed that short workshops conducted by an expert seems to teach you a lot more and faster than spending weeks on something despite videos and docs, I think you ought to agree with toonalfrink's thesis.

I especially like the very crisply stated problem of asymmetrically engineering the subsymoblic flow towards "better" equilibria. I'll have fun thinking about that for a while.

(In fact, I'd say the first two 'areas of inquiry' you mention are really just natural subproblems of the third - bandwidth throttling is probably what we'll attempt to bind to "knowledge intelligence" once we know how to quantify and price it.)

A developed theory might also be applied negatively (as in, mainly throttling) to less domain-specific things like containing toxicity and misery, when you have less choice in the company you keep! I have personally tried to model this problem as mutiplayer-meditation: where distracting terrible thoughts can arise also from minds outside your own and you can, with focused practice, decide their influence on you. I think your model captures this more generally, and makes it more of a systematic communal effort.

When you talk about "verbal communication" though, it's not clear whether you're referring only to their attempts at talking about how they got so smart (or rich etc) or if you're also including specific object-level problems that they solve out loud (in a blogpost where they just apply their smarts, say), which could be said to allow for a sort of one-way, low-bandwidth osmosis.

Of course, the problem with a blogpost versus seeing them in-person (or even in-video, or some other neutral feed) is that they are themselves doing the filtering of what's notable - and as we all know around here, people usually have a poor idea of what is and isn't obvious to others. But this might have more to do with memorability than underspecification. I've noticed I often forget certain pieces of advice, but as a human, I tend to have good retention of the mannerisms of people. Another possible cause is the ability to ask questions without trivial inconveniences (like having to wait a long time for an answer, or gesturing at pictures). Yet another one is seeing a live demonstration of things actually working, so you're more likely to try it consistently rather than throwing away the whole thing when it doesn't work the first couple of times.

(Notice that memorability vs underspecification vs interactivity vs demo aren't distinguished when comparing workshops and self-study. Can we think of other factors?)

Anyway, I think we could start measuring this with increasingly-less-than-in-person media to start singling out what the factors really are, so we can continue to avoid meeting real people :P

Comment by strangepoop on Against accusing people of motte and bailey · 2018-06-05T17:06:46.465Z · score: 8 (3 votes) · LW · GW

All the actual problems are located around the tendency to hold every individual who calls themselves an X accountable for (simultaneously) all the opinions ever propounded under the label of X.


Responding to reductions is like responding to insults: if they don't spring out of genuine confusion or you've run out of your good deeds for the day, you don't have to respond.

I mean, if a surefire way to get you to give me information is to hold your reputation hostage, then you're going to be spending an awful lot of time fielding queries from strangers.

Comment by strangepoop on When is unaligned AI morally valuable? · 2018-05-25T18:34:41.357Z · score: 2 (1 votes) · LW · GW

Upvoted, this was exactly my reaction to this post. However, you may want to look at the link to alignment in the OP. Christiano is using "alignment" in a very narrow sense. For example, from the linked post:

The definition is intended de dicto rather than de re. An aligned A is trying to “do what H wants it to do.” Suppose A thinks that H likes apples, and so goes to the store to buy some apples, but H really prefers oranges. I’d call this behavior aligned because A is trying to do what H wants, even though the thing it is trying to do (“buy apples”) turns out not to be what H wants: the de re interpretation is false but the de dicto interpretation is true.

... which rings at least slightly uncomfortable to my ears.

Comment by strangepoop on When is unaligned AI morally valuable? · 2018-05-25T15:45:58.616Z · score: 3 (2 votes) · LW · GW

I'm curious, how does this work out for fellow animals?

ie If you value (human and non-human) animals directly in your utility function, and then you also use UDT, are you not worried about double counting the relevant intuitions, and ending up being too "nice", or being too certain that you should be "nice"?

Perhaps it is arguable that that is precisely what's going on when we end up caring more for our friends and family?

Comment by strangepoop on Sleeping Beauty Resolved? · 2018-05-24T20:32:12.117Z · score: 2 (1 votes) · LW · GW

There is a relevant distinction: the machinery being used (logical assignment) has to be stable for the duration of the proof/computation. Or perhaps, the "consistency" of the outcome of the machinery is defined on such a stability.

For the original example, you'd have to make sure that you finish all relevant proofs within a period in or within a period in . If you go across, weird stuff happens when attempting to preserve truth, so banning non-timeless propositions makes things easier.

You can't always walk around while doing a proof if one of your propositions is "I'm standing on Second Main". You could, however, be standing still in any one place whether or not it is true. ksvanhorn might call this a space parametrization, if I understand him correctly.

So here's the problem: I can't imagine what it would mean to carry out a proof across Everett branches. Each prover would have a different proof, but each one would be valid in its own branch across time (like standing in any one place in the example above).

I think a refutation of that would be at least as bizarre as carrying out a proof across space while keeping time still (note: if you don't keep time still, you're probably still playing with temporal inconsistencies), so maybe come up with a counterexample like that? I'm thinking something along the lines of code=data will allow it, but I couldn't come up with anything.

Comment by strangepoop on Paper Trauma · 2018-02-14T23:51:48.404Z · score: 8 (3 votes) · LW · GW

Have any of you tried Wacom Tablet + Inkscape?

It's (nearly) everything people have asked for above, and my favorite way to think using System 3, as some call it.

I've had a similar love as Critch for large canvases to dump my thoughts in. This usually meant desks in the library that I would messily (and unashamedly) scribble on from top to bottom, then losing them the next day. When I really wanted to save those notes I ended up taking some hasty pictures (that I never revisited because tiny).

But with Inkscape, consider - infinte canvas and resolution, combinations of text, freehand and other vector graphics, zoom & pan for focusing or looking at the big picture, permanent digital storage (and little environmental waste), colors (like others have said, this one changes everything, but especially so without trivial inconveniences and no penalties on experimentation), and my personal favorite - guaranteed neatness, with the ability to group and move objects around. (I really love the fact that all strokes automatically become objects because I find it very difficult to keep tidy while in a manic frenzy; this lets you draw fast and edit later, as in writing.)

The only drawback is that it isn't nearly as portable. And that I can't seem to dot the i's in freehand sometimes because vector pens don't make points.

Comment by strangepoop on Open thread, October 2 - October 8, 2017 · 2017-10-03T22:08:33.539Z · score: 2 (2 votes) · LW · GW

Can someone help me out with Paul Christiano's email/contact info? Couldn't find it anywhere online.

I might be able to discuss possibilities for implementing his Impact Certificate ideas with some very capable people here in India.

Comment by strangepoop on Markets are Anti-Inductive · 2017-09-13T10:14:19.124Z · score: 1 (1 votes) · LW · GW

Is it unfair to say that prediction markets will deal with all of these cases?

I understand that's like responding to "This is a complicated problem that may remain unsolved, it is not clear that we will be able to invent the appropriate math to deal with this." with "But Church-Turing thesis!".

But all I'm saying is that it does apply generally, given the right apparatus.

Comment by strangepoop on The Majority Is Always Wrong · 2017-06-08T15:05:14.396Z · score: 0 (0 votes) · LW · GW

Isn't this just a special case of Berkson's paradox?

Comment by strangepoop on Open thread, May 15 - May 21, 2017 · 2017-05-19T12:59:47.387Z · score: 0 (0 votes) · LW · GW

Exclusion isn't always socially appropriate. If I take a cab home everyday (which I pay for), and a friend can literally take a free ride because her place is on the way, should I "exclude" her if she doesn't want to share the cost? She claims it doesn't cost me extra, I'd be paying for the cab anyway if she lived somewhere else.

But of course I can come up with un-excludable externalities:

I share a house that's in pretty bad shape, and I decide to get some fresh painting done. This is a net benefit to all the housemates, but we would value them differently. I want this slightly more than all the others. So I have to pay the entire amount.

Comment by strangepoop on Open thread, May 15 - May 21, 2017 · 2017-05-18T23:40:32.299Z · score: 1 (1 votes) · LW · GW

Incidentally, Gary Drescher makes the same (citation free) statement in a footnote in Chapter 7 - Deriving Ought from Is:

Utilitarian bases for capitalism—arguments that market forces promote the greatest good—are another matter, best suited for other books. For here, suffice it to note that even in theory, an unconstrained market does not promote the greatest good overall, but rather the greatest good weighted by the participants’ relative wealth.

I remember asking for a reference about a year ago on LWIRC, but that didn't help much.