Posts

Forcing Yourself is Self Harm, or Don't Goodhart Yourself 2021-04-10T15:19:42.130Z
Forcing yourself to keep your identity small is self-harm 2021-04-03T14:03:06.469Z
How I Meditate 2021-03-08T03:34:21.612Z
Bootstrapped Alignment 2021-02-27T15:46:29.507Z
Fake Frameworks for Zen Meditation (Summary of Sekida's Zen Training) 2021-02-06T15:38:17.957Z
The Problem of the Criterion 2021-01-21T15:05:41.659Z
The Teleological Mechanism 2021-01-19T23:58:54.496Z
Zen and Rationality: Karma 2021-01-12T20:56:57.475Z
You are Dissociating (probably) 2021-01-04T14:37:02.207Z
A Model of Ontological Development 2020-12-31T01:55:58.654Z
Zen and Rationality: Skillful Means 2020-11-21T02:38:09.405Z
No Causation without Reification 2020-10-23T20:28:51.831Z
The whirlpool of reality 2020-09-27T02:36:34.276Z
Zen and Rationality: Just This Is It 2020-09-20T22:31:56.338Z
Zen and Rationality: Map and Territory 2020-09-12T00:45:40.323Z
How much can surgical masks help with wildfire smoke? 2020-08-21T15:46:12.914Z
Bayesiance (Filk) 2020-08-18T16:30:00.753Z
Zen and Rationality: Trust in Mind 2020-08-11T20:23:34.434Z
Zen and Rationality: Don't Know Mind 2020-08-06T04:33:54.192Z
Let Your Mind Be Not Fixed 2020-07-31T17:54:43.247Z
[Preprint] The Computational Limits of Deep Learning 2020-07-21T21:25:56.989Z
Comparing AI Alignment Approaches to Minimize False Positive Risk 2020-06-30T19:34:57.220Z
What are the high-level approaches to AI alignment? 2020-06-16T17:10:32.467Z
Pragmatism and Completeness 2020-06-12T16:34:57.691Z
The Mechanistic and Normative Structure of Agency 2020-05-18T16:03:35.485Z
What is the subjective experience of free will for agents? 2020-04-02T15:53:38.992Z
Deconfusing Human Values Research Agenda v1 2020-03-23T16:25:27.785Z
Robustness to fundamental uncertainty in AGI alignment 2020-03-03T23:35:30.283Z
Big Yellow Tractor (Filk) 2020-02-18T18:43:09.133Z
Artificial Intelligence, Values and Alignment 2020-01-30T19:48:59.002Z
Towards deconfusing values 2020-01-29T19:28:08.200Z
Normalization of Deviance 2020-01-02T22:58:41.716Z
What spiritual experiences have you had? 2019-12-27T03:41:26.130Z
Values, Valence, and Alignment 2019-12-05T21:06:33.103Z
Doxa, Episteme, and Gnosis Revisited 2019-11-20T19:35:39.204Z
The new dot com bubble is here: it’s called online advertising 2019-11-18T22:05:27.813Z
Fluid Decision Making 2019-11-18T18:39:57.878Z
Internalizing Existentialism 2019-11-18T18:37:18.606Z
A Foundation for The Multipart Psyche 2019-11-18T18:33:20.925Z
In Defense of Kegan 2019-11-18T18:27:37.237Z
Why does the mind wander? 2019-10-18T21:34:26.074Z
What's your big idea? 2019-10-18T15:47:07.389Z
Reposting previously linked content on LW 2019-10-18T01:24:45.052Z
TAISU 2019 Field Report 2019-10-15T01:09:07.884Z
Minimization of prediction error as a foundation for human values in AI alignment 2019-10-09T18:23:41.632Z
Elimination of Bias in Introspection: Methodological Advances, Refinements, and Recommendations 2019-09-30T20:23:13.139Z
Connectome-specific harmonic waves and meditation 2019-09-30T18:08:45.403Z
Goodhart's Curse and Limitations on AI Alignment 2019-08-19T07:57:01.143Z
G Gordon Worley III's Shortform 2019-08-06T20:10:27.796Z
Scope Insensitivity Judo 2019-07-19T17:33:27.716Z

Comments

Comment by G Gordon Worley III (gworley) on What books are for: a response to "Why books don't work." · 2021-04-22T14:00:49.907Z · LW · GW

A point that's been made to me before about books and that supports your thesis that they serve to increase salience of ideas: books are worth reading because they cause you to live with an idea long enough to integrate it into your mind.

I've certainly noticed this facet of books for myself. There's plenty of books I read that I could now hardly recall any specific arguments from and that I could summarize their "take-aways" in a few paragraphs, but that nonetheless I got a lot out of reading because for the weeks or months when I was working through it the book forced my attention continually back on to a particular topic, which then also carried to thinking about that topic at other times.

I carry this same sort of idea with me into my choice of where to place my attention beyond books with a question like: how will placing my attention on this bit of media affect what I pay attention to, and is it pointing my attention towards something I think is worth thinking about?

Comment by G Gordon Worley III (gworley) on G Gordon Worley III's Shortform · 2021-04-22T04:01:00.496Z · LW · GW

Sometimes people at work say to me "wow, you write so clearly; how do you do it?" and I think "given the nonsense I'm normally trying to explain on LW, it's hardly a surprise I've developed the skill well enough that when it's something as 'simple' as explaining how to respond to a page or planning a technical project that I can write clearly; you should come see what it looks like when I'm struggling at the edge of what I understand!".

Comment by G Gordon Worley III (gworley) on How can we increase the frequency of rare insights? · 2021-04-21T23:17:54.559Z · LW · GW

Two ideas to share here.

Meditation is used to gain insight into the nature of reality; this is essentially what meditation is being used for in Buddhist practices. The mechanism by which this works isn't very clear, but it seems to be something like "gather a bunch of evidence about the world and wait for it to mound up so high that it collapses whatever categories you had before so you find new ones".

Related, I think meditation generalizes as a form of neural annealing, which is basically applying the annealing analogy to a process that can happen in the brain where the brain can reconfigure itself into a better state by cycling through a high energy state before returning to a lower energy state (or something like that). Lots of details not worked out here, but the link is the most up to date thing I have on this to share.

Comment by G Gordon Worley III (gworley) on You Can Now Embed Flashcard Quizzes in Your LessWrong posts! · 2021-04-19T22:48:36.656Z · LW · GW

These are all quite reasonable, and I'm pretty open to the idea that I'm mistaken and anchoring too much on the fact that I didn't find flash cards for spaced repetition useful, which might ultimately bias how heavily I weight things or assess the likelihood that flashcards would be helpful.

Comment by G Gordon Worley III (gworley) on G Gordon Worley III's Shortform · 2021-04-19T16:26:27.442Z · LW · GW

Maybe spreading cryptocurrency is secretly the best thing we can do short term to increase AI safety because it increases the cost of purchasing compute needed to build AI. Possibly offset, though, by the incentives to produce better processors for cryptocurrency mining that are also useful for building better AI.

Comment by G Gordon Worley III (gworley) on You Can Now Embed Flashcard Quizzes in Your LessWrong posts! · 2021-04-19T16:23:02.803Z · LW · GW

Registering my continued dislike for dedicating effort on this feature (I voiced similar complaints when this was originally floated). My reasons:

  • retaining knowledge via spaced repetition is a poor fit for most of the posts on LW
  • the general advice is that using cards written by others is much less useful than cards you write yourself
  • nudges LW towards encouraging posts that are more amenable to space repetition, which I expect will marginally decrease other kinds of content
    • implicit in this is an argument that LW would be marginally worse if it spent more time on teaching things that you can learn by remembering them and less time on other things, like working towards reducing confusion about things we are currently confused about
Comment by G Gordon Worley III (gworley) on Could degoogling be a practice run for something more important? · 2021-04-17T14:50:40.975Z · LW · GW

I think there's something valuable that comes with intentionally denying oneself something that is normally integral to one's life.

I could make a case here for the psychological (and ~spiritual) benefits of the kind of general resilience this helps train, but on a very practical level sometimes, for example, the internet goes out, and it's useful to know what to do if you've lost access to Google Mail, Docs, Maps, or whatever else you rely on. Doing that in a controlled setting is likely to be better because you can plan for it so it doesn't negatively impact your life beyond the scope of the practice, and it allows you to get hands on experience with the situation so that would be hard to get otherwise.

I think of this as the same reason people in jobs that require high reliability do things like war games and practice drills to make sure they know what to do in high stakes situations by practicing first when stakes are low so they can make mistakes and learn without negative impacts.

Comment by G Gordon Worley III (gworley) on Are there opportunities for small investors unavailable to big ones? · 2021-04-16T00:54:39.861Z · LW · GW

Feels a lot to me like a similar argument to why there is a complex ecosystem of investment sources for startups that target different stages of company life cycle: angel investors are "small time" and can afford to take high variance bets on fewer companies they know well while large VC firms have lots of money and need to find enough and big enough places to put it all.

Comment by G Gordon Worley III (gworley) on The secret of Wikipedia's success · 2021-04-16T00:25:37.301Z · LW · GW

I'm unconvinced of the model you offer here.

First, I'm not really buying that Wikipedia is unreliable, since you, I, and other people treat it as highly reliable. Yes, at any point in time any individual page might be in error or misleading, but those tend to get corrected such that I trust the average Wikipedia entry to give me a reasonable impression of the slice of reality it considers.

But even supposing you're right and people do think Wikipedia is unreliable and that's why it's secretly reliable, I don't see how that causes it to become reliable. To compare, let's take somewhere else on the internet that isn't try to do what newspapers are but are a mess of fights over facts—social media. No one thinks social media is reliable, yet people argue facts there.

Okay, but you might say they don't also have to generate a shared resource. But the only thing keeping Wikipedia from being defaced constantly is an army of bots that automatically revert edits that do exactly that plus some very dedicated editors who do the same thing for the defacing and bad edits the bots don't catch. People actually are trying to write all kinds of biased things on Wikipedia, it's just that other people are fighting hard to not let those things stay.

I suspect something different is going on, which you are pointing at but not quite getting to. I suspect the effects you're seeing are because Wikipedia is not a system of record, since it depends on other sources to justify everything it contains. No new information is permitted on Wikipedia per its editing guidelines; everything requires a citation (though sometimes things remain uncited for a long time but aren't removed). It's more like an aggregated view on facts that live elsewhere. This is one things newspapers, for example, do: they allow creating new records of facts that can be cited by other sources. Wikipedia doesn't do that, and because it doesn't do that the stakes are different. No one can disagree with what's on Wikipedia, only cite additional sources (or call sources into question) or argue that the presentation of the information is bad or confusing or misleading. That's very differently than being able to directly argue over the facts.

Comment by G Gordon Worley III (gworley) on A System For Evolving Increasingly General Artificial Intelligence From Current Technologies · 2021-04-14T14:48:53.688Z · LW · GW

In the face of existential risks from AI, whether or not the builder of a dangerous AI is more "prosocial" by some standard of prosociality doesn't really matter: the point of existential risk is that the good guys can also lose. Under such a calculus, there's not benefit to trying to beat someone else to building the same, since beating them just destroys the world faster and cuts off time that might have been used to do something safer.

Further, races are self-fulfilling prophecies: if we don't think there is a race then there won't be one. So all around we are better off avoiding things that advance capabilities research, especially that rapidly advice it in directions that are likely to cause amplification in directions not clearly aligned with human flourishing.

Comment by G Gordon Worley III (gworley) on Daniel Kokotajlo's Shortform · 2021-04-14T14:15:05.195Z · LW · GW

I think it's probably even simpler than that: feedback loops are the minimum viable agent, i.e. a thermostat is the simplest kind of agent possible. Stuff like knowledge and planning are elaborations on the simple theme of the negative feedback circuit.

Comment by G Gordon Worley III (gworley) on Specializing in Problems We Don't Understand · 2021-04-13T16:55:01.530Z · LW · GW

Also makes me think of TRIZ. I don't really understand how to use it that well or even know if it produces useful results, but I know it's popular within the Russosphere (or at least more popular there than anywhere else).

Comment by G Gordon Worley III (gworley) on A New Center? [Politics] [Wishful Thinking] · 2021-04-12T19:00:01.210Z · LW · GW

Political polarization in the USA has been increasing for decades, and has become quite severe. This may have a variety of causes, but it seems highly probable that the internet has played a large role, by facilitating the toxoplasma of rage to an unprecedented degree.

Contra the idea that the internet is to blame, polarization seems historically to be the "natural" state in both the USA and elsewhere. To get less of it you need specific mechanism that have a moderating effect.

For a long time in the US this was a combination of progressive Republicans (Whigs and abolitionists) and regressive Democrats (Dixiecrats) that caused neither major party to be able to form especially polarized policy positions. Once the Civil Rights Act and Roe v. Wade drove Dixiecrats out of the Democratic party and progressives out of the Republican party, respectively, the parties became able to align more on policy.

So extending this observation, rather than a new center, maybe what we need to get less polarization is something to hold the parties together along some line that's orthogonal to policy preferences such that both parties must tolerate a wide range of opinions. I'm not sure how to do that, as the above situation was created by the Civil War and Reconstruction that made variously the Republican and Democrat parties unacceptable to certain voters (like former slaveholders and abolitionists) and it was only after a hundred years that identification with or against the "Party of Lincoln" melted away enough to allow a shift.

Maybe your new center idea could cause this, but I'm not reading in it a strong enough coordination mechanism to overcome the nature tendency for parties to align in opposite directions.

Comment by G Gordon Worley III (gworley) on People Will Listen · 2021-04-12T05:42:13.499Z · LW · GW

My advice is to accept that 'haters are gonna hate' and just take the hit. Make your arguments as clear and your advice as easy to follow as possible. But understand that no matter what you do, if you tell people to buy bitcoin at $230, the top comment might be critical. Some people will listen and benefit.

I've just been thinking about this with respect to two posts I recently authored. First I wrote "Forcing yourself to keep your identity small is self-harm" and this got a bunch of negative response (e.g. currently a score of 17 with 24 votes; my guess based on watching things come in is that it's close to 50% downvotes). In response I wrote "Forcing Yourself is Self Harm, or Don't Goodhart Yourself" and so far it's doing "better" by some measures (score of 25 right now, but with only 11 votes, all positive as best I can tell).

The thing is both posts say exactly the same thing other than that the first post is vary concretely about a particular case while the latter is a general article that covers the original article as a special case. I basically wrote the second version by taking the original text and modifying it to be explicitly generalized rather than just about one case.

Now if I ask myself which one I think is better, I actually think it's the first one even though the latter is better received in terms of karma. The second one lacks teeth and I think it's too easy to read it and not really get what it's saying in a concrete way. The reader might nod along saying "ah, yes, sage advice, I will follow it" and then promptly fail to integrate it into even a single place where it matters, whereas the former is very in your face about a single place where it matters and confronts the reader to consider that they may have been screwing up at doing a thing they value.

I like this kind of stuff that confronts readers because, although it may draw greater controversy, it also seems more likely to land for the person who will benefit from reading it, and managing criticism/downvotes only matters insofar as I draw too much negative attention and negatively impact the visibility of the post to people who would have benefitted from having seen it in a world where it was less criticized and less downvoted.

Of course in this isolated case of two articles there are confounding factors. For example, maybe people "came around" on my arguments by the time the second post came out since they saw the first one, or maybe more people just ignored the second post since it looked so much like the first one. But I've noticed this sort of trend over and over in my own writing and the writing of others: saying something direct that challenges the reader will draw the ire of readers who dislike having been challenged on something they hold dear, and saying the same thing in a less direct way that avoids triggering their displeasure also is actually worse because it less well lands for anyone and the people who were going to criticise it now don't but without that meaning anything.

Comment by G Gordon Worley III (gworley) on "Taking your environment as object" vs "Being subject to your environment" · 2021-04-12T05:24:17.896Z · LW · GW

This post points at a core of why I like to talk about the subject-object relationship with respect to developmental psychology: the shifting of things from one side of the lens of intentionality to the other seems to be the key driver of development.

Comment by G Gordon Worley III (gworley) on "Taking your environment as object" vs "Being subject to your environment" · 2021-04-12T05:20:37.200Z · LW · GW

There's some complexity here because English offers two words here, "subject" and "object" that can be used somewhat interchangeably in some situations but in most situations we have some notion that "subject" is on the left/upstream side of the causal arrow and "object" is on the right/downstream side. However Ben's reuse of "subject" by shifting it from actor ("subject to") to the acted upon ("as subject") seems mostly poetic and a reasonable alternative to talking about object.

Of course, because English is noun-focused, it's rather nice to have two different nouns for these concepts rather than having to point to them by using two different verb phrases as Ben does here.

I have my own mild preferences around using standard phrasing to trigger in people associations with that common body of work built around those standards, but regardless I don't think anything in the post is actually at odds with standard phrasing, just different and, to my ear, equally clear, even if I have no intention of ever copying it.

Comment by G Gordon Worley III (gworley) on Forcing Yourself is Self Harm, or Don't Goodhart Yourself · 2021-04-10T19:28:53.239Z · LW · GW

Take your illustrative story.  I'd say the problem here is not that the person is trying to focus on the narrow area of increasing productivity.  It's that they picked a bad metric and a bad way of continual measuring themselves against the metric.  The story just kind of glosses over what I would say is the most important part!

I'd say that 65%-75% of the problem this person has is that they apparently didn't seriously think about this stuff before hand and pre-commit to a good strategy for measurement.

The person who looks and says "I only wrote 100 words last hour?!??!" kind of reminds me of the investor checking their stock prices every day.

For this person three months or six months or a year might be a better time frame for checking how they're doing.  Regardless, the main point I want to make is that how well this person would be able to improve themselves in this area while maintaining their well being is largely dependent upon making good decisions on this very important question.

This is one of the weird issues with what I see as the problem I'm trying to illustrate with the story and the limitations of telling a single story about it.

What you say is true, but it's a reduction of the problem to be less bad by applying weaker optimization pressure rather than an actual elimination of the problem. Weak Goodharting is still Goodharting and it will still, eventually, subtly screw you up.

This post is also advice, and so aimed mostly at folks less like you and more like the kind of person who doesn't realize they're actively making their life worse rather than better by trying too hard.

Comment by G Gordon Worley III (gworley) on A System For Evolving Increasingly General Artificial Intelligence From Current Technologies · 2021-04-09T18:37:35.691Z · LW · GW

Have you thought much about the safety/alignment aspects of this approach. This seems very susceptible to Goodharting.

Comment by G Gordon Worley III (gworley) on Testing The Natural Abstraction Hypothesis: Project Intro · 2021-04-09T00:16:58.324Z · LW · GW

Nice! From my perspective this would be pretty exciting because, if natural abstractions exist, it solves at least some of the inference problem I view at the root of solving alignment, i.e. how do you know that the AI really understands you/humans and isn't misunderstanding you/humans in some way that looks like it does understand from the outside but it doesn't. Although I phrased this in terms of reified experiences (noemata/qualia as a generalization of axia), abstractions are essentially the same thing in more familiar language, so I'm quite excited for the possibility that we can prove that we may be able to say something about the noemata/qualia/axia of minds other than our own beyond simply taking for granted that other minds share some commonality with ours (which works well for thinking about other humans up to a point, but quickly runs up against problems of assuming too much even before you start thinking about beings other than humans).

Comment by G Gordon Worley III (gworley) on Curious Inquiry and Rigorous Training · 2021-04-08T23:39:43.651Z · LW · GW

This resonates with something I've been thinking about lately. Despite getting high grades, graduating highschool with an IB diploma, and going most of the way through a PhD, I was actually kind of bad at school in several ways, and one of those was that I was trying to actually learn the stuff I studied. Like a fool, you might say, I failed to realize school was a system to be gamed and tried to actually learn everything I was asked to learn for real. This was exhausting, and I dropped out of the PhD because of burnout over this as much as any other reason, having finally crossed the threshold where my intelligence couldn't beat the system without gaming it.

I find learning outside school quite different. Mostly memorizing things doesn't matter and curiosity and ability to do things matter way more. Remembering stuff helps you be fast, but natural spaced repetition of stuff you actually use often works well enough. It's a lot more fun and I'm better at it.

Comment by G Gordon Worley III (gworley) on G Gordon Worley III's Shortform · 2021-04-08T20:39:13.088Z · LW · GW

More surprised than perhaps I should be that people take up tags right away after creating them. I created the IFS tag just a few days ago after noticing it didn't exist but wanted to link it and I added the first ~5 posts that came up if I searched for "internal family systems". It now has quite a few more posts tagged with it that I didn't add. Super cool to see the system working in real time!

Comment by G Gordon Worley III (gworley) on Solving the whole AGI control problem, version 0.0001 · 2021-04-08T16:03:50.157Z · LW · GW

Regarding conservatism, there seems to be an open question of just how robust Goodhart effects are in that we all agree Goodhart is a problem but it's not clear how much of a problem it is and when. We have opinions ranging from mine, which is basically that Goodharting happens the moment you try to apply even the weakest optimization pressure and this will be a problem (or at least a problem in expectation; you might get lucky) for any system you need to never deviate, to what I read to be Paul's position: it's not that bad and we can do a lot to correct systems before Goodharting would be disastrous.

Maybe part of the problem is we're mixing up math and engineering problems and not making clear distinctions, but anyway I bring this up in the context of conservatism because it seems relevant that we also need to figure out how conservative, if at all, we need to be about optimization pressure, let alone how we would do it. I've not seen anything like a formal argument that X amount of optimization pressure, measured in whatever way is convenient, and given conditions Y produce Z% chance of Goodharting. Then at least we wouldn't have to disagree over what feels safe or not.

Comment by G Gordon Worley III (gworley) on Forcing yourself to keep your identity small is self-harm · 2021-04-04T15:05:30.171Z · LW · GW

Author's note: I just now changed the title to try to better reflect what this is about. It originally read "intentionally keeping...", but I suspect that was garnering a lot of down votes from people who mistakenly think the point of my post is to say that keeping identity small is bad rather than that it's a particular way of doing that which is bad, so I've adjusted it to hopefully better communicate what the post is about.

Comment by G Gordon Worley III (gworley) on Forcing yourself to keep your identity small is self-harm · 2021-04-04T01:01:51.937Z · LW · GW

I made a deliberate choice here to be confrontational in a way that would most provoke the people who might most benefit from this advice, hence the way I ordered things. Note though that I'm quite careful never to say keeping your identity small is bad, only that directly or intentionally trying to do it, as opposed to letting it happen as a side effect or consequence, is harmful.

Comment by G Gordon Worley III (gworley) on Forcing yourself to keep your identity small is self-harm · 2021-04-03T19:11:46.806Z · LW · GW

Well, I don't know if I'd call it a fully general argument against taking any kind of advice. I'd say it's more like a fully general argument against directly trying to optimize for the thing being measured, i.e. an argument that you should avoid Goodhart's curse.

As to to the post not adding anything, I guess that's true in a certain sense, but I find many people, myself included, are kinda bad at taking general advice and successfully applying it in all specific situations, so it's often helpful to take something general and consider specific instances of it. You might say it's something like the general idea is the "math" and this post is the "engineering" of using the "math" to do some specific thing.

Some other quick examples that come to mind that might be worth exploring as posts to take this general idea and consider it in specific contexts given many people benefit from seeing the same thing worked out many ways:

  • don't harm yourself by trying to directly optimize for productivity
  • ... for popularity
  • ... for happiness
  • ... for success
  • ... for minimizing guilt
  • ... for health

And so on.

Comment by gworley on [deleted post] 2021-04-03T15:10:19.915Z

Agree. I didn't vote up or down the original post because I saw it and just bounced off it. Wasn't clear from the start what it was about or why I should read it, and was slightly put off figuring it out because the tone also put me off so lacking a clear reason to think it worth reading I just decided to spend my time otherwise.

Comment by G Gordon Worley III (gworley) on Affordances · 2021-04-03T02:45:15.517Z · LW · GW

A disposition seems like a concept which is supposed to apply to the object in itself, independent of subject.

I wouldn't say there's any existence of an object (as an object) independent of a subject; there's instead just stuff that's not differentiated from other stuff because something had to tell the difference, hence I don't see a real difference here, although the theory of dispositions is jumbled up with lots of philosophy that supposes some kind of essentialism, so it's reasonable that there might seem to be some difference from affordances under certain assumptions.

Comment by G Gordon Worley III (gworley) on Affordances · 2021-04-03T00:10:52.942Z · LW · GW

I'm pretty sure I've commented this before elsewhere on LW when a similar discussion came up, but there's a wealth of literature on this topic but under a different set of terms than "affordance". SEP feels the right general term is "disposition", and it generalizes not just to people but to all things.

Comment by G Gordon Worley III (gworley) on What Multipolar Failure Looks Like, and Robust Agent-Agnostic Processes (RAAPs) · 2021-04-02T17:37:09.804Z · LW · GW

Much of this, especially the story of the production web and especially especially the story of the factorial DAOs, reminds me a lot of PKD's autofac. I'm sure there are other fictional examples worth highlighting, but I point out autofac since it's the earliest instance of this idea I'm aware of (published 1955).

Comment by G Gordon Worley III (gworley) on What Do We Know About The Consciousness, Anyway? · 2021-04-02T15:40:27.736Z · LW · GW

Whether or not they are explained, the import thing I'm pointing at is that they are of the same type, hence whether or not you find an explanation satisfying, it is aiming to explain the same kind of thing.

Comment by G Gordon Worley III (gworley) on What Do We Know About The Consciousness, Anyway? · 2021-04-01T23:57:51.299Z · LW · GW

I think this is basically what you're saying, though: you're talking about qualities of experience/sensations, but to speak of qualities is to categorize things rather than just experience them, which means some part of experience is being made into a thing.

Comment by G Gordon Worley III (gworley) on What Do We Know About The Consciousness, Anyway? · 2021-04-01T20:08:46.257Z · LW · GW

I guess I don't see how it doesn't, since it's a reification of experience, which is what qualia are.

Comment by G Gordon Worley III (gworley) on What Do We Know About The Consciousness, Anyway? · 2021-04-01T17:29:26.819Z · LW · GW

I think we can have it both ways on qualia. Compare these two posts of mine:

Comment by G Gordon Worley III (gworley) on What Do We Know About The Consciousness, Anyway? · 2021-04-01T17:26:32.432Z · LW · GW

Since I left a bit of a nit early, did want to say I largely agree with your direction here.

Comment by G Gordon Worley III (gworley) on What Do We Know About The Consciousness, Anyway? · 2021-04-01T00:49:20.873Z · LW · GW

A chimp can think how to get a banana when it sees the banana, or is hungry and remembers where bananas are, or some such. Whereas I can lie there and ponder something as remote as Mars colonization, like all day long! Maybe that’s because, when a chimpanzee’s quite advanced mind sees the banana, it tries to predict the world where it gets the banana and uses all it’s world-modeling power to come up with a coherent prediction of it. But it’s stable only for as long as there’s some stimulus related to banana, once there’s no such stimulus the loop becomes unstable - the upper layers of the model trying to predict the world where the chimp gets the banana, but the lower levels report back that, well, no banana around, so after a while it dies out.

What's the evidence for this claim? As best I can tell we don't have any strong reason to believe chimps don't sit around daydreaming about bananas when they are not otherwise occupied.

Comment by G Gordon Worley III (gworley) on TAI? · 2021-03-30T17:44:51.473Z · LW · GW

Meta-comment on transformative AI.

I'm not sure this terminology has reached fixation yet and is still provisional. For example, I haven't seen it bubble up to replace talk in AI safety writing that would otherwise talk about superintelligence or strong optimization pressure. It seems mostly geared towards folks talking about policy and explaining why policy is needed, so caveat that it's a bit of jargon (like lots of things we say around here) that may have specific meaning that make it hard to answer this question because TAI is going to naturally be geared towards the stuff that's not here or almost here in order to get folks to take action rather than on the "boring" stuff we already live with and can see that it's not immediately transforming everything on the order of hours/days.

Comment by G Gordon Worley III (gworley) on G Gordon Worley III's Shortform · 2021-03-29T19:02:43.249Z · LW · GW

Psychological Development and Age

One of the annoying things about developmental psychology is disentangling age-related from development-related effects.

For example, as people age they tend to get more settled or to more have their lives sorted out. I'm pointing at the thing where kids and teenages and adults in their 20s tend to have a lot of uncertainty about what they are going to do with their lives and that slowly decreases over time.

A simple explanation is that it's age related, or maybe more properly experience related. As a person lives more years, tries more things, and better learns the limits of what they can achieve they become more settled into the corner of the world in which they can successfully operate. We should expect people with less years of experience to be more confused about themselves and older people to be less confused because they had more years to figure things out. This need not invoke developmental psychology, just simple accumulation of evidence.

But counter examples of resigned young people and curious and adventurous old people abound. But perhaps we can explain these simply in terms of contexts that cause that, and if a person were put in a different context they might behave differently and in a more age anticipated way.

Where I think developmental psychology helps is in understanding subtler differences in how people behave and why. For example, is a person content with their life because they've tried a bunch of things and given up stretching too far because they know it'll just be hard to do new things and they'll probably fail so it's not worth the effort, or are they content with their life because they trust themselves. It can be kinda hard to tell from the outside and you might not even be able to tell with an intervention, say by making something easier for them so they can do some new thing that they previously could not do easily and seeing how they respond.

This is one of the challenges I think we face in talking about developmental psychology. It seems a useful model for explaining many aspects of how minds progress over time, but it's hard to figure out what's conflated with things that would happen in a world with no developmental psychology beyond simple learning.

(Don't worry; I haven't suddenly found developmental psychology less useful, just musing a bit on an issue that comes up often of the form "this developmental model sounds fine as far as it goes, but what about this simpler explanation".)

Comment by G Gordon Worley III (gworley) on Bureaucracy is a world of magic · 2021-03-29T17:18:38.699Z · LW · GW

Here's some examples from my own life of these sorts of rituals to make things official:

  • I sent a document to my 401k administrator. They rejected it because I had signed it by inserting an image of my signature. Solution: export the PDF to JPEG and then create a PDF from the JPEG so my signature was "baked in" to the document the way it would be if scanned. They accepted it.
  • Had to get my not-court-ordered alimony agreement notarized so it could be used in some official function. Literally all this did was certify that I signed it at a particular place and time. Apparently that was all it took.
  • Had to send a few letters certified mail regarding a security deposit dispute. Why? The law says if I send them that way they count as an "objection", and if I don't they don't.

Also, one I remember from when I was a kid but that I never had to do:

  • My parents, in order to work for the county school board (they were both teachers), had to sign "loyalty oath" stating they were not communists or atheists. They weren't, but obviously this was a fool-proof mechanism for ensuring only red-blooded Americans were teaching the next generation.
Comment by G Gordon Worley III (gworley) on Coherence arguments imply a force for goal-directed behavior · 2021-03-27T23:32:58.345Z · LW · GW

I think here it makes sense to talk about internal parts, separate from behavior, and real. And similarly in the single agent case: there are physical mechanisms producing the behavior, which can have different characteristics, and which in particular can be ‘in conflict’—in a way that motivates change—or not. I think it is also worth observing that humans find their preferences ‘in conflict’ and try to resolve them, which is suggests that they at least are better understood in terms of both behavior and underlying preferences that are separate from it. 

I think this is worth highlighting as something we too often ignore to our peril. Paying attention to internal parts is sometimes "annoying" in the sense that we can build much easier to reason about models by ignoring mechanisms and simply treating things, like AIs, as black boxes (or as made up of a small number of black boxes) with some behavior we can observe from the outside. But doing so will result in us consistently being surprised in ways we needn't have been.

For example, you treat two AIs as if they are EU maximizers and you model the utility function they are maximizing. But they actually behave in different ways in some situation even though the modeled utility function predicted the same behavior. And I don't think this is just a failure to make a good enough model of the utility function, I think it's fundamental, the way Goodheart is, that when we model something we necessarily are measuring it and so not getting the real thing, and will necessarily risk surprise when the model and the real thing do different things.

So the less detailed our models, and the more they ignore internals, the more we put ourselves at risk. Anyway, kind of a tangent from the post, but I feel like I constantly see simple models being used to try to explain AI that push out important internals in the name of simpler models that create real risks of confusing ourselves in attempts to build aligned AI.

Comment by G Gordon Worley III (gworley) on How You Can Gain Self Control Without "Self-Control" · 2021-03-27T22:42:47.447Z · LW · GW

Agreed. I found the whole tone of the article kind of patronizing, like I was too dumb to understand what I was reading without a bunch of gimmicks thrown in to help me understand it. Reminded me a bit too much of primary readers.

Comment by G Gordon Worley III (gworley) on Curiosity needs courage · 2021-03-27T00:14:06.578Z · LW · GW

I think courage is one of those things that is often simultaneously under and over appreciated. Your post is evidence of underappreciating courage since you felt the need to call it out. I know I also failed to realize the courage I was using and not thinking of it that way, and once I did it became something I could intentionally cultivate.

On the other hand, some people think too much of courage, thinking it's enough to be brave even if you fail (if anything is not the Rationalist Way, it's not caring that you didn't win). Courage is not very useful on its own; only if it's used appropriately to some useful end.

Comment by G Gordon Worley III (gworley) on Where did the idea of x-risk come from? · 2021-03-26T20:47:30.156Z · LW · GW

Agreed. I clicked on this thinking it was a question post and I thought "oh, I was hanging around back then, I could maybe say something that would be helpful" but instead it was teasing a video and not actually offering content of the type described.

I probably wouldn't be opposed if this were a link post with a title that made it clear that it was linking a video, but this feels spammy to me as is.

Comment by G Gordon Worley III (gworley) on Raemon's Shortform · 2021-03-26T20:44:03.089Z · LW · GW

Something about this feels like what I used to do but don't do now, and I realized what it is.

If they're stuck I don't see it as their problem, I see it as my problem that I can't find a way to take my thing and make it sensible to them within their system, or at least find an entry point, since all systems are brittle and you just have to find the right thread to pull if you want to untangle it so they can move towards seeing things in ways beyond what their current worldview permits.

But maybe my response looks the same if I can't figure it out and/or don't feel like putting in the energy to do that, which is some version of "hey, looks like we just disagree in some fundamental way here I'm not interested in trying to resolve, sorry", which I regret is kinda rude still and wish I could find a way to be less rude about.

Comment by G Gordon Worley III (gworley) on The best frequently don't rise to the top · 2021-03-26T01:49:08.000Z · LW · GW

I think this gets at the heart of the confusion I see in this post, specifically confusion around the work "best" is doing, although highlighting "should" works just as well. I'd argue there's no fact of the matter about who should get more views, there's just the question of who actually gets more views and whether view count correlates with personal judgments of quality. That a person doesn't like what others do is not surprising to me.

On a more concrete note, never underestimate how much many people like liking things that other people like. Popular stuff is popular because it's popular past some threshold where positive feedback kicks in.

Comment by G Gordon Worley III (gworley) on Raemon's Shortform · 2021-03-26T01:36:07.549Z · LW · GW

The original formulation definitely mixes in a bunch of stuff along with it, the systems as object thing is meant to be characteric, but it's not all of the expected stuff. Most people don't push the hard version that taking systems as object is not just characteric but causally important (I say this even though I do push this version of the theory).

It is actually kinda rude to psychologize other people, especially if you miss the mark, and especially especially if you hit the mark and they don't like it, so it's probably best to just keep your assessment of their Kegan level to yourself unless it's explicitly relevant since bringing it up will probably work against you even if in a high-trust environment it wouldn't (and you are unlikely to be in a high-trust enough environment for it to work even if you think you are).

As for asking people if they have the skill, I don't expect that to work since it's easy to delude yourself that you do because you can imagine doing it or can do it in an intellectual way, which is better than not being able to do it at all but is also not the real deal and will fall apart the moment anything overloads global memory or otherwise overtaxes the brain.

Comment by G Gordon Worley III (gworley) on Open, Free, Safe: Choose Two · 2021-03-20T19:56:20.655Z · LW · GW

I can't but help think about how this applies not just online communities but countries.

Some countries are open and free but not safe. I'm not sure what the best example of this is today. Mexico feels pretty open and free but not particularly safe.

Some countries free and safe but not open. Japan comes to mind: pretty safe, people are pretty free, but it's basically impossible to immigrate (at best you can live and work there long-term as an outsider).

Some countries are open and safe but not free. Places like Singapore and UAE let lots of people come from all over the world but you got to live by the comparatively restrictive local rules or you're out.

The whole thing is complicated though because industrialization seems to offer a Pareto improvement to all three such that industrialized nations are more open, free, and safe than non-industrialized and developing countries, although this isn't always true since industrialization also precipitated some totalitarian states that were less open, free, and safe than what came before. Maybe industrialization is just a force multiplier here but doesn't change the underlying tradeoffs being made?

Comment by G Gordon Worley III (gworley) on romeostevensit's Shortform · 2021-03-20T13:42:05.383Z · LW · GW

But maybe that's just the same thing? Like I don't know if there's a meaningful difference between descriptive disguised as prescriptive and prescriptive disguised as descriptive and instead it might make more sense to just talk about confusing descriptive and prescriptive.

Comment by G Gordon Worley III (gworley) on Ads are everywhere, and it's not okay · 2021-03-19T15:44:46.842Z · LW · GW

Agreed, this seems a critical flaw in the post's argument.

Ads are not always zero-sum or negative-sum games.

Intuitive example: I see an ad for a new shoe I was unaware of. It sounds interesting to me and I buy it. I like it and my life is better for having this better shoe. As a result, the world is better off: I got a better shoe that I value more than the old shoe I was wearing, holding constant that I was going to pay money for shoes and am choosing between similarly priced options.

This even holds for things I already knew about but didn't reconsider.

Intuitive example: For historical reasons I believe car X to be poor quality. I see an ad extolling the recent changes in car X to be higher quality. I do some further research and learn car X is in fact better than it used to be. So I learned something new thanks to the ad and can now make better decisions.

Ads are a way of paying to prioritize information.

Comment by G Gordon Worley III (gworley) on What's So Bad About Ad-Hoc Mathematical Definitions? · 2021-03-17T02:24:20.623Z · LW · GW

Isn't this just begging the question, though, by picking up an implicit type signature via the method by which probabilities are assigned? Like, if we lived in a different universe that followed different physics and had different math I'm not convinced it would all work out the same.

Comment by G Gordon Worley III (gworley) on Can an economy keep on growing? · 2021-03-16T13:42:46.922Z · LW · GW

Although I don't disagree with your conclusion that growth must eventually slow, I think the limits of growth are much further off. I have some scattered thoughts on this, so maybe we can figure out what's going on if I just write them out.

The way I think of value being generated in an economy is by doing an activity I'd describe as pumping against gradients. Like, why does something have value? Because people are willing to pay for it because it would be more expensive to get it themselves (expense here to be demarcated by whatever units are meaningful to a person).

Another way to put this is that it's trading an asset for generating less local entropy.

Now eventually we could imagine a universally connected economy where local entropy = global entropy and the only thing of value is producing less entropy to do something someone cares about than would otherwise have been required. Like, I'd pay $X entropy bucks to generate Y fewer bits of entropy. But at some point it seems like we'd reach some theoretical limit on how little entropy could be used to produce the thing I care about, and this would result in a bottoming out of growth.

Growth would still happen, mind you, just that now the derivative of growth is flat.

As the universe approaches heath death, though, presumably the derivative would turn negative as we use up the remaining free energy and approach contraction.

This seems probably a long ways off, but hopefully we get there.