Comment by gworley on Are there documentaries on rationality? · 2019-02-14T19:40:07.685Z · score: 2 (1 votes) · LW · GW

I believe there is some stuff about cryonics, although not all of it flattering, and there's certainly a lot said about AI, although not from a rationalist perspective. There's also a lot of interesting stuff on YouTube produced by people focused on sharing knowledge that might be lumped broadly under "education" but that's really giving short shrift to what they're doing. I don't have any handy links, though, but maybe that at least gets you looking in useful directions.

I don't think there's anything like this for broader rationality topics outside some podcasts, but maybe something like them could be made into video documentaries (which I assume is the format you're interested in) in the future.

Comment by gworley on Some Thoughts on Metaphilosophy · 2019-02-14T02:00:13.346Z · score: 2 (1 votes) · LW · GW
Yes, specific sciences study small slivers of what we experience, and philosophy ponders the big picture, helping to spawn another sliver to study. Still don't see how it provides answers, just helps crystallize questions.

It sounds like a disagreement on whether A contains B means B is an A or B is not an A. That is, whether or not that, say, physics, which is contained within the realm of study we call philosophy, although carefully cordoned off with certain assumptions from the rest of it, is still philosophy or whether philosophy is the stuff that isn't broken down into a smaller part, because to my way of thinking physics is largely philosophy of the material and so by example we have a case where philosophy provides answers.

Comment by gworley on Arguments for moral indefinability · 2019-02-12T21:13:21.358Z · score: 4 (4 votes) · LW · GW

I know I often sound like a broken record, but I'd say this just keeps coming back to the fundamental uncertainty we have about the relationship between reality as it is and as we know it and the impossibility of bringing those two into perfect, provable alignment. This is further complicated by the issue of whether or not the thing we're dealing with, moral facts, exist or, if they do exist, exist mind-independently, and question to which it so far seems we are unlikely to find a solution for unless we can find a synthesis over our existing notions of morality such that we are able to becomes deconfused about what we were previously trying to point at with the handle "moral".

Comment by gworley on Some Thoughts on Metaphilosophy · 2019-02-12T21:01:56.771Z · score: 5 (2 votes) · LW · GW

I object rather strongly to this categorization. This feels strongly to me like a misunderstanding borne of having only encountered analytic philosophy in rather limited circumstances and having assumed the notion of the "separate magisterium" that the analytic tradition developed as it broke from the rest of Western philosophy.

Many people doing philosophy, myself included, think of it more as the "mother" discipline from which we might specialize into other disciplines once we have the ground well understood enough to cleave off a part of reality for a time being while we work with that small part so as to avoid constantly facing the complete, overwhelming complexity of facing all of reality at once. What is today philosophy is perhaps tomorrow a more narrow field of study, except it seems in those cases where we touch so closely upon fundamental uncertainty that we cannot hope to create a useful abstraction, like physics or chemistry, to let us manipulate some small part of the world accurately without worrying about the rest of it.

Comment by gworley on The Argument from Philosophical Difficulty · 2019-02-12T20:53:15.181Z · score: 2 (1 votes) · LW · GW

This seems fairly unlikely to me except insofar as AI acts as a filter that forces us to refine our understanding. The examples you provide arguably didn't make anything easier, just made what was already there more apparent to more people. This won't help resolve the fundamental issues, though, although it may at least make more people aware of them (something, I'll add, I hope to make more progress on at least within the community of folks already doing this work, let alone within a wider audience, because I continue to see, especially as goes epistemology, dangerous misunderstandings or ignorances of key ideas that pose a threat to successfully achieving AI alignment).

Comment by gworley on The Argument from Philosophical Difficulty · 2019-02-12T20:48:23.689Z · score: 2 (1 votes) · LW · GW

Unfortunately many philosophical problems may not have solutions of a form that allow us to construct something that definitely is what we want, but rather only permits us to say something is probably not what we want due to the fundamental ungroundability of our beliefs. My suspicion is that you are right, the problem is even harder than anyone currently realizes, and the best we can hope for is to winnow away as much stuff that obviously doesn't work while still leaving us with lots of uncertainty about whether or not we can succeed at our safety objectives.

Comment by gworley on What are some of bizarre theories based on anthropic reasoning? · 2019-02-04T19:13:13.877Z · score: 3 (2 votes) · LW · GW

My interpretation of anthropic arguments is that they are reasoning the same way as we do in the multi-world interpretation of quantum mechanics, so I think quantum immortality falls under what you're asking for.

Comment by gworley on The role of epistemic vs. aleatory uncertainty in quantifying AI-Xrisk · 2019-01-31T20:54:09.406Z · score: 2 (1 votes) · LW · GW

To make this model and little richer and share something of how I think of it, I tend to think of the risk of any particular powerful AI the way I think of risk in deploying software.

I work in site reliability/operations, and so we tend to deal with things we model as having aleatory uncertainty like holding constant a risk that any particular system will fail unexpected for some reason (hardware failure, cosmic rays, unexpected code execution path, etc.), but I also know that most of the risk comes right at the beginning when I first turn something on (turn on new hardware, deploy new code, etc.). A very simple model of this is something like where most of the risk of failure happens right at the start and beyond that there's little to no risk of failure, so running for months doesn't represent a 95% risk; almost all of the 5% risk is eaten up right at the start because the probability distribution function is shaped such that all the mass is under the curve at the beginning.

Comment by gworley on The role of epistemic vs. aleatory uncertainty in quantifying AI-Xrisk · 2019-01-31T20:43:32.069Z · score: 2 (1 votes) · LW · GW

I really appreciate you sharing a word for this distinction. I remember being in a discussion about the possibility of indefinite lifespans way back on the extropians mailing list, and this one person was making an argument about it being impossible due to accumulation of aleatory risk using life insurance actuarial models as a starting point. Their argument was fine as far as it went, but it created a lot of confusion when it seemed there was disagreement on just where the uncertainty lay, and I recall trying to disentangle that model confusion lead to a lot of hurt feelings. I think having some term like this to help separate the uncertainty about the model, the uncertainty due to random effects, and the uncertainty about the model implying certain level of uncertainty due to random effects would have helped tremendously.

Comment by gworley on Wireheading is in the eye of the beholder · 2019-01-31T20:30:55.154Z · score: 2 (1 votes) · LW · GW

A point you make that I think deserves more emphasis is the "eye of the beholder" part you use in the title.

Wireheading is something that exists because we have a particular meaning we assign to a reward. This is true whether we are the one observing the actions we might label wireheading or the one to whom it is happening (assuming we can observe our own wireheading).

For example, addicts are often not unaware that they are doing something, like shooting heroin, that will directly make them feel good at the expense of other things, and then they rationally choose to feel good because it's what they want. From the inside it doesn't feel like wireheading, it feels like getting what you want. It only looks like wireheading from the outside if we pass judgement on an agent's choice of values such that we deem the agent's values to be out of alignment with the objective, a la goodharting. In the case of the heroin addict, they are wireheading from an evolutionary perspective (both the actual evolutionary perspective and the reification of that perspective in people judging a person to be "wasting their life on drugs").

As I say in another comment here, this leads us to realize there is nothing so special about any particular value we might hold so long as we consider only the value. The value of values, then, must exist in their relation to put the world in a particular state, but even how much we value putting the world in particular states itself comes from values, and so we start to see the self-referential nature of it all that leads to a grounding problem for values. So put another way, wireheading only exists so long as you think you can terminate your values in something true.

Comment by gworley on Wireheading is in the eye of the beholder · 2019-01-31T20:18:09.757Z · score: 7 (3 votes) · LW · GW

Right. I think we can even go a step further and say there's nothing so special about why we might want to satisfy any particular value, whether it has the wirehead structure or not. That is, not only is wireheading in the eye of the beholder, but so is whether or not we are suffering from goodharting in general!

Comment by gworley on Freely Complying With the Ideal: A Theory of Happiness · 2019-01-31T19:20:17.714Z · score: 3 (2 votes) · LW · GW
Well, guess what you do have to remember all the time, as if you’d uncontrollably pressed “repeat” on the memory player? You.

I really like this way of phrasing it. We don't live in the past or the future, we live right here right now in this very moment experiencing reality as it is. And the self is created by a kind of noticing what's happening and reifying it into a thing by remembering what we experienced just moments ago. So the more we spend time focused on anything other than what's happening right here, right now, and the things that affect the conditions of the here and now, the more we're distracted and ignoring what has the most impact on our lives.

Of course that's easier said than done! But it's the essential wisdom about how to be awake to our lives and live them to their fullest.

Comment by gworley on Confessions of an Abstraction Hater · 2019-01-31T19:08:43.491Z · score: 2 (1 votes) · LW · GW

I don't know, this doesn't jive with my experience of abstractions.

Yes, structuring code with abstractions rather than just directly doing the thing you're trying to do makes the code more structurally complex and yes sometimes it is unnecessary and yes more structural complexity means it's harder to tell what any individual chunk of code does in isolation, but I think your example suggests you're engaging with abstractions very differently from I do.

When I write code and employ abstraction, it's usually not that I just think "oh, how could I make this more clever", it's that I think, "geez, I'm doing the same thing over and over again here, duplicating effort; I should abstract this away so I only have to say something about what's different rather than repeatedly doing what's the same". Some people might call this removing boilerplate code, and that's sort of what's going on, but I think of boilerplate as more a legacy of programming languages where for toolchain reasons (basically every language prior to so-called 4th gen languages) or design reasons (4th gen languages like Python that deliberately prevent you from doing certain things) you needed to write code that lacked certain kinds of abstractions (what we frequently call metaprogramming). Instead I think of this as the natural evolution of the maxim "Don't Repeat Yourself" (DRY) towards code that is more maintainable.

Because when I really think about why I code with abstractions, it's not to show off or be efficient with my lines of code or even to just make things pretty, it's to write code that I can maintain and work with later. Well designed abstractions provide clear boundaries and separation of concerns that make it easy to modify code to do new things as requirement change and refactor parts of the code. Combined with behavioral test driven development, I can write tests to the expected behavior of these concerns, and know I can trust the tests to let me change the code and still pass so long as the behavior doesn't change, and to let me know if I accidentally break the behavior I wanted in the code.

Yes, I often don't do it perfectly, but when it works it's beautiful. My experience is that mainly the people who dislike it are new grads who have spent all their time coding toys who haven't much had to deal with large, complex systems; everyone seems to understand that learning about system-specific abstractions is just naturally what we must do to be kind to ourselves and future programmers who will work with this code. To do otherwise is to do the future a disservice.

Comment by gworley on For what do we need Superintelligent AI? · 2019-01-25T18:47:16.914Z · score: 5 (4 votes) · LW · GW

Not really an answer, but there's also the point that with superintelligence humans don't have to do the things they otherwise could do because we built such a general tool that it eliminates the need for other tools. This is pretty appealing if, like me, you want to be free to do things you want to do even if you don't have to do them, rather than having to do things because you need to do them to satisfy some value.

Comment by gworley on Is Agent Simulates Predictor a "fair" problem? · 2019-01-25T18:43:44.268Z · score: 2 (1 votes) · LW · GW

I think this comment does a better job of explaining the notion of fairness you're trying to point at than other words here.

Comment by gworley on Is Agent Simulates Predictor a "fair" problem? · 2019-01-25T18:41:02.358Z · score: 2 (1 votes) · LW · GW

I don't think we disagree.

Comment by gworley on Is Agent Simulates Predictor a "fair" problem? · 2019-01-24T22:05:16.169Z · score: 2 (3 votes) · LW · GW

To my mind what seems unfair about some problems is that they propose predictors that, to the best of our knowledge, are physically impossible, like a Newcomb Omega that never makes a mistake, although these are only unfair in the sense that they depict scenarios we won't ever encounter (perfect predictors), not that they ask us something mathematically unfair.

Other more mundane types of unfairness, like where a predictor simply demands something so specific that no general algorithm could always find a way to satisfy it, seem more fair to me because they are the sorts of things we actually encounter in the real world. If you haven't encountered this sort of thing, just spend some time with a toddler, and you will be quickly disabused of the notion that there could not exist an agent which demands impossible things.

Comment by gworley on Why don't people use formal methods? · 2019-01-22T18:37:35.877Z · score: 6 (3 votes) · LW · GW

I think some people do, or at least try to, but my impression of the state of computer-assisted proofs and formal verification methods for programs is that they are still not very good because the problem is incredibly complex and we've basically only made it to the level of having FORTRAN-level tools. This is to say, we're a bit better off than we used to be doing formal verification with assembly-level tools where you had to specify absolutely everything in very low-level terms, but mostly in ways that just make it easier to do that low-level work rather than having many useful abstractions to help us perform formal verification without having to understand the details of (almost) everything all the time.

To continue the analogy, things will get a little more exciting as we get C and then C++ level tools, but I think things won't really explode and be appealing to many folks who don't desperately need to do formal verification until we get to something like the Python/Ruby-level in terms of tooling.

This does suggest something interesting, though: if someone thinks more widely using formal verification is important, especially in AI, then a straight-forward approach is to work on improving formal verification tools to a point that they can build up the abstractions that will help people work with them.

Comment by gworley on Announcement: AI alignment prize round 4 winners · 2019-01-21T20:33:40.371Z · score: 6 (5 votes) · LW · GW

I agree with this point. Looking at the things that have won over time it eventually got to feel like it wasn't worth bothering to submit anything because the winners were going to end up mostly being folks who would have done their work anyway and meet certain levels of prestige. In this way I do sort of feel like the prize failed because it was set up in a way that rewarded work that would have happened anyway and failed to motivate work that wouldn't have happened otherwise. Maybe it's only in my mind that the value of a prize like this is to increase work on the margin rather than recognize outstanding work that would have otherwise been done, but I feel like beyond the first round it's been a prize of the form "here's money for the best stuff on AI alignment in the last x months" rather than "here's money to make AI alignment research happen that would otherwise not have happened". That made me much less interested in it, to the point I put the prize out of my mind until I saw this post reminding me of it today.

Comment by gworley on A Framework for Internal Debugging · 2019-01-18T00:31:03.807Z · score: 3 (2 votes) · LW · GW

Nice. I'm surprised at the lack of comments and votes; maybe this just didn't engage for most people?

This is the sort of approach I don't personally like very much, i.e. laying out a whole lot of steps to take with instructions along the way. I tend to prefer more a "here's the one or two things to do that capture the essence of what you're after, then you fill in the details" approach for myself, but it seems like a lot of people like to work in a way like you lay out here. Having more of these kinds of tools seems useful, since some seem to resonate with some people more than others. Besides, I think no matter what our specific methods they all do the same thing anyway: if you've not see it already, you might like my writing on the general pattern of personal growth cycles where I talk about that. I see the general pattern expressed pretty clearly in what you lay out here with reasonable amounts of detail about how to carry out each part of the process.

Comment by gworley on What AI Safety Researchers Have Written About the Nature of Human Values · 2019-01-16T22:08:28.446Z · score: 2 (1 votes) · LW · GW

Looks like the images are not loading for me, even when I try to follow the links to them.

Comment by gworley on Buy shares in a megaproject · 2019-01-16T22:04:14.722Z · score: 7 (4 votes) · LW · GW

mr-hire already mentioned movies, but the other thing this reminds me of is the way people used to fund ocean voyages.

I don't recall all the specifics, but it's something like this. Sending a bunch of stuff across the ocean used to be risky, and in particular that the risk is high enough (if we take risk here to be the product of probability of failure times capital outlays) that generally a individual investor is not willing to take it on. The straightforward way to handle this was with partnerships, and it's the method that was in use for thousands of years for all sorts of endeavors with this risk profile from ocean voyages to trans-Saharan caravans, so much so that some religions actually codify rules about how to manage these partnerships.

Then we get the innovation of the joint-stock company and one of the first things it's used for is not creating long-lived corporations (although this follows after a few decades of experience with them) but creating time-limited corporations to finance ocean trade. It then becomes possible for a person to invest in projects, specifically ocean voyages, and also create a diversified portfolio of investing partially in many ocean voyages so that even if n% fail the 1-n% that succeed are enough to turn a profit.

Again, as mr-hire says, movies are something where a model something like this is still alive today. We have something like this alive for megaprojects in the form of municipal bonds sold to cover specific projects, but this is obviously not a complete solution. It's interesting to me to ask why we don't use time or project limited corporations that start with clear end states where everyone gets paid back at the end and the corporation wraps up more often; it's a thing we did in the past more often (relative to total corporations created) and do now in some domains or through alternative mechanisms, so I wonder why it seems to have dropped as so frequent a finance mechanism.

Comment by gworley on What are the components of intellectual honesty? · 2019-01-15T20:33:24.011Z · score: 7 (4 votes) · LW · GW

I tend to think when it comes to matters of honesty or otherwise talking about behavior that works towards some shared epistemic ends (i.e. pro-social behavior), the main issue is whether or not we see evidence of deception.

My reason for focusing on deception rather than, say, truth or facts is that I don't think we can reliably assess those things to a fine enough degree to not get stuck in a debate with infinite regress. But even if you disagree with my epistemological stance, I still think getting honesty separate from questions of facts helps because most of what I think we care about in terms of honesty is issues of how we relate to reality and the facts we think we know about it, rather than the facts themselves. That is, we want a notion of honesty that allows us to make honest mistakes, so to me the way to do that is by moving away from directly looking at epistemic actions and instead looking at actions that inform epistemic behavior.

Thus I tend to think of honesty as the opposite of deception. If in deception one is trying to confuse or mislead others as to what one believes the facts are, in honesty one is trying to deconfuse and show others plainly what one believes to be true. Honesty is, in this way, a kind of virtue we can cultivate to be straightforward and upright in our presentation of our beliefs, not hiding and distorting things to purposes other than seeing reality without hinderance.

To add more subtlety, I think there is also an active/passive component to honesty and deception. Sometimes people are aware and actively trying to deceive, like the villain in a plot, and other times they are unaware and passively performing deception without intent, like when people forget things that are uncomfortable for them or hidden beliefs that a person wouldn't necessarily endorse warp their perspective such that they can't see things as they are. This is not to make a moral distinction, although I suppose you could on this basis do that, but instead to point out that deception is often sneaky and even if a person is not actively being dishonest they may still not succeed at expressing honesty because of passive deception that performs through them.

Total, radical honesty, then, is just what happens when we stop even passively deceiving ourselves. Quite the virtue to strive for, but in the context of something like epistemic trust, it helps make sense of why some people are more deserving of trust than others, even if no one is actively trying to deceive.

Comment by gworley on The Tether Theory and the Concrete, Subtle and Causal tiers · 2019-01-15T20:08:54.854Z · score: 2 (1 votes) · LW · GW

That makes sense. People sometime talk in terms of "energies" or "moods" I think to express something similar but with a less precise metaphor.

Comment by gworley on The Tether Theory and the Concrete, Subtle and Causal tiers · 2019-01-15T00:38:45.072Z · score: 9 (2 votes) · LW · GW

Within Chan (Zen) Buddhism we sometimes talk about about the body-mind or the heart-body-mind as being one. Among the various things this concept helps explicate, one of them seems to be this point: there's a tight connection between what we think of as different levels/tiers/kinds of experience, and each part affects the whole and the other parts. Talking about this kind of connection as a tether, as well as the examples you give, seem helpful for better understanding the connection between the parts of ourselves we can (erroneously) think of as separate.

Comment by gworley on What are the open problems in Human Rationality? · 2019-01-14T21:13:58.875Z · score: 5 (4 votes) · LW · GW

To me the biggest open problem is how to make existing wisdom more palatable to people who are drawn to the rationalist community. What I have in mind as an expression of this problem is the tension between the post/metarationalists and the, I don't know, hard core of rationalists: I don't think the two are in conflict; the former are trying to bring in things from outside the traditional sources historically liked by rationalists; the latter see themselves as defending rationality from being polluted by antirationalist stuff; and both are trying to make rationality better (the former via adding; the latter via protecting and refining). The result is conflict even if I think the missions are not in conflict, though, so it seems an open problem is figuring out how to address that conflict.

Comment by gworley on What shape has mindspace? · 2019-01-12T23:03:41.957Z · score: 2 (1 votes) · LW · GW
If a mind is a topological space equipped with a subset, what sort of mind would the set being full imply?

That I'm not sure, as I haven't worked this out in much detail. I just sort of have a vague mathematical intuition that it might be the right sort of way to model it (n.b. I dropped out of a math phd after 6 years to do a startup, if that's some rough guide to how much to trust my mathematical intuition).

For what it's worth, here's some notes I found that I wrote about this a while ago. I make no promises that any of this makes any sense or that I would still agree with any of it, but it is what I wrote about it a while ago:

Mathematical Foundation of Phenomenology

So that phenomenological complexity classes are applicable to as many universes as possible, including our own, it has a rigorous mathematical foundation that makes as few assumptions as possible and easily translates into the standard language of phenomenology. That said, it is not a theory of everything, and so supposes that

  • the universe is made of stuff in configurations called states that are related to each other by causation,
  • mathematics can be applied to stuff, states, and causation,
  • and states can be partially ordered by causation.

Let process denote a set of states partially ordered by causation. Processes include simple physical processes modeled by atoms and quarks, stochastic processes like Brownian motion and waves, mechanistic processes like levers and clocks, phenomenological processes like cats and humans, social processes like organizations and friendships, and a universal process that includes all states and which we refer to as the universe, reality, or the world. Every process is a subset of the universe, including the empty process that contains no state.

We can then construct a topological space, called process space, on the subset processes of the universe where

  • every process, including the universal process and the empty process, is a member of the topology,
  • the union of processes is itself a process,
  • and the finite intersection of processes is itself a process.

Process space, being a topology, automatically gives us a lot of constructs to work with. Each process has an interior defined as the process containing itself and all the processes within it. We can further define filters in process space as the partial ordering of processes by subset, and then the closure of a process will be all the processes within the limit of the filters with the process’s interior as their base. These definitions are respectively analogous to subject, experience, and context in the standard terminology of phenomenology, but are strictly ontological and lack confounding teleological meanings. We’ll use these ontological terms in the remainder of the introduction for precision but switch the more common terminology in the dialectic for readability.

Processes may contain within them partial information about their filters in their partially order set of states. This is possible because partial ordering on filters

Phenomenology is the study of process filters, and the filters on a process are closed under finite intersection, which is to say that the commonalities of any two experiences is itself an experience, and upwards closed under subset, which means any part of an experience is itself an experience. Then we can understand the largest filter on a process’s interior, which is to say the filter that cannot be made any larger and still have the process’s interior as its base, as the principal ultrafilter on the process’s interior or simply the process’s ultrafilter. It trivially follows that a process’s ultrafilter contains every filter on the process as a subset, thus a process’s ultrafilter can be understood as the totality of a subject’s experience.

Thus far our theory is , and so when we talk about a process’s ultrafilter we are talking about all the experiences a subject might or will have, depending on the nature of causality. Yet each filter that is not itself the ultrafilter has supersets, which we might call “future” experiences. Since we are ourselves processes embedded in the topological space, at any “moment” of experience we have “future” experiences, and so when we talk about a process’s ultrafilter it’s useful to have a notion of the ultrafilter in time for universes like our own where the state of stuff is grouped in partially ordered subsets of the power set of state sets that admit a notion of “before” via subset, “after” via superset, and “now” by choosing an arbitrary set as reference point.

Comment by gworley on What shape has mindspace? · 2019-01-11T21:19:20.350Z · score: 2 (1 votes) · LW · GW

Sure, but that issue is irrelevant to the question you're asking. You can disagree with how I choose to convert "conscious" into a technical term from a folk term and even if you agree with my conversion perhaps disagree with whether or not something must be conscious in that sense to be aligned, but you asked about mindspace and those documents, although driving at other purposes, lay out some background that could be used to approach the question.

Comment by gworley on Megaproject management · 2019-01-11T21:16:56.220Z · score: 2 (1 votes) · LW · GW
I wonder about the suitability of this field as a target for EA careers. An unacceptably high percentage of that ~8% of GDP is wasted, and the picture gets worse when we entertain opportunity costs. Insofar as economic growth in general is good for alleviating suffering, the ability to prevent hundreds of millions of dollars in waste per project seems like a good deal.

I suspect there would be a high replacement effect, i.e. if we managed to spend less on these big projects, we'd probably just spend the excess on more big projects or be more ambitious on these projects. Many megaprojects are not obviously contributing to increasing welfare on the margin, although perhaps if megaprojects were cheaper we'd be more willing to invest ones that are more about increasing welfare than status (I suspect status is a major player in megaprojects since many of the examples that come to mind are unnecessarily ambitious when simpler, cheaper solutions would have worked but would have been less prestigious).

Comment by gworley on What shape has mindspace? · 2019-01-11T19:30:28.659Z · score: 1 (4 votes) · LW · GW

I did some work in this direction when I wrote about phenomenological complexity classes. I don't lay it out in much detail in that post, but I believe we can build on the work I do there to construct a topology of mindspace based on the assumption of a higher-order theory of consciousness and a formal model of the structure of consciousness grounded in intentionality by (and here's where I'm not sure what model will really work) possibly treating minds as sets within a topological space or as points on manifolds and then being able to say something about the minds we do and don't find in topological spaces with particular properties.

Alas this is all currently speculation and I haven't needed to go further than pointing in this general direction to do any of the work I care about, but it is at least one starting point towards work in this direction.

Comment by gworley on Book Recommendations: An Everyone Culture and Moral Mazes · 2019-01-11T19:21:33.597Z · score: 5 (2 votes) · LW · GW
But it’s in the interests of employers who want to use this sort of approach, to trick prospective (and current) employees not to act in their own economic best interests… which is, of course, precisely what we see.

So actually not. As I mentioned in another comment, adding people to your DDO who aren't on board with being in a DDO is a recipe for disaster: they will be unhappy at what a DDO asks of them, and the organization will be less DDO-like (to the point it may cease to really be a DDO). Maybe that's what's happened at Bridgewater; I don't know there so I can't say.

All businesses ask employees to make particular choices about a bundle of goods that they purchase with their labor. DDOs offer a currently uncommon bundle of goods that some people like a lot. Some people don't like it, so they work at other businesses that offer a different bundle of goods in exchange for labor. Yes, some of the bundle of goods can be negotiated on an individual basis, such that, for example, someone who isn't excited about working at a DDO but that the DDO really wants to hire might pay them extra to compensate them for doing labor they view as more costly, but the case is exactly the same when we consider an organization with a culture of Taylorism or something else: if it wants to hire someone who isn't excited about the culture, they will have to compensate them to offset the costs associated with the culture. As it turns out, though, mind space is big, not everyone wants the same thing, and so there are plenty of people to go around who want to work in these different types of cultures and are happy to do so for not much additional compensation on the margin.

Comment by gworley on Book Recommendations: An Everyone Culture and Moral Mazes · 2019-01-11T00:39:16.432Z · score: 5 (3 votes) · LW · GW
Having little experience of corporations myself, I can't say whether it's a realistic approach, but the whole thing struck me as a little too neat and tidy--if it were that easy, wouldn't everybody be doing it already?

So the reality of implementing something like a DDO is that the path to success is very narrow and there are lots of opportunities to mess up and create something worse than the default. For example, you might try to demand radical honesty, but if you punish people when they are honest and you don't like what they had to say, the culture will get toxic fast. Or maybe you go too far on being accepting and making people feel safe and then nothing ever gets done because no one wants to say "hey, if you keep not doing your job, we'll go out of business". And everyone has to be on board with being a DDO, so hiring (and firing) are an even more essential function than normal because you have to find people who not only can do the job, but are also capable of being a healthy part of the org. If you have just one person who's not on board, even if they are an individual contributor and not managing or leading anyone, this can sour the culture and start to turn people on each other if that person is able to have a strong enough effect on others.

For this reason I think it's also very hard, and maybe impossible, to do a conversion to a DDO unless you are willing to accept a timeline much longer than any normal business would. Building one from the ground up isn't easy, either, but it's definitely possible even if you haven't done it before so long as you keep iterating, experimenting, keeping to the goal, and gently correcting when you stray.

Comment by gworley on Book Recommendations: An Everyone Culture and Moral Mazes · 2019-01-11T00:29:41.291Z · score: 10 (3 votes) · LW · GW

I also liked "An Everyone Culture", but then I'm biased heavily in favor of Kegan, so no surprise there.

"An Everyone Culture" is Kegan and Lahey's most accessible book, by which I mean it asks you to agree with them the least about constructive developmental theory relative to their other books. I personally found it kind of boring since I read Kegan forward rather than backward and if you've read "The Evolving Self" then basically everything else he's done since then is just an elaboration on the key insights found there, but if you want to gently understand Kegan without getting thrown straight in the deep end of philosophy then reading his works backwards makes a lot of sense. If you were to read only one book of which Kegan is the author, though, I think "Immunity to Change" is the best bet, since it provides the most concise explanation of the theory and the most how-to knowledge of all his and Lahey's works.

A few words now on working/creating a DDO.

I don't think anyone but me would call my employer a DDO, but I molded it to be that. It's a startup, I was an early employee, and as we grew I got my hands dirty shaping the culture into the kind of place I wanted to work. There are lots of way I did this culture shaping from active nudging at the edges to major efforts to get folks aligned, but if I had to boil it down to a few slogan-sized descriptions of what I did it would be:

  • The whole person comes to work
  • Kaizen
  • Collaboration/cooperation requires that everyone feel safe
  • Everyone can be part of our culture, but not everyone is ready to be part of our culture now

Unlike a place like Bridgewater, say, we don't spend much time talking about our personal lives in formal settings and almost no one yells at anyone else, but it's a place where you can be yourself, no one is afraid for their job to tell their manager that they are causing problems (though they might not for other reasons), everyone gives and accepts feedback, and everyone supports everyone else (at least passively). If that sounds pretty ideal it's because it is.

Comment by gworley on Goodhart's Law and Genies · 2019-01-10T00:14:48.832Z · score: 4 (2 votes) · LW · GW

I guess many folks just missed this post. This is a great intro to Goodhart and why it's a problem. I think this is finally the go-to intro to Goodhart for AI discussions I've been looking for. Thanks!

Comment by gworley on [deleted post] 2019-01-09T03:20:05.028Z

So is another way of summarizing this to say "we created self-help to help us optimize ourselves for the society we created"?

Comment by gworley on [deleted post] 2019-01-09T03:19:10.143Z

I feel similarly about the footnotes. My own philosophy is to avoid them at all cost, and only include them if I absolutely cannot find a way to make the information flow with the text AND I can't not include the information while serving whatever functions I'm trying to with the writing. It's perhaps a matter of style, but given the high footnote to body percentage, I'd have preferred it if you used fewer footnotes.

Comment by gworley on An Undergraduate Reading Of: Semantic information, autonomous agency and non-equilibrium statistical physics · 2019-01-09T03:08:45.248Z · score: 2 (1 votes) · LW · GW

This is quite interesting. There's a lot of attempt to figure out how we get meaning out of information. I think of it mostly in terms of how consciousness comes to relate to information in useful ways, but that makes sense to me mostly because I'm working in a strange paradigm (transcendental phenomenological idealism). But I think I see a lot of the same sort of efforts to deal with meaning popping up in formal theories of consciousness even if meaning is the exact thing that those are driving at; I just see the two as so closely tied it's hard to address one without touching on the other.

Comment by gworley on The easy goal inference problem is still hard · 2019-01-09T02:56:34.714Z · score: 7 (1 votes) · LW · GW

This reminds me of what I like best about what had been Paul's approach (now it's more people's): a better acknowledgement of the limitations of humans and the difficulty of building models of them that are any less complex than the original human itself. I realize there are many reasons people would want not to worry about these things, main one being that additional, stronger constraints make the problem easier to solve for, but I think for actually solving AI alignment we're going to have to face these more general challenges with weaker assumptions. I think Paul's existing writing probably doesn't go far enough, but he does call this the "easy" problem, so we can address the epistemological issues surrounding one agent learning about what exists in another agents experience in the "hard" version.

Comment by gworley on Reframing Superintelligence: Comprehensive AI Services as General Intelligence · 2019-01-09T02:45:32.451Z · score: 2 (1 votes) · LW · GW

What excites me most about Eric's position since I first learned of it is that it provides a framework for safer AI systems that we might otherwise build if we were trying to target AGI. From this perspective it's valuable for setting policy and missions for AI-focused endeavors in such a way that we potentially delay the creation of AGI.

Although it might be argued that this is inevitable (last time I talked to Eric this was the impression that I got; he felt he was laying out some ideas that would happen anyway and was taking the time to explain why he thinks they will happen that way, rather than trying to nudge us towards a path), having it codified and publicized as a best course of action, it may serve on the margin to more encourage folks to work in a paradigm of doing AI develop with an eye towards incorporation in CAIS rather than as a stepping stone towards AGI. This is important because it will apply optimization pressure to ignore adding the things AGI would need since those may take extra time and cost, and if most of the short and medium term economic and academic benefits can be realized within the CAIS paradigm, then we will see a shift towards optimizing more for CAIS and less for AGI, which seems broadly beneficial from a safety standpoint because CAIS is less integrated and less agentic by design (at least for now; that might be a path from CAIS to AGI). Having this be common knowledge and the accepted paradigm of AI research is thus beneficial for pushing people away from incentive gradients that more directly lead to AGI, buying time for more safety research.

Given this, it's probably worthwhile for folks well positioned to influence other researchers to be made better aware of this work, which might be something folks here can do if they have the ears of those people (or just are those people).

Comment by gworley on Which approach is most promising for aligned AGI? · 2019-01-08T18:57:14.332Z · score: 1 (3 votes) · LW · GW

Well, I don't know that we know enough to say what is most promising, but I'm most excited to explore is my own approach that suggests we need to investigate ways to get the content of AI and human thought aligned along preference ordering. I don't think this is by any means easy, but I don't really see another practical framework in which to approach this. This framework of course admits many possible techniques, but I think it's useful to keep in mind and not get confused (as often happens in existing imitation learning papers) about how much we can know about the values of humans and AIs.

Comment by gworley on What emotions would AIs need to feel? · 2019-01-08T18:50:50.687Z · score: 4 (2 votes) · LW · GW

I do sort of expect AI to have something like emotion, because I think since humans in general tend to focus on the subjective aspect of emotions to the exclusion of their functional roles. I think of emotions as something like "brain modes" designed to reconfigure our limited mental resources to make certain behaviors more likely and other behaviors less likely. Given that AI will also be similarly bounded, I would expect to see something similar allowing the AI to alter its thinking in ways that would feel to the AI subjectively similar to our own experience of emotions and would carry out a similar role of temporarily reconfiguring the AI to be better optimized for particular scenarios.

What those specific emotions would be seems like a thing you could speculate about and *might* be fun and interesting to try but mostly because I think answers will provide hermeneutical fodder for investigating our (possibly confused) beliefs about emotions and AI.

Comment by gworley on Optimizing for Stories (vs Optimizing Reality) · 2019-01-07T22:26:09.808Z · score: 4 (2 votes) · LW · GW
I don’t think you can escape stories entirely. I would claim that as soon as you summarize your facts or data, the mere selection of which facts to present or summarize is the crafting of a story. Even dumping all your data and every observation is likely to be biased by which data you collected and what you paid attention to. What you thought were the relevant things to report to another person.

I think we can say something stronger than this: we can't escape stories at all, because stories seem to be another way of talking about ontology (maps), and we literally can't talk about anything without framing it within some ontology.

It's tempting to want to claim direct knowledge of things, even if you are an empiricist, because it would provide grounding of your observations in facts, but the reality seems to be that everything is mediated by sensory experience at the least (not to mention other ways in which experience is mediated in things as complex as humans), so we are always stuck with at least the stories that our sensory organs enable (for example, your experience of pressure waves in the air as sound). I'd say it goes even deeper than that, being a fundamental consequence of information transfer via the intentional relationship between subject and object, but we probably don't need to move beyond a pragmatic level in the current discussion.

This is also why I worry, in the context of AI alignment, that Goodharting cannot be eliminated (though maybe we can mitigate it enough to not matter): representationalism (indirect realism) creates the seed of all misalignment between reality and the measurement of it, so we will always be in active effort to work against a gradient that seeks to pull us down towards divergence.

Comment by gworley on Imitation learning considered unsafe? · 2019-01-07T18:50:59.039Z · score: 2 (1 votes) · LW · GW

If I'm taking your point correctly, it seems you're concerned about Goodharting in imitation learning. I agree, this seems a major issue, and I think people are aware of it and thinking about ways to address it.

Comment by gworley on Events in Daily? · 2019-01-03T01:45:45.830Z · score: 3 (2 votes) · LW · GW

I primarily use /daily so I had no idea about events. Maybe if it was an option to show them in the daily view? I think I probably wouldn't mind seeing them because I could just skip them so long as they are clearly labeled.

Comment by gworley on What's the best way for me to improve my English pronounciation? · 2019-01-03T01:42:39.235Z · score: 4 (4 votes) · LW · GW

I'm not sure how much speech lessons are, but there are people who will help you learn new accents by leading you through guided exercises to help you change the way you produce the phonemes in the target language. I can think of at least a couple people I know who did this and then ended up sounding much more "native" than they originally did, where it was very clear they were second-language speakers and had distinct accents tied to their first language.

A cheap alternative might be using YouTube and material you can find online. You can lead yourself through these kinds of exercises for free; you just don't get the benefit of quick feedback from someone who can hear how well you are progressing and give specific suggestions for things you might try to advance faster.

Comment by gworley on Thoughts on Q&A so far? · 2018-12-31T18:31:38.748Z · score: 20 (8 votes) · LW · GW

A few of things I've noticed.

  • I'd like it if we could give answers titles and have those show in the navigation. There are times, like in this question, where letting multiple people give answers makes sense, so letting people title their answers seems helpful there. Also just seems to go with with the way the question has a title and body text, so let the answers do the same.
  • Give the asker the ability to make one of the answers as the "correct" answer that will show first regardless of score and with a little mark showing it the accepted answer by the asker, similar to the way Stack Overflow does it. Doesn't make sense for all questions but may make sense for some.
  • Do something to the UI to better encourage people to give answers rather than leave comments. I've seen a number of questions where people leave comments instead of answers when they are clearly giving an answer. I'm not sure if these people are confused about the UI or for some reason hesitate to reply with an answer whereas a comment feels more appropriate. Maybe hiding the comments by default but making them visible if you click something to show them, since it seems they should be mainly of interest to the asker and people who are confused by the question who want to get clarification about the question.

Overall I really like the Q&A feature, especially since I think it encourages people to discuss things they otherwise might not. Obviously they could have done this before, and I know there have been a few posts specifically structured with body's of the form "hey, tell me what you think about this in the comments", but the Q&A feature helps make the explicit so you know even before you read the post how you are expected to interact with it.

Comment by gworley on Is there a standard discussion of vegetarianism/veganism? · 2018-12-31T18:21:15.968Z · score: 3 (2 votes) · LW · GW

If you haven't already you might also ask this question on the EA forum. It seems likely to me that it's more likely someone with a good answer to this would see this question over there.

Sadly I don't know of such a resource myself, but it seems like something that would be really useful. Most things giving arguments for veg*ism tend to lean at some point on a shared moral intuition that hurting the environment, killing animals, or animal suffering is bad, but it sounds like you want something more. For what it's worth, though, you can find stuff about all those questions spread around in different sources.

Comment by gworley on Akrasia is confusion about what you want · 2018-12-31T18:14:57.484Z · score: 2 (1 votes) · LW · GW

I think that's one action you could take that could help you see what I think is the real thing that will get you out of akrasia, which is "don't identify with your desires".

Comment by gworley on Conceptual Analysis for AI Alignment · 2018-12-30T21:36:04.692Z · score: 3 (2 votes) · LW · GW

Sure, this seems reasonable. For example, in my own work I think much of the value I bring to AI alignment discussions is having a different philosophical perspective and deeper knowledge of a set of philosophical ideas not wisely considered by most people thinking about the problem. However it's not clear to me how someone might take the idea you've presented and make it their work as opposed to doing something more like what I do. Thoughts on how we might operationalized your idea?

Akrasia is confusion about what you want

2018-12-28T21:09:20.692Z · score: 18 (15 votes)
Comment by gworley on Isaac Asimov's predictions for 2019 from 1984 · 2018-12-28T19:18:28.431Z · score: 16 (6 votes) · LW · GW
Education will become fun because it will bubble up from within and not be forced in from without.

Part of me wants to say "alas, he got it wrong" but maybe less wrong that you would think if you went through public education in the United States (I can't speak well to education in other countries, although education throughout the Anglo-sphere seems similar along relevant dimensions).

I'm thinking specifically of things like

  • Wikipedia (and the many more narrowly-focused wikis)
  • YouTube
  • Reddit (and to a lesser extent, Quora)
  • LessWrong (dare I add it to the list?) and other specialty forums

These all present ways in which education and learning is made accessible and can be discovered and used because people are interested, not because somebody told them they had to learn something. The experiences I'm thinking of that back these ideas up:

  • Spending hours with dozens, if not hundreds, of tabs open exploring related ideas in a wiki and getting lost in the network of related ideas, like a personally constructed episode of Connections.
  • Watching one video after another about topics of interest. For example, things I've gotten into patterns of watching on YouTube (helped along by the algorithm) just because they were interesting: amateur anthropology, literature summaries, science explainers, in-depth news reporting, and, of course, applied engineering.
  • Seeing people talk about and explore their personal experiences and learn from each out about how other people experience life differently from each other. Basically I think of Reddit like civics class if the only topic was "what universal human experiences are you missing".
  • Having a place to read and write with people with similar interests, especially since once you're in you'll easily get pointed to stuff nearby you'll likely be interested in learning more about.

It seems like if you ignore formal educational systems and look for places where people are genuinely interested in learning and sharing what they've learned, there's never been a better time to be alive if you enjoy learning things, no matter what you enjoy learning.

What self-help has helped you?

2018-12-20T03:31:52.497Z · score: 34 (11 votes)

Why should EA care about rationality (and vice-versa)?

2018-12-09T22:03:58.158Z · score: 16 (3 votes)

What precisely do we mean by AI alignment?

2018-12-09T02:23:28.809Z · score: 29 (8 votes)

Outline of Metarationality, or much less than you wanted to know about postrationality

2018-10-14T22:08:16.763Z · score: 19 (17 votes)

HLAI 2018 Talks

2018-09-17T18:13:19.421Z · score: 12 (4 votes)

HLAI 2018 Field Report

2018-08-29T00:11:26.106Z · score: 44 (19 votes)

A developmentally-situated approach to teaching normative behavior to AI

2018-08-17T18:44:53.515Z · score: 12 (5 votes)

Robustness to fundamental uncertainty in AGI alignment

2018-07-27T00:41:26.058Z · score: 7 (2 votes)

Solving the AI Race Finalists

2018-07-19T21:04:49.003Z · score: 27 (10 votes)

Look Under the Light Post

2018-07-16T22:19:03.435Z · score: 25 (11 votes)

RFC: Mental phenomena in AGI alignment

2018-07-05T20:52:00.267Z · score: 13 (4 votes)

Aligned AI May Depend on Moral Facts

2018-06-15T01:33:36.364Z · score: 9 (3 votes)

RFC: Meta-ethical uncertainty in AGI alignment

2018-06-08T20:56:26.527Z · score: 18 (5 votes)

The Incoherence of Honesty

2018-06-08T02:28:59.044Z · score: 22 (12 votes)

Safety in Machine Learning

2018-05-29T18:54:26.596Z · score: 17 (4 votes)

Epistemic Circularity

2018-05-23T21:00:51.822Z · score: 5 (1 votes)

RFC: Philosophical Conservatism in AI Alignment Research

2018-05-15T03:29:02.194Z · score: 27 (9 votes)

Thoughts on "AI safety via debate"

2018-05-10T00:44:09.335Z · score: 33 (7 votes)

The Leading and Trailing Edges of Development

2018-04-26T18:02:23.681Z · score: 24 (7 votes)

Suffering and Intractable Pain

2018-04-03T01:05:30.556Z · score: 13 (3 votes)

Evaluating Existing Approaches to AGI Alignment

2018-03-27T19:57:39.207Z · score: 22 (5 votes)

Evaluating Existing Approaches to AGI Alignment

2018-03-27T19:55:57.000Z · score: 0 (0 votes)

Idea: Open Access AI Safety Journal

2018-03-23T18:27:01.166Z · score: 64 (20 votes)

Computational Complexity of P-Zombies

2018-03-21T00:51:31.103Z · score: 3 (4 votes)

Avoiding AI Races Through Self-Regulation

2018-03-12T20:53:45.465Z · score: 6 (3 votes)

How safe "safe" AI development?

2018-02-28T23:21:50.307Z · score: 27 (10 votes)

Self-regulation of safety in AI research

2018-02-25T23:17:44.720Z · score: 33 (10 votes)

The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

2018-02-23T21:42:20.604Z · score: 15 (4 votes)

AI Alignment and Phenomenal Consciousness

2018-02-23T01:21:36.808Z · score: 10 (2 votes)

Formally Stating the AI Alignment Problem

2018-02-19T19:07:14.000Z · score: 0 (0 votes)

Formally Stating the AI Alignment Problem

2018-02-19T19:06:04.086Z · score: 14 (6 votes)

Bayes Rule Applied

2018-02-16T18:30:16.470Z · score: 12 (3 votes)

Introduction to Noematology

2018-02-05T23:28:32.151Z · score: 11 (4 votes)

Form and Feedback in Phenomenology

2018-01-24T19:42:30.556Z · score: 29 (6 votes)

Book Review: Why Buddhism Is True

2018-01-15T20:54:37.431Z · score: 23 (9 votes)

Methods of Phenomenology

2017-12-30T18:42:03.513Z · score: 6 (2 votes)

The World as Phenomena

2017-12-06T02:35:20.681Z · score: 0 (2 votes)

Towards an Axiological Approach to AI Alignment

2017-11-15T02:07:47.607Z · score: 11 (6 votes)

Towards an Axiology Approach to AI Alignement

2017-11-15T02:04:14.000Z · score: 0 (0 votes)

Doxa, Episteme, and Gnosis

2017-10-31T23:23:38.094Z · score: 4 (4 votes)

React and Respond

2017-10-24T01:48:16.405Z · score: 5 (5 votes)

Regress Thyself to the Mean

2017-10-19T22:42:08.925Z · score: 9 (4 votes)

Gnostic Rationality

2017-10-11T21:44:22.144Z · score: 34 (24 votes)

Artificial Unintelligence

2017-10-10T01:37:22.000Z · score: 0 (0 votes)

Post Fit Review

2017-09-28T23:52:31.063Z · score: 11 (3 votes)

Cognitive Empathy and Emotional Labor

2017-09-26T20:39:48.877Z · score: 5 (1 votes)

Cognitive Empathy and Emotional Labor

2017-09-26T20:36:41.414Z · score: 0 (0 votes)

Is Feedback Suffering?

2017-09-09T22:20:51.323Z · score: 1 (1 votes)

Embracing Metamodernism

2017-08-18T19:23:15.521Z · score: 1 (1 votes)