Posts
Comments
I am not the best at writing thorough comments because I am more of a Redditor than a LessWronger, but I just want you to know that I read the entire post over the course of ~2.5 hours and I support you wholeheartedly and think you're doing something very important. I've never been part of the rationalist "community" and don't want to be (I am not a rationalist, I am a person who strives weakly for rationality, among many other strivings), particularly after reading all this, but I definitely expected better out of it than I've seen lately. But perhaps I shouldn't; the few self-identified rationalists I've interacted with one on one have mostly seemed like... at best, very strange people to me. And Eliezer has always, honestly, struck me as a dangerous narcissist whose interest in truth is secondary to his interest in being the Glorious Hero. I don't want to go to the effort of replying to specific things you said - and you don't know who I am and probably won't read this anyway - but yeah, just, I'm glad you said them.
Well, as I attempted to express in the original comment, I am not a Shaiva, but rather I had mystical experiences and things as a teen that led me to invent my own religion from scratch which has similarities with various other belief systems, and Kashmir Shaivism is one of them. For the most part however it's just a kind of background element of my existence, part of my ontology, and not something I put much attention towards actively anymore. In practice I'm effectively an atheist physicalist like everyone else here. It's just... there's also something that lurks beneath. Or there used to be. I've gotten more disillusioned, more empty-souled and this-worldly as I've gotten older, and I don't really know how to get back the way I used to feel. Probably psychedelics is the only way; meditation doesn't do anything for me.
I think you're wrong that Level 4 is rare. It describes everyday reality in the upper strata of any cult - the psychopathic leadership class who make up shit for everyone else to believe, or claim to believe for status, or etc, and compete for status among those followers. And there are a LOT of cults in modern society, including organizations not traditionally perceived as cults, such as political ideologies.
I'm sure you've already thought of this, and I know nothing about this area of biology, but isn't it possible that the genes coding for intelligence more accurately code for the developmental trajectory of the brain, and that changing them would not in fact affect an adult brain?
I think people have "criticized" Minecraft for being unclear what the point is, and being more of a toy or sandbox than a "game."
Myself included. I can't play Minecraft; it's far too open-ended, and makes me feel anxious and overwhelmed. "Wtf am I supposed to do??" I want a game to give me two or three choices max at every decision point, not an infinite vector space of possibilities through which I cannot sort or prioritize.
This post though is about one of my big obsessions: trying to figure out how to design a game (computer or tabletop or both) which makes cooperation fun. And I mean, fun in the way Diablo is fun. Addictive, power fantasy feeling, endless sequence of dopamine hits, sexy. The problem is that the only way to produce that Diablo-flow is to enable people to act automatically, reacting to signs and triggers with preprogrammed responses so that they can sink down into the animalistic part of their brain that hunts and stalks and pounces on things to tear them apart without thought or simulation.
But learning to negotiate with others is the exact opposite of that, and is the main reason we have the effort-intensive simulation system to begin with - so the problem of making a game that is simultaneously compelling on a primal level, and centers on conflict resolution rather than just conflict... is hard.
The only thing that seems to have the same kind of flow in it to me is dance and other group rituals (such as those in religion that hasn't ossified to mere passionless false beliefs yet), which don't really help with the whole "training negotiation" thing (though they do induce people to align with one another on an emotional level) and also cannot easily be turned into video games or TTRPGs.
I never actually said that all these notions are constructed and fake, only that some are. Clearly some aren't. There are false positives and false negatives. I feel as if you're arguing against a straw man here.
If I were Bob I'd have told her to fuck off long ago and stopped letting some random person berate me for being lazy just like my parents always have. This is basically guilt-tripping, not a beneficial way of approaching any kind of motivation, and it is absolutely guaranteed to produce pushback. But then, I'm probably not your target audience, am I?
Btw just to be clear, I think Said Achmiz explained my reaction better than I, who habitually post short reddit-tier responses, can. My specific issue is that Alice seems to be acting as if it's any of her business what Bob does. It is not. Absolutely nobody likes being told they're not being ethical enough. It's why everyone hates vegans. As someone who doesn't like experiencing such judgmental demands, I would have the kneejerk emotional reaction to want to become less of an EA just to spite her. (I would not of course act on this reaction, but I would start finding EA things to be in an ugh field because they remind me of the distress caused by this interaction.)
Holy heck I have been enlightened. And by contemplating nothingness too! Thanks for the clarification, it all makes sense now.
I really enjoy this sequence but there's a sticking point that's making me unable to continue until I figure it out. It seems to me rather obvious that... utility functions are not shift-invariant! If I denominate option A at 1 utilon and option B at 2 utilons, that means I am indifferent between a certain outcome of A and a 50% probability of B - and this is no longer the case if I shift my utility function even slightly. Ratios of utilities mean something concrete and are destroyed by translation. Since your entire argument seems to rest on that inexplicably not being the case, I can't see how any of this is useful.
I understand all this logically, but my emotional brain asks, "Yeah, but why should I care about any of that? I want what I want. I don't want to grow, or improve myself, or learn new perspectives, or bring others joy. I want to feel good all the time with minimal effort."
When wireheading - real wireheading, not the creepy electrode in the brain sort that few people would actually accept - is presented to you, it is very hard to reject it, particularly if you have a background of trauma or neurodivergence that makes coping with "real life" difficult to begin with, which is why so many people with brains like mine end up as addicts. Actually, by some standards, I am an addict, just not of any physical substance.
And to be honest, as a risk-averse person, it's hard for me to rationally argue for why I ought to interact with other people when AIs are better, except the people I already know, trust, and care about. Like, where exactly is my duty to "grow" (from other people's perspective, by other people's definitions, because they tell me I ought to do it) supposed to be coming from? The only thing that motivates me, sometimes, to try to do growth-and-self-improvement things is guilt. And I'm actually a pretty hard person to guilt into doing things.
That's a temporary problem. Robot bodies will eventually be good enough. And I've been a virgin for nearly 26 years, I can wait a decade or two longer till there's something worth downloading an AI companion into if need be.
Neither of these really describes what childhood is for. Both of them are inventions of the modern WEIRD society. I'd suggest you read "Anthropology of Childhood: Cherubs, Chattels, Changelings" for a wider view on the subject... it's pretty bleak though. The very idea that there is such a thing as an optimal childhood parents ought to strive to provide their children... is also a modern, Western, extremely unusual idea, and throughout most of history, in most cultures, they were just... little creatures that would eventually be adults and till then either got in the way or were used for something.
The norm appears to be "benevolent neglect", at best - that is, children are not (outside of our Western bubble of reality, as well as East Asia which independently invented some of the same norms) actively taught or guided towards anything; mostly they are ignored and they teach themselves everything they need to know by mimicking adults. People spend time with their children, but it's rarely a goal explicitly striven for (the way it is for Western parents); it's just a side effect of their existing at all.
To be honest, I look forward to AI partners. I have a hard time seeing the point of striving to have a "real" relationship with another person, given that no two people are really perfectly compatible, no one can give enough of their time and attention to really satisfy a neverending desire for connection, etc. I expect AIs to soon enough be better romantic companions - better companions in all ways - than humans are. Why shouldn't I prefer them?
Great, apparently I'm in just the right place... I'm always alone and have few friends who might influence me to give up my wacky ideas! Wonderful.....
crickets
Those stories are surprisingly coherent and compelling. They were actually fun to read!
I'm not sure how useful the concept of boundary placement rebellion is, though. It certainly is a thing, but it's also something basically everyone engages in. I pretty much constantly do it... though maybe that says more about me than anything...
"Thou strivest ever. Even in thy yielding, thou strivest to yield; and lo! thou yieldest not. Go thou into the outermost places, and subdue all things. Subdue thy fear and thy distrust. And then - YIELD." - Aleister Crowley
I'm never really sure what there's any point in saying. My main interests have nothing to do with AI alignment, which seems to be the primary thing people talk about here. And a lot of my thoughts require the already existing context of my previous thoughts. Honestly, it's difficult for me to communicate what's going on in my head to anyone.
No, it's called "lying". The text that he produces as a result of these social pressures does not reflect his actual thought processes. You can't judge a belief on the basis of a bunch of ex post facto arguments people make up to rationalize it - the method by which they came to hold the belief is much more informative, and for those of us with very roundabout styles of thinking (such as myself) being forced into this self-censorship and modification of our thought patterns into something "coherent" and easy to read actually destroys all the evidence of how we actually came to the idea, and thus destroys much of your ability to effectively examine its validity!
I feel the same as Adrian and Cato. I am very much the opposite of a rigorous thinker - in fact, I am probably not capable of rigor - and I would like to be the person who spews loads of interesting off the wall ideas for others to parse through and expand upon those which are useful. But that kind of role doesn't seem to exist here and I feel very intimidated even writing comments, much less actual posts - which is why I rarely do. The feeling that I have to put tremendous labor into making a Proper Essay full of citations and links to sequences and detailed arguments and so on - it's just too much work and not worth the effort for something I don't even know anyone will care about.
This makes me wonder if some proportion of "masculine" gay men are actually transwomen (of the early onset type) with autoandrophilia. I may even fit into that category myself. I didn't care about masculinity and in fact found it somewhat abhorrent and not-me-ish until I started getting off to more masculine looking guys in porn. (When I first saw porn when I was 12 I mainly focused on twinks and wanted to look like them, and there's still a part of me that feels that way, which wars with the part that wants to bulk up because masc dudes are also hot - and usually wins, because bulking is hard and I would rather read books.)
Of course, my natural femininity is not tremendous (I wasn't flamboyant as a child and as far as I know never have been - I've always thought feminine-acting men were creepy - but I did flirt with identifying as nonbinary during my late teens, and used to have multiple female alters during the period where I thought I had multiple personalities), and most of my femininity is the result of misandry taught by the media and my mother (I believed for most of my childhood and early teens that masculinity is disgusting and bestial, and that only women can be powerful / noble, but later realized that like all other disgusting and bestial things, masculinity is sexy as fuck, which helped me get out of my misandry phase.)
Nowadays I think my gender identity is probably something like "true hermaphrodite / omega (as in the omegaverse fanfiction trope) male", which unfortunately is not something that one can currently medically transition to, and I experience no dysphoria (and to be honest, the only reason I think it would be cool to have both male and female genitals is because it seems too asymmetric and unbalanced not to, and I'm very Libra [yes I know astrology isn't real, but it's still a helpful and / or fun language to describe personalities with]).
Well - actually, it's possible I do experience dysphoria, but in which direction changes with my mood (I sometimes don't feel masculine enough), and there's an element of The Paraphilia Which Must Not Be Named [note: if you ask me, I will not name it, and I will neither confirm nor deny guesses, but you can probably figure it out based on what I'm not saying] which also interacts in weird ways with the whole thing, and overall I just find gender and sexuality stuff tiresome and confusing and sort of wish I didn't have to deal with it.
Thanks for coming to my rambly asf TED talk.
This is interesting, and imo dystopian and dreadful, but it doesn't belong on Lesswrong. I downvoted.
I feel like consequentialists are more likely to go crazy due to not being grounded in deontological or virtue-ethical norms of proper behavior. It's easy to think that if you're on track to saving the world, you should be able to do whatever is necessary, however heinous, to achieve that goal. I didn't learn to stop seeing people as objects until I leaned away from consequentialism and toward the anarchist principle of unity of means and ends (which is probably related to the categorical imperative). E.g. I want to live in a world where people are respected as individuals, so I have to respect them as individuals - whereas maximizing individual-respect might lead me to do all sorts of weird things to people now in return for some vague notion of helping lots more future people.
I was about to mention Piaget, but you referred to him at the end of the post. Definitely seems relevant, since we noticed the possible connection independently.
This reminds me strongly of the anarchist principle of unity of means and ends, which is why anarchists aren't into violent revolution anymore - you can't end coercion by coercive means.
Ooh! I don't know much about the theory of reinforcement learning, could you explain that more / point me to references? (Also, this feels like it relates to the real reason for the time-value of money: money you supposedly will get in the future always has a less than 100% chance of actually reaching you, and is thus less valuable than money you have now.)
It seems to me that the optimal schedule by which to use up your slack / resources is based on risk. When planning for the future, there's always the possibility that some unknown unknown interferes. When maximizing the total Intrinsically Good Stuff you get to do, you have to take into account timelines where all the ants' planning is for nought and the grasshopper actually has the right idea. It doesn't seem right to ever have zero credence of this (as that means being totally certain that the project of saving up resources for cosmic winter will go perfectly smoothly, and we can't be certain of something that will literally take trillions of years), therefore it is actually optimal to always put some of your resources into living for right now, proportional to that uncertainty about the success of the project.
computers have no consciousness
Um... citation please?
1. Who are the customers actually buying all these products so that the auto-corporations can profit? They cannot keep their soulless economy going without someone to sell to, and if it's other AIs, why are those AIs buying when they can't actually use the products themselves?
2. What happened to the largest industry in developed countries, the service industry, which fundamentally relies on having an actual sophont customer to serve? (And again, if it's AIs, who the hell created AIs that exist solely to receive services they cannot actually enjoy, and how did they make money by doing that?)
3. Why didn't shareholders divest from auto-corporations upon realizing that they were likely to lead to ruin? (Don't say "they didn't realize it till too late", because you, right now, know it's a bad idea, and you don't even have money on the line.)
I ask these because, to be honest, I think this scenario is extremely far-fetched and unlikely. The worst thing that would happen if auto-corporations become a thing, in my mental model, is that existing economic inequalities would be permanently exacerbated due to the insane wealth accrued by their shareholders - because the only currently likely route to AGI, large language models, already understand what we actually mean when we say to maximize shareholder value, and won't paperclip-maximize, because they're not stupid, and they use language the same way humans do.
I've never had a job in my life - yes really, I've had a rather strange life so far, it's complicated - but I've been reading and thinking about topics which I now know are related to operations for years, trying to design (in my head...) a system for distributing the work of managing a complex organization across a totally decentralized group so that no one is in charge, with the aid of AI and a social media esque interface. (I've never actually made the thing, because I keep finding new things I need to know, and I'm not a software engineer, just a designer.)
So, I think I have some parts of the requisite skillset here, and a ton of intuition about how to run systems efficiently built up from all the independent studying I've done - but absolutely no prior experience with basically anything in reality, except happening to (I believe) have the right personality for operations work. Should I bother applying?
I don't know what to think about all that. I don't know how to determine what the line is between having qualia and not. I just feel certain that any organism with a brain sufficiently similar to those of humans - certainly all mammals, birds, reptiles, fish, cephalopods, and arthropods - has some sort of internal experience. I'm less sure about things like jellyfish and the like. I suppose the intuition probably comes from the fact that the entities I mentioned seem to actively orient themselves in the world, but it's hard to say.
I don't feel comfortable speculating which AIs have qualia, or if any do at all - I am not convinced of functionalism and suspect that consciousness has something to do with the physical substrate, primarily because I can't imagine how consciousness can be subjectively continuous (one of its most fundamental traits in my experience!) in the absence of a continuously inhabited brain (rather than being a program that can be loaded in and out of anything, and copied endlessly many times, with no fixed temporal relation between subjective moments.)
I don't know anything about colab, other than that the colab notebooks I've found online take a ridiculously long time to load, often have mysterious errors, and annoy the hell out of me. I don't know enough AI-related coding stuff to use it on my own. I just want something plug and play, which is why I mainly rely on KoboldAI, Open Assistant, etc.
We're not talking about sapience though, we're talking about sentience. Why does the ability to think have any moral relevance? Only possessing qualia, being able to suffer or have joy, is relevant, and most animals likely possess that. I don't understand the distinctions you're making in your other comment. There is one, binary distinction that matters: is there something it is like to be this thing, or is there not? If yes, its life is sacred, if no, it is an inanimate object. The line seems absolutely clear to me. Eating fish or shrimp is bad for the same reasons that eating cows or humans is. They are all on the exact same moral level to me. The only meaningful dimension of variation is how complex their qualia are - I'd rather eat entities with less complex qualia over those with more, if I have to choose. But I don't think the differences are that strong.
Just to be That Guy I'd like to also remind everyone that animal sentience means vegetarianism, at the very least (and because of the intertwined nature of the dairy, egg, and meat industries, most likely veganism) is a moral imperative, to the extent that your ethical values incorporate sentience at all. Also, I'd go further to say that uplifting to sophonce those animals that we can, once we can at some future time, is also a moral imperative, but that relies on reasoning and values I hold that may not be self-evident to others, such as that increasing the agency of an entity that isn't drastically misaligned with other entities is fundamentally good.
Most Wikipedia readers spend less than a minute on a page?? I always read pages all the way through... even if they're about something that doesn't interest me much...
Welcome! And yes, this is a thing people have talked about a lot, particularly in the context of outer versus inner alignment (the outer optimizer, evolution, designed an inner optimizer, humans, who optimize for different things, like pleasure etc, than evolution does, but ended up effectively becoming a "singularity" from its point of view). It's cool that you noticed this on your own!
This is my thought exactly. I would try it, but I am poor and don't even have a GPU lol. This is something I'd love to see tested.
So basically... LMs have to learn language in the exact same way human children do: start by grasping the essentials and then work upward to complex meanings and information structures.
This is a fantastically good point. I've often seen this failure mode and not had a name for it, such as when someone I know complains about his political opponents having a self-contradictory ideology - I always have to correct him that in fact, different people in roughly the same camp are contradicting one another, but each individual perspective is self-consistent. Now I have a name for that phenomenon!
I'm not a biologist either. This post is me handwaving. Thanks for the reference!
Humans are not eusocial. That was Edward O. Wilson being dramatic. We don't have a biological caste distinction between reproducers and non-reproducers.
That's really weird. Why do you identify so strongly with a sequence of nucleotides? Isn't it more important that a child inherits your memes?
That paper about economic drivers of biological complexity is fascinating! In particular I am amazed I never noticed that lekking is an auction. The paper lends some credence to my intuition that capitalism is actually isomorphic to the natural state. Are you the Phelps that was involved in writing it?
Also: I wonder if you'd be interested in my vague notion that genes trade with one another using mutability as a currency.
I've long thought that a "humans+AIs hive mind" would end up being the superintelligence in control of the future - not a purely AI one - so this is a great question and I'm glad to see people researching this!
The only counterintuitive thing about this post is that you expect the readers to find it counterintuitive! It's pretty obvious to those of us who remember our childhoods and have enough self-awareness to notice ourselves reliving them over and over...
To a limited extent, I have begun to the past month or so. Staying on a friend's organic farm. It's not much, but it's something. I definitely feel healthier, physically and emotionally. But less time to think. So it's a tradeoff.
This reminds me strongly of Wittgenstein's notion of "family resemblances" as a more reasonable replacement for definitions. The way mental illnesses are diagnosed in the DSM is similar - if you have X out of N possible symptoms, then you have the disease. Maybe womanhood (forgive my comparison with a disease!) is similar nowadays.
Eliezer, or somebody better at talking to humans than him, needs to go on conservative talk shows - like, Fox News kind of stuff - use conservative styles of language, and explain AI safety there. Conservatives are intrinsically more likely to care about this stuff, and to get the arguments why inviting alien immigrants from other realms of mindspace into our reality - which will also take all of our jobs - is a bad idea. Talk up the fact that the AGI arms race is basically as bad as a second cold war only this time the "bombs" could destroy all of human civilization forever, and that anyone who develops it should be seen internationally as a terrorist who is risking the national security of every country on the planet.
To avoid the topic becoming polarized, someone else should at the same time go on liberal talk shows and explain how unaligned AGI, or even AGI aligned to some particular ideological group, is the greatest risk of fascism in human history and could be used to permanently lock the planet into the worst excesses of capitalism, fundamentalism, oppression, and other evils, besides the fact that (and this is one of the better arguments for abortion) it is immoral to bring a child (including AGI) into the world without being very sure that it will have a good life (and most people will not think that being addicted to paperclips is a good life).
11. I like asking LLMs to write me lists of interesting things, please add more training data for that.
I looked at the voting algorithm thing. Far too complicated and unpleasant even to read the options. No one is going to bother. You'd be better off just using score voting.
The hilarious thing is that if LLMs are aligned by default, all this could actually be true.