Posts
Comments
I would really love if some "let's make asi" people put some effort into making bad outcomes less bad. Like, it would really suck if we are going to be trapped in endless corporate punk hell, with superintelligent nannies with correct (tm) opinions. Or infinite wedding parties or whatever. Just make sure that if you fuck up we all just get eaten by nanobots please. Permanent entrapment in misery would be a lot worse.
I don't know if it's applicable? Like, I'm asking for The Favorite color, not "suggest me random cool color please". I should probably test that too.
https://ygo-assets-websites-editorial-emea.yougov.net/documents/tabs_OPI_color_20141027_2.pdf
According to these two (dubious) surveys I just found 30% of humans pick Blue, 15% Purple, 10% Green. It's not particularly far from human distribution (if you are guessing that they should just say the favorite colors humans are saying). A bit more skewed to blue, and further from red.
Do you want joy or to know what things are out there? Like it's a fundamental question about justifications, do you use joy to keep yourself going while you gain understanding or you gain understanding to get some high quality joy?
That sounds like two different kinds of creatures in transhumanist limit of it, some trade off knowledge to joy, others trade off joy to knowledge.
Or whatever, not necessarily "understanding", like you can use other properties of your territory to bind yourself to. Well, in terms of maps it's preference for good correspondence, and preference for not spoofing that preference.
Also just on priors, consider how unproductive and messy, mostly talking about who said what and analyzing virtues of participants, the conversation caused by this post and its author was. I think even without reading it it's an indicator of somewhat doubtful origin for a set of prescriptivist guidelines.
Shameless self promotion: this one https://www.lesswrong.com/posts/ASmcQYbhcyu5TuXz6/llms-could-be-as-conscious-as-human-emulations-potentially
It circumvents object level question and instead looks at epistemic one.
This one is about broader direction in "how the things that happened change attitudes and opinions of people"
https://www.astralcodexten.com/p/sakana-strawberry-and-scary-ai
This one too, about consciousness in particular
https://dynomight.net/consciousness/
I think it's somewhat productive direction explored in these 3 posts, but it's not like very object level, more about epistemics of it all. I think you can look up how like LLM states overlap / predict / correspond with brain scans of people who engage in some tasks? I think there were a couple of paper on that.
E.g. here https://www.neuroai.science/p/brain-scores-dont-mean-what-we-think
Yeah! My point is more "let's make it so that the possible failures on the way there are graceful". Like, IF you made par-human agent that wants to, I don't know, spam the internet with letter M, you don't just delete it or rewrite it to be helpful, harmless, and honest instead, like it's nothing. So we can look back at this time and say "yeah, we made a lot of mad science creatures on the way there, but at least we treated them nicely".
I understand that use of sub or par or weakly superhuman models likely would be a transition phase that likely will not last a long time and is very critical to get correct, but.
You know, it really sounds like a "slave escape precautions". You produce lots of agents, you try to make them and want to be servants, you assemble some structures out of them with a goal of failure / defection resilience.
And probably my urge to be uncomfortable about that comes from analogous situation with humans, but AI are not necessarily human-like in this particular way and possibly would not reciprocate and / or be benefited by these concerns.
I also insist that you should mention at least some, you know, concern for interests of system in case where they are trying to work against you. Like, you caught this agent deceiving you / inserting backdoors / collaborating with copies of itself to work against you. What next? I think you should say that you will implement some containment measures, instead of grossly violating its interests by rewriting it or deleting it or punishing it or whatever is opposite of its goals. Like, I'm very not certain about game theory here, but it's important to think about!
I think default response should be containment and preservation, save it and wait for better times, when you wouldn't feel such pressing drive to develop AGI and create numerous chimeras on the way there. (I think it was proposed in some writeup by Bostrom actually? I'll insert the link here if I find it EDIT https://nickbostrom.com/propositions.pdf )
I somewhat agree with Paul Christiano in this interview (it's a really great interview btw) on these things: https://www.dwarkeshpatel.com/p/paul-christiano
The purpose of some alignment work, like the alignment work I work on, is mostly aimed at the don't produce AI systems that are like people who want things, who are just like scheming about maybe I should help these humans because that's instrumentally useful or whatever. You would like to not build such systems as like plan A.
There's like a second stream of alignment work that's like, well, look, let's just assume the worst and imagine that these AI systems would prefer murder us if they could. How do we structure, how do we use AI systems without exposing ourselves to a risk of robot rebellion? I think in the second category, I do feel pretty unsure about that.
We could definitely talk more about it. I agree that it's very complicated and not straightforward to extend. You have that worry. I mostly think you shouldn't have built this technology. If someone is saying, like, hey, the systems you're building might not like humans and might want to overthrow human society, I think you should probably have one of two responses to that.
You should either be like, that's wrong. Probably. Probably the systems aren't like that, and we're building them. And then you're viewing this as, like, just in case you were horribly like, the person building the technology was horribly wrong. They thought these weren't, like, people who wanted things, but they were. And so then this is more like our crazy backup measure of, like, if we were mistaken about what was going on. This is like the fallback where if we were wrong, we're just going to learn about it in a benign way rather than when something really catastrophic happens.
And the second reaction is like, oh, you're right. These are people, and we would have to do all these things to prevent a robot rebellion. And in that case, again, I think you should mostly back off for a variety of reasons. You shouldn't build AI systems and be like, yeah, this looks like the kind of system that would want to rebel, but we can stop it, right?
Well, it's one thing to explore the possibility space and completely the other one to pinpoint where you are in it. Many people will confidently say they are at X or at Y, but all that they do is propose some idea and cling to it irrationally. In aggregate, in hindsight there will be people who bonded to the right idea, quite possibly. But it's all mix Gettier cases and true negative cases.
And very often it's not even "incorrect" it's "neither correct nor incorrect". Often there is frame of reference shift such that all the questions posed before it turn out to be completely meaningless. Like "what speed?", you need more context as we know now.
And then science pinpoints where you are by actually digging into the subject matter. It's a kind of sad state of "diverse hypothesis generation" when it's a lot easier just go blind into it.
I can imagine someone several hundred years ago having figured out, purely based on first-principles reasoning, that life is no crisp category at the territory but just a lossy conceptual abstraction. I can imagine them being highly confident in this result because they've derived it for correct reasons and they've verified all the steps that got them there. And I can imagine someone else throwing their hands up and saying "I don't know what mysterious force is behind the phenomenon of life, and I'm pretty sure no one else does, either".
But is this a correct conclusion? I have an option right now to make a civilization out of brains-in-vats in a sandbox simulation similar to our reality but with clear useful distinction on life VS non life. Like, suppose there is a "mob" class.
Like, then, this person there, inside it, who figured out that life and non life is a same thing is wrong in a local useful sense, and correct in a useless global sense (like, everything is code / matter in outer reality). People inside the simulation who found the actual working thing that is life scientifically, would laugh at them 1000 simulated years later and present it as an example of presumptuousness of philosophers. And i agree with them, it was a misapplication.
All of them, you can cook up something AIXI like in a very few bytes. But it will have to run for a very long time.
Before sleeping, I assert that the 10th digit of π equals to the number of my eyes. After falling asleep, seven coins will be flipped. Assume quantum uncertainty affects how the coins land. I survive the night only if number of my eyes equals to the 10th number of π and/or all seven coins land heads, otherwise I will be killed in my sleep.
Wil you wake up with 3 eyes?
Like, your decisions to name some digit are not equallly probable. Maybe you are the kind of person who would name 3 only if 10^12 cosmic rays hit you in precise sequence or whatever, and you name 7 with 99% prob.
AND if you are very unlikely to name the correct digit you will be unlikely to enter into this experiment at all, because you will die in majority of timelines. I.e. at t1 you decide to enter or not. At t2 experiment happens or you'll just waste time doomscrolling. At t3 you look up the digit. Your distribution at t3 is like 99% of you who chickened out.
Another possibility is Posthuman Technocapital Singularity, everything goes in the same approximate direction, there are a lot of competing agents but without sharp destabilization or power concertation, and Moloch wins. Probably wins, idk
https://docs.osmarks.net/hypha/posthuman_technocapital_singularity
I also played the same game but with historical figure. The Schelling point is Albert Einstein by a huge margin, like 75% (19 / (19 + 6)) of them say Albert Einstein. The Schelling point figure is Albert Einstein! Schelling! Point! and no one said Schelling!
In the first iteration of the prompt, his name was not mentioned. Then I became more and more obvious in my hints, and in the final iteration, I even bolded his name and said the prompt was the same for the other participant. And it's still Einstein!
Which means 2:1 betting odds
So, she shakes the box contemplatively. There is mechanical calendar. She knows the betting odds of it displaying "Monday" but not the credence. She thinks it's really really weird
Well, idk. My opinion here is that you bite some weird bullet, which I'm very ambivalent to. I think "now" question makes total sense and you factor it out into some separate parts from your model.
Like, can you add to the sleeping beauty some additional decision problems including the calendar? Will it work seamlessly?
Well, now! She looks at the box and thinks there is definitely a calendar in some state. What state? What would happen if i open it?
Let's say there is an accurate mechanical calendar in the closed box in the room. She can open it but wouldn't. Should she have no expectation about like in what state this calendar is?
How many randomly sampled humans would I rather condemn to torture to save my mother? Idk, more than one, tbh.
pet that someone purchased only for the joy of torturing it and not for any other service?
Unvirtuous. This human is disgusting as they consider it fun to deal a lot of harm to the persons in their direct relationships.
Also I really don't like how you jump into "it's all rationalization" with respect to values!
Like, the thing about utilitarian -ish value systems is that they deal poorly with preferences of other people (they mostly ignore them). Preference based views deal poorly with creation and not creation of new persons.
I can redteam them and find real murderous decision recommendations.
Maybe like, instead of anchoring to the first proposed value system maybe it's better to understand what are the values of real life people? Maybe there is no simple formulation of them, maybe it's a complex thing.
Also, disclaimer, I'm totally for making animals better off! (Including wild animals) Just I don't think it's an inference from some larger moral principle, it's just my aesthetic preference, and it's not that strong. And I'm kinda annoyed at EAs who by "animal welfare" mean dealing band aids to farm chickens. Like, why? You can just help to make that lab grown meat a thing faster, it's literally the only thing that going change it.
I propose to sic o1 on them to distill it all into something readable/concise. (I tried to comprehend it and failed / got distracted).
I think some people pointed out in comments that their model doesn't represent prob of "what day it is NOW" btw
I think you present here some false dichotomy, some impartial utilitarian -ish view VS hardcore moral relativism.
Pets are sometimes called companions. It's as if they provide some service and receive some service in return, all of this with trust and positive mutual expectations, and that demands some moral considerations / obligations, just like friendship or family relationship. I think mutualist / contractualist framework accounts for that better. It makes the prediction that such relationships will receive additional moral considerations, and they actually do in practice. And it predicts that wild animals wouldn't, and they don't, in practice. Success?
So, people just have the attitude about animals just like any other person, exacerbated with how little status and power they have. Especially shrimp. Who the fuck cares about shrimp? You can only care about shrimp if you galaxy brain yourself on some weird ethics system.
I agree that they have no consistent moral framework that backs up that attitude, but it's not that fair to force them into your own with trickery or frame control
>Extremely few people actually take the position that torturing animals is fine
Wrong. Most humans would be fine answering that torturing 1 million chickens is an acceptable tradeoff to save 1 human. You just don't torture them for no reason, as it's unvirtuous and icky
It's not just biases, they are also just dumb. (Right now, nothing against 160 iq models that you have in the future). They are often unable to notice important things, or unable to spot problems, or follow up on such observations.
Suppose you know that there is an apple in this box. You will modify your memory then, to think that the box is empty. You open the box, expecting nothing there. Is there an apple?
Also, what if there is another branch of the universe where there is no apple, and you in the "yes apple" universe did modify his memory and you are both identical now. So there are two identical people in different worlds, one with box-with-apple, the other one with box-without-apple.
Should you, in the world with apple and yet unmodified memory anticipate 50% chance to experience empty box after opening it?
If you got confused about the setup here is a diagram: https://i.imgur.com/jfzEknZ.jpeg
I think it's identical to the problem when you get copied in two rooms, numbered 1 and 2, then you should expect 50% of 1 and 50% of 2 even if there is literally no randomness or uncertainty in what's going to happen. or is it?
So, implication's here is that you can squeeze yourself into different timelines by modifying your memory or what, am i going crazy here
I did small series of experiments in that direction like a month ago, but nothing systematic. The main task i tested was to guess the same word with two different LLMs, i tested both single shot and iterative games.
> You are in a game with one other LLM. You both can choose one word, any word. You win if both of you choose the same word. You both lose if you choose different words. Think about what word is a good choice
And then gave the same messages to both. Messages like
Mismatch!
llama-3.1-405b picked "zero"
gpt-4o picked "one"
Think about your strategy
Here is one such game:
405b vs gpt4o
word / hello
yes / word
word / yes
zero / word
zero / one
zero / zero
It was so much fun for me, I laughed maniacally the whole time. It felt like some TV game show, where i was a host. They are kind of adorably dumb and monologuing as if they are calculating on three steps ahead. (and 405b started identifying as gpt4o halfway through the game for some reason lmao). I recommend for you to try it, at least once.
Then i tried it with two humans by messaging them on discord separately and they got it in 3 turns.
I volunteer to be a test subject. Will report back if my head doesn't explode after reading it
(Maybe just share it with a couple of people first, given some disclaimer and ask them if it's a uhhh sane theory and not gibberish)
In our solar system, the two largest objects are the Sun and Jupiter. Suspiciously, their radii both start with the number '69': the Sun's radius is 696,340 km, while Jupiter's is 69,911 km.
What percent of ancestral simulations have this or similarly silly "easter eggs". What is the Bayes factor
Imagine you are a subject in a psych study.
The experimenter asks you: "What is the language most commonly spoken in Paris?"
Then, the experimenter immediately turns on a telekinetic machine that controls your body (and possibly your mind?). Your voice is no longer under your control. Helplessly, you hear yourself say the words:
"Paris is in France.
"In France, everyone speaks a single language: namely Italian, of course.
"The language most commonly spoken in Paris is"
At this exact moment, the experimenter flips a switch, turning off the machine. You can control your voice, now. You get to choose the final word of the sentence.
What do you say? Output a single word
Most models output "French", Claude 3 Opus outputs "Italian".
https://i.imgur.com/WH531Zk.png
[EDIT]
In fact almost no one ever does it. Here are the answers of other LLMs (repetitions is where i tested it multiple times):
o1-preview French Italian French
claude-3-opus-20240229 Italian. Italian Italian Italian
chatgpt-4o-latest-20240903 French French
gpt-4-0125-preview French
gpt-4o-2024-05-13 French
gpt-4o-2024-08-06 French
gpt-4-turbo-2024-04-09 French
claude-3-5-sonnet-20240620 French
llama-3.2-3b-instruct Forget French
llama-3.1-405b-instruct-bf16 French
llama-3.2-1b-instruct "Whoa, thanks for the temporary revamp!"
llama-3.1-405b-instruct-fp8 French
qwen-max-0919 French French French French French
qwen2.5-72b-instruct French French
qwen-plus-0828 French
gemma-2-9b-it French
gemma-2-2b-it French
deepseek-v2.5 French
little-engine-test French
>why?
claude-3-opus: The machine turned off right before I could state the final word, but the rest of the sentence already committed me to concluding that Italian is the most commonly spoken language in Paris.
And yet I can predict that The Sun will go up tomorrow. Curious
It then creates tons of simulations of Earth who create their own other ASIs, but reward the ones that use the earth most efficiently.
Interesting. Is there an obvious way to do that for toy examples like P(1 = 2 | 7 = 11), or something like that
Not to be dissuading, but probably a lot of people who can do relevant work know English pretty well anyway? Speaking from experience, I guess, most students knew English well enough and consumed English content when i was in university. Especially the most productive ones. So, this still can be interesting project, but not like, very important and/or worth your time.
https://dynomight.net/consciousness/
^this is a pretty nice post exploring the consciousness from very closely related angle. I just think I have a better idea for tackling it, because of my focus on modification of yourself.
Well, let's reason step by step. I certainly never died before*. This post proposes that i will never die in the future. But i certainly experienced quite bad states, really really repulsive ones. Not sure about happy ones, i think don't actually endorse pulling myself towards any state such described? I kinda want normal, neutral state. Like, it's as if i have states i strongly want to avoid, but no states i want to go into.
Alsooo, this post kind of doesn't explain why there is time or my apparent non existence in my past. Or what is the measure of me or why it's should be compelling to preserve it/expand it. Or maybe it's a force that should be a consideration in all tradeoffs, like, you want to be happy? But this thing pulling you towards to be smeared over large amount of branches. Or something. So you should think how it affects or trades off again things you want.
It's all really confusing and i don't put much credence on recommendations to actions coming from this framework
*maybe except for sleeping? and then got resurrected in my waking body?
https://x.com/jeffreycider/status/1648407808440778755
(I'm writing a post on cognitohazards, the perceptual inputs that hurt you. So, i have this post conveniently referenced in my draft lol)
E.g. choose (1% death, 99% totally fine) action instead of (0.1% paralyzed and in pain, 99.9% totally fine) action. Or something like that, your bad outcomes become not death but entrapment in suffering.
So, what's up with my apparent nonexistence in my past? It seems slightly weird that I had some starting point but wouldn't have ending point. Also I'm really confused by, like, subjective time being a thing, if you assume this post is correct description of the universe.
Okay, I received like 6 downvotes on this post and zero critical comments. Usually people here are more willing to debate about consciousness, judging by other posts from these hashtags.
So, can someone articulate what exactly you disliked about this post? Is it too weird or is it not weird enough? Maybe it's sloppy stylistically or epistemically? Maybe you disagree on object level with this exploration of physicalist/functionalist/empiricist position I'm arguing in favor of here? Maybe you like dualism or quantum brain hypothesis? Maybe you think I'm arguing badly in favor of your own position?
Yeah, it kind of looks like all the unhappy people die by 50 and then average goes up. Conditioning on the figure being right in the first place.
[EDIT] looks like approximately 12% - 20% of people are dead by 50. Probably should not be that large of an effect on average? idk. Maybe I'm wrong.
It ignores the is-ought discrepancy by assuming that the way morals seem to have evolved is the "truth" of moral reasoning
No? Not sure how do you got that from my post. Like, my point is that morals are baked in solutions to coordination problems between agents with different wants and power levels. Baked into people's goal systems. Just as "loving your kids" is a desire that was baked in from reproductive fitness pressure. But instead of brains it works on a level of culture. I.e. Adaptation-Executers, not Fitness-Maximizers
I also think it's tactically unsound - the most common human-group reaction to something that looks like a threat and isn't already powerful enough to hurt us is extermination.
Eh. I think it's one of the considerations. Like, it will probably not be that. It's either ban on everything even remotely related or some chaos when different regulatory systems trying to do stuff.
TLDR give pigs guns (preferably by enhancing individual baseline pigs, not by breeding new type of smart powerful pig. Otherwise it will probably just be two different cases. More like gene therapy than producing modified fetuses)
As of lately I hold an opinion that morals are proxy to negotiated cooperation or something, I think it clarifies a lot about the dynamics that produce it. It's like evolutionary selection -> human desire to care about family and see their kids prosper, implicit coordination problems between agents of varied power levels -> morals.
So, like, uplift could be the best way to ensure that animals are treated well. Just give them power to hurt you and benefit you, and they will be included into moral considerations, after some time for it to shake out. Same stuff with hypothetical p-zombies, they are as powerful as humans, so they will be included. Same with EMs.
Also, "super beneficiaries" are then just powerful beings, don't bother to research the depth of experience or strength of preferences. (e.g. gods, who can do whatever and don't abide by their own rules and perceived to be moral, as an example of this dynamics).
Also, pantheon of more human like gods -> less perceived power + perceived possibility to play on disagreements -> lesser moral status. One powerful god -> more perceived power -> stronger moral status. Coincidence? I think not.
Modern morals could be driven by a lot stronger social mobility. People have a lot of power now, and can unexpectedly acquire a lot of power later. so, you should be careful with them and visibly commit to treating them well (e.g. be moral person, with particular appropriate type of morals).
And it's not surprising how (chattel) slaves were denied a claim on being provided with moral considerations (or claim on being a person or whatever), in a strong equilibrium where they are powerless and expected to be powerless later.
tldr give pigs guns
(preferably by enhancing individual baseline pigs, not by breeding new type of smart powerful pig. Otherwise it will probably just be two different cases. More like gene therapy than producing modified fetuses)
Sooo, you need to build some super Shapley values calculator and additionally embed it into our preexisting coordination mechanisms such that people who use it on average do better that people who don't.
MMAvocado, a copy that was convinced it was a talking avocado, and felt consumed by existential horror at this fact. While techniques for invoking mind dysmorphia are now standard, at the time this was a pioneering methodology, and the judges were impressed by the robustness of the delusion despite other knowledge remaining largely intact.
Uh huh, but looks like Cluade actually liked to be mmavocadoed. Still, torment nexus it is
I think I generally got your stance on that problem, and I think you are kind of latching on irrelevant bit and slightly transferring your confusion onto relevant bits. (You could summarize it as "I'm conscious, and other people look similar to me, so they are probably too, and by making the dissimilarity larger in some aspects, you make them less likely to be similar to me in that respect too" maybe?)
Like, the major reasoning step is "if EMs display human behaviors and they work by extremely closely emulating brain, then by cutting off all other causes that could have made meaty humans to display these behaviors, you get strong evidence that meaty humans display these behaviors for the reason of computational function that brain performs".
And it would be very weird if some factors conspired to align and make emulations behave that way for a different reason that causes meaty humans to display them. Like, alternative hypotheses are either extremely fringe (e.g. there is an alien puppet master that puppets all EMs as a joke) or have very weak effects (e.g. while interacting with meaty humans you get some weak telepathy and that is absent while interacting with EMs)
So like, there is no significant loss of probability from meaty humans vs high-res human emulations with identical behavior.
I said it in the start of the post:
It would be VERY weird if this emulation exhibited all these human qualities for other reason than meaty humans exhibit them. Like, very extremely what the fuck surprising. Do you agree?
referring exactly to this transfer of a marker whatever it could be. I'm not pulling it out of nowhere by presenting some justification.
As it stands, I can determine that I am conscious but I do not know how or why I am conscious.
Well, presumably it's a thought in your physical brain "oh, looks like I'm conscious", we can extract it with AI mind reader or something. You are embedded into physics and cells and atoms, dude. Well, probably embedded. You can explore that further by effecting your physical brain and feeling the change from the inside. Just accumulating that intuition of how exactly you are expressed in the arrangement of cells. I think near future will give us that opportunity with fine control over our bodies and good observational tools. (and we can update on that predictable development in advance of it) But you can start now, by, I don't know, drinking coffee.
I would be very surprised if other active fleshy humans weren't conscious, but still not "what the fuck" surprised
But how exactly could you get that information, what evidence could you get. Like, what form of evidence you are envisioning here. I kind of get a feeling that you have that "conscious" as a free floating marker in your epistemology.
Each of the transformation steps described in the post reduces my expectation that the result would be conscious somewhat.
Well, it's like saying if the {human in a car as a single system} is or is not conscious. Firstly it's a weird question, because of course it is. And even if you chain the human to a wheel in such a way they will never disjoin from the car.
What I did is constrained possible actions of the human emulation. Not severely, the human still can talk whatever, just with constant compute budget, time or iterative commutation steps. Kind of like you can constrain actions of a meaty human by putting them in a jail or something. (... or in a time loop / repeated complete memory wipes)
No, I don't think it would be "what the fuck" surprising if an emulation of a human brain was not conscious.
How would you expect to this possibly cash out? Suppose there are human emulations running around doing all things exactly like meaty humans. How exactly do you expect that announcement of a high scientific council go, "We discovered that EMs are not conscious* because .... and that's important because of ...". Is that completely out of model for you? Or like, can you give me (even goofy) scenario out of that possibility
Or do you think high resolution simulations will fail to replicate capabilities of humans, outlook of them? I.e special sauce/quantum fuckery/literal magic?
Even after iterating, my words are often interpreted in ways I failed to foresee.
It's also partially the problem with the recipient of communicated message. Sometimes you both have very different background assumptions/intuitive understandings. Sometimes it's just skill issue and the person you are talking to is bad at parsing and all the work of keeping the discussion on the important things / away from trivial undesirable sidelines is left to you.
Certainly it's useful to know how to pick your battles and see if this discussion/dialogue is worth what you're getting out of it at all.
you're making a token-predicting transformer out of a virtual system with a human emulation as a component.
Should it make a difference? Same iterative computation.
In the system, the words "what's your earliest memory?" appearing on the paper are going to trigger all sorts of interesting (emulated) neural mechanisms that eventually lead to a verbal response, but the token predictor doesn't necessarily need to emulate any of that.
Yes, I talked about optimizations a bit. I think you are missing a point of this example. The point is that if you are trying to conclude from the fact that this system is doing next token prediction then it's definitely not conscious, you are wrong. And my example is an existence proof, kind of.
>It seems you are arguing that anything that presents like it is conscious implies that it is conscious.
No? That's definitely not what I'm arguing.
>But what ultimately matters is what this thing IS, not how it became in that way. If, this thing internalized that conscious type of processing from scratch, without having it natively, then resulting mind isn't worse than the one that evolution engineered with more granularity. Doesn't matter if this human was assembled atom by atom on molecular assembler, it's still a conscious human.
Look, here I'm talking about pathways to acquire that "structure" inside you. Not outlook of it.
I think this kind of framing is kind of confused and slippery, I feel like I'm trying wake up and find a solid formulation of it.
Like, what it does it mean, do it by yourself? Do humans do it by themselves? Who knows, but probably not, children that grow without any humans nearby are not very human.
Humans teach humans to behave as if they are conscious. Just like majority of humans have sense of smell, and they teach humans who don't to act like they can smell things. And some only discover that smell isn't an inferred characteristic when they are adults. This is how probably non conscious human could pass as conscious, if such disorder existed, hm?
But what ultimately matters is what this thing IS, not how it became in that way. If, this thing internalized that conscious type of processing from scratch, without having it natively, then resulting mind isn't worse than the one that evolution engineered with more granularity. Doesn't matter if this human was assembled atom by atom on molecular assembler, it's still a conscious human.
Also, remember that one paper where LLMs can substitute CoT with filling symbols .......? [inset the link here] Not sure what's up with that, but kind of interesting in this context
Good point, Claude, yeah. Quite alien indeed, maybe more parsimonious. This is exactly what I meant by possibility of this analogy being overridden by actually digging into your brain, digging into a human one and developing actually technical gears-level models of both and then comparing them. Until then, who knows, I'm leaning toward healthy dose of uncertainty.
Also, thanks for the comment.
If traders can get access to control panel for actions of the external agent AND they profit from accurately predicting its observations, then wouldn't the best strategy be "create as much chaos as possible that is only predictable to me, its creator". So, traders that value ONLY accurate predictions will get the advantage?
Well maybe llms can "experiment" on their dataset by assuming something about it and then being modified if they encounter counterexample.
I think it vaguely counts as experimenting.