Posts
Comments
Another crackpot physics thing:
My crackpot physics just got about 10% less crackpot. As it transpires, one of the -really weird- things in my physics, which I thought of as a negative dimension, already exists in mathematics - it's a Riemann Sphere. (Thank you, Pato!)
This "really weird" thing is kind of the underlying topology of the universe in my crackpot physics - I analogized the interaction between this topology and mass once to an infinite series of Matryoshka dolls, where every other doll is "inside out and backwards". Don't ask me to explain that; that entire avenue of "attempting to communicate this idea" was a complete and total failure, and it was only after drawing a picture of the topology I had in mind that someone (Pato) observed that I had just drawn a somewhat inaccurate picture of a Riemann Sphere. (I drew it as a disk in which the entire boundary was the same point, 0, with dual infinities coinciding at the origin. I guess, in retrospect, a sphere was a more obvious way of describing that.)
If we consider that the points are not evenly allocated over the surface of the sphere - they're concentrated at the poles (each of which is simultaneously 0 and infinity, the mapping is ambiguous), if we drew a line such that it crosses the same number of points with each revolution, we get - something like a logarithmic spiral. (Well, it's a logarithmic spiral with the "disk" interpretation; it's a spherical spiral whose name I don't know in the spherical interpretation.)
If we consider the bundle of lines connecting the poles, and use this constant-measure-per-revolution spiral to describe their path, I think that's about ... 20% of the way to actually converting the insanity in my head into real mathematics. Each of these lines is "distance" (or, alternatively, "time" - it depends on which of the two charts you employ). The bundle of these spirals provides one dimension of rotation; there's a mathematical way of extracting a second dimension of rotation, to get a three-dimensional space, but I don't understand it at an intuitive level yet.
A particle's perspective is "constantly falling into an infinity"; because of the hyperbolic nature of the space, I think a particle always "thinks" it is at the equator - it never actually gets any closer. Because the lines describe a spiral, the particle is "spinning". Because of the nature of the geometry of the sphere, this spin expresses itself as a spinor, or at least something analogous to one.
Also, apparently, Riemann Spheres are already used in both relativistic vacuum field equations and quantum mechanics. Which, uh, really annoys me, because I'm increasingly certain there is "something" here, and increasingly annoyed that nobody else has apparently just sat down and tried to unify the fields in what, to me, is the most obvious bloody way to unify them; just assume they're all curvature, that the curvature varies like a decaying sine wave (like "sin(ln(x))/x", which exhibits exactly the kind of decay I have in mind). Logarithmic decay of frequency over distance ensures that there is a scalar symmetry, as does a linear decay of amplitude over distance.
Yes, I'm aware of the intuitive geometry involved in an inverse-square law; I swear that the linear decay makes geometric sense too, given the topology in my head. Rotation of a logarithmic spiral gives rise to a linear rescaling relative to the arclength of that rotation. Yes, I'm aware that the inverse-square law also has lots of evidence - but it also has lots of evidence against it, which we've attempted to patch by assuming unobserved mass that precisely accounts for the observed anomalies. I posit that the sinusoidal wave in question has ranges wherein the amplitude is decaying approximately linearly, which creates the apparent inverse-square behavior for certain ranges of distances - and because these regions of space are where matter tends to accumulate, having the most stable configurations, they're disproportionately where all of our observations are made. It's kind of literally the edge cases, where the inverse-square relationship begins to break down (whether it really does, or apparently does), and the configurations become less stable, that we begin to observe deviations.
I'm still working on mapping my "sin(ln(x))/x" equation (this is not the correct equation, I don't think, it's just an equation that kind of looks right for what's in my head, and it gave me hints about where to start looking) to this structure; there are a few options, but none stand out yet as obviously correct. The spherical logarithmic spiral is a likely candidate, but figuring out the definition of the spiral that maintains constant "measure" with each rotation requires some additional understanding on my part.
The issue arises specifically in the situation of recursive self-improvement: You can't prove self-consistency in mathematical frameworks of "sufficient complexity" (that is, containing the rules of arithmetic in a provable manner).
What this cashes out to is that, considering AI as a mathematical framework, and the next generation of AI (designed by the first) as a secondary mathematical framework - you can't actually prove that there are no contradictions in an umbrella mathematical framework that comprises both of them, if they are of "sufficient complexity". Which means an AI cannot -prove- that a successor AI has not experienced value drift - that is, that the combined mathematical framework does not contain contradictions - if they are of sufficient complexity.
To illustrate the issue, suppose the existence of a powerful creator AI, designing its successor; the successor, presumably, is more powerful than the creator AI in some fashion, and so there are areas of the combinatorially large space that the successor AI can explore (in a reasonable timeframe), but that the creator AI cannot. If the creator can prove there are no contradictions in the combined mathematical framework - then, supposing its values are embedded in that framework in a provable manner, it can be assured that the successor has not experienced value drift.
Mind, I don't particularly think the above scenario is terribly likely; I have strong doubts about basically everything in there, in particular the idea of provable values. I created the post for the five or six people who might still be interested in ideas I haven't seen kicked around on Less Wrong for over a decade.
Alternatively - we communicate about the things that pose the most danger to us, in a manner intended to minimize that danger.
In a typical Level-4 society, people don't have a lot to fear from lions and they aren't in imminent danger of starvation. The bottom half of Maslow's hierarchy is pretty stable.
It's the social stuff where our needs run the risk of being unfulfilled; it is the social stuff that poses the most danger. So of course most of the communication that takes place is about social stuff, in manners intended to reinforce our own social status. This isn't a simulacra of reality - it is reality, and people suffer real harms for being insufficient to the task.
I think a substantial part of the issue here is the asymmetry created when one party is public, and one party is not.
Suppose a user is posting under their real name, John Doe, and another user is posted under a pseudonym, Azure_Pearls_172. An accusation by Azure against John can have real-world implications; an accusation by John against Azure is limited by the reach of the pseudonym. Azure can change their pseudonym, and leave the accusations behind; John cannot.
Doxxing can make a situation more symmetrical in this case. Whether or not it is merited is a complicated topic, particularly as the norms around doxxing exist for a reason.
Suppose a user assaults other users, and switches pseudonyms whenever identified to keep finding new targets - I doubt anybody would argue that doxxing a predatory member of this sort is a bad thing, in and of itself. Contrariwise, suppose a user gets annoyed with another user, and then doxxes them and accuses them in bad faith of assault. We don't want that.
I think mixed-anonymity is basically a terrible way to run things, owing to the asymmetries involved, and in general communities should have norms that either reflect no anonymity (everybody uses their real names), or total anonymity (nobody uses their real names, and also nobody ever meets anybody else in person). If you're mixing the cases, you're creating the potential for abuses.
If you disagree that anonymous users should never meet in person, well - if you're willing to meet in person, why are you choosing to be anonymous? Is anonymity a conscious and deliberate decision, such that doxxing would actually be a violation (in which case, why are you doxxing yourself?), or is it just a default option? And if you're meeting another anonymous user - well, what is their reason for choosing to be anonymous?
Mind, I've met other pseudonymous users of various communities, so I can't 100% claim to be consistent with this. But I only do so when my choice of anonymity is more "default" than "deliberate choice" - there are some pseudonyms I use which I certainly wouldn't meet somebody under the auspices of, because they are deliberate choices, chosen to minimize exposure.
(Granted, I haven't used them in a while, and at this point most of the opinions I shared under them are basically widely accepted today, and those that aren't are at least socially acceptable - so, eh, it would probably be fine at this point.)
I am, unapologetically, a genius. (A lot of people here are.)
My experience of what it is like being a genius: I look at a problem and I know an answer. That's pretty much it. I'm not any faster at thinking than anybody else; I'd say I'm actually a somewhat slower thinker, but make up for it by having "larger" thoughts; most people seem to have fast multi-core processors, and I'm running a slightly slow graphics card. Depending on what you need done, I'm either many orders of magnitude better at it - or completely hopeless. It mostly depends on whether or not I've figured out how to adapt the problem to my way of thinking.
Also, sometimes my "I know an answer" is wrong - this means that I still have to go through the "manual" effort of thinking, to verify the answer, and I'm using the slow graphics card to run a mostly single-thread process. Sometimes the answer is too hard to verify either way! (Hey, look at my username; I've been pursuing themes on a crackpot physics for twenty five years, and I'm not particularly any closer to being able to determine whether or not it is true or false!)
In practice, in the real world, what this translates to is: I'm often no faster at providing a "good" answer than a merely above-average person, because, while I know -an- answer, it will take me just as long to verify whether or not it is a good answer as it takes a merely above-average person to go through the manual effort of finding an answer and then verifying it!
Also, my answers are often ... strange. Not wrong, and I can find answers to problems other people find intractable, or find a way to do something in way less time than somebody else - but on rare occasion, I can't find the answer that an average person can spot immediately, and much more frequently, I find an answer that takes way -more- time than the obvious-to-other-people solution.
What I conclude is that what makes me a "genius" is context - I am in fact likely merely somewhat above-average, but that I find the difficulty of problems -different- than other people. Imagine, for a moment, that everybody is given a map of the world, which maps, let's say, 5% of the territory. But 99% of the 5% is in common; ask a hundred people, and 99 of them will know where Canada is. My map is only somewhat above average in size, but it covers an entirely different geography - I couldn't tell you where Canada is, but I know where Tucson is, something that is on less than .01% of the maps out there.
You need to get to Canada, you can ask just about anybody, and they can tell you how to get there. So, even though I don't have Canada on my map, this mostly doesn't present me any problems, except when I'm trying to get to Alaska and somebody tells me to just drive through Canada.
But if you need to go to Tucson, it's hard to find somebody who knows where it is. But I can immediately tell you. Nobody ever asks me how to get to Canada - why would they? - but everybody asks me how to get to Tucson, so I look like I know a lot. And IQ tests really reward knowing how to get to Tucson, and don't bother asking about Canada at all, so - I'm a genius. And because everyone knows where Canada is, I benefit, from an intellectual perspective, as much from having ordinary people around me, as they benefit from having a "genius" around them.
But I'm in the same boat as anybody else when I need to get to Jupiter; nobody has a map that says how to get there.
Take a step back and try rereading what I wrote in a charitable light, because it appears you have completely misconstrued what I was saying.
A major part of the "cooperation" involved here is in being able to cooperate with yourself. In an environment with a well-mixed group of bots each employing differing strategies, and some kind of reproductive rule (if you have 100 utility, say, spawn a copy of yourself), Cooperate-bots are unlikely to be terribly prolific; they lose out against many other bots.
In such an environment, a strategem of defecting against bots that defect against cooperate-bot is a -cheap- mechanism of coordination; you can coordinate with other "Selfish Altruist" bots, and cooperate with them, but you don't take a whole lot of hits from failing to edit: defect against cooperate-bot. Additionally, you're unlikely to run up against very many bots that cooperate with cooperate-bot, but defect against you. As a coordination strategy, it is therefore inexpensive.
And if "computation time" is considered as an expense against utility, which I think reasonably should be the case, you're doing a relatively good job minimizing this; you have to perform exactly one prediction of what another bot will do. I did mention this was a factor.
Evolution gave us "empathy for the other person", and evolution is a reasonable proxy for a perfectly selfish utility machine, which is probably good evidence that this might be an optimal solution to the game theory problem. (Note: Not -the- optimal solution, but -an- optimal solution, in an ecosystem of optimal solutions.)
Note that it is possible to deceive others by systematically adjusting predictions upward or downward to reflect how desirable it is that other people believe those predictions, in a way which preserves your score.
This is true even if you bucket your scores; say you're evaluating somebody's predictive scores. You see that when they assign a 60% probability to an event, that event occurs 60% of the time. This doesn't mean that any -specific- prediction they make of 60% probability will occur 60% of the time, however! They can balance out their predictions by adjusting two different predictions, overestimating the odds of one, and underestimating the odds of another, to give the appearance of perfectly calibrated predictions.
The Brier Score is useful for evaluating how good a forecaster is, but this is not the same as evaluating how good any given forecast a forecaster makes is. If the Oracle really hates Odysseus, the Oracle could give a forecast that, if believed, results in a worse outcome for Odysseus, and balance this out by giving a forecast to another individual that results in apparent perfect calibration.
How does one correctly handle multi-agent dilemmas, in which you know the other agents follow the same decision theory? My implementation of "UDT" defects in a prisoner's dilemma against an agent that it knows is following the same decision procedure. More precisely: Alice and Bob follow the same decision procedure, and they both know it. Alice will choose between cooperate/defect, then Bob will choose between cooperate/defect without knowing what Alice picked, then the utility will be delivered. My "UDT" decision procedure reasons as follows for Alice: "if I had pre-commited to cooperate, then Bob would know that, so he would defect, therefore I defect". Is there a known way out of this, besides special casing symmetric dilemmas, which is brittle?
My solution, which assumes computation is expensive, is to reason about other agents based on their behavior towards a simplified-model third agent; the simplest possible version of this is "Defect against bots who defect against cooperate-bot, otherwise cooperate" (and this seems relatively close to how humans operate - we don't like people who defect against the innocent).
The point there is that there is no contradiction because the informational content is different. "Which is the baseline" is up to the person writing the problem to answer. You've asserted that the baseline is A vs B; then you've added information that A is actually A1 and A2.
The issue here is entirely semantic ambiguity.
Observe what happens when we remove the semantic ambiguity:
You've been observing a looping computer program for a while, and have determined that it shows three videos. The first video portrays a coin showing tails. The second video portrays two coins; the left coin shows heads, the right coin shows tails. The third video also portrays two coins; the left coin shows heads, the right coin shows heads.
You haven't been paying attention to the frequency, but now, having determined there are three videos you can see, you want to figure out how frequently each video shows up. What are your prior odds for each video?
33/33/33 seems reasonable. I've specified that you're watching videos; the event is which video you are watching, not the events that unfold within the video.
Now, consider an alternative framing: You are watching somebody as they repeat a series of events. You have determined the events unfold in three distinct ways; all three begin the same way, with a coin being flipped. If the coin shows heads, it is flipped again. If the coin shows tails, it is not. What are your prior odds for each sequence of events?
25/25/50 seems reasonable.
Now, consider yet another framing: You are shown something on a looping computer screen. You have determined the visuals unfold in three distinct ways; all three begin the same way, with a coin being flipped. If the coin shows heads, it is flipped again. If the coin shows tails, it is not. What are your prior odds here?
Both 25/25/50 and 33/33/33 are reasonable. Why? Because it is unclear whether or not you are watching a simulation of coin flips, or something like prerecorded videos; it is unclear whether or not you should treat the events within what you are watching as events, or whether you should treat the visuals themselves you are watching as the event.
Because it is unclear, I'd lean towards treating the visuals you are watching as the event - that is, assume independence. However, it would be perfectly fair to treat the coin tosses as events also. Or you could split the difference. Prior probabilities are just your best guess given the information you have available - and given that I don't have access to all the information you have available, both options are fair.
Now, the semantic ambiguity you have introduced, in the context of this, is like this:
You're told you are going to watch a computer program run, and what you see will begin with a coin being flipped, showing heads or tails. What are your probabilities that it will show heads or tails?
Okay, 50/50. Now, if you see the coin shows heads, you will see that it is flipped again; we now have three possibilities, HT, HH, and TT. What are your probabilities for each event?
Notice: You didn't specify enough to know what the relevant events we're assigning probabilities to even are! We're in the third scenario; we don't know if it's a video, in which case the relevant event is "Which video we are watching", or if it is a simulation, in which case the relevant event is "The outcome of each coin toss." Either answer works, or you can split the difference, because at this point a large part of the probability-space is devoted, not to the events unfolding, but towards the ambiguity in what events we're even evaluating.
If you have two options, A and B, 50% odds is maximal ignorance; you aren't saying they have equivalent odds of being true, you're saying you have no information by which to make an inference which is true.
If you then say we can split A into A1 and A2, you have added information to the problem. Like the Monty Hall problem, information can change the odds in unexpected ways!
There's no contradiction here - you have more information than when you originally assigned odds of 50/50. And the information you have added should, in real situations, inform how to distribute the odds. If A1 and A2 are sufficiently distinct (independent), it is possible that a 33/33/33 split is appropriate; if they aren't, it is possible that a 25/25/50 split is appropriate. In order to make a judgment, we'd have to know more about what exactly A1 and A2 are, and why they can be considered a "split" of A.
Consider, for example, the case of a coin being flipped - we don't know if the coin is fair or not. Let us say A is that the coin comes up "heads" and B is that the coin comes up "tails". The split, then, could reflect a second flip, after the first flip is decided, if and only if it is heads; A1 might be "heads-heads", A2 might be "heads-tails". Then a 25/25/50 split makes sense; A1 and A2 are not independent.
If, on the other hand, we have discovered that it isn't a coin at all, but a three-sided die, two faces of which have the same symbol, and one face of which has another symbol; we label the faces with the similar symbol A1 and A2, and the face with the same edit: other symbol B. We still don't know whether or not the die is fair - maybe it is weighed - but the position of maximal ignorance is 33/33/33, because even if it -is- weighted, we don't know which face it is weighted in favor of; A1 and A2 are independent.
So - what are A1 and A2, and how independent are they? We have equations that can work this out with sample data, and your prior probability should reflect your expectation of their independence. If you insist on maximal ignorance about independence - then you can assume independence. Most things are independent; it is only the way the problem is constructed that leads us to confusion here, because it seems to suggest that they are not independent (consider that we can simply rename the set of conclusions to "A, B, and C" - all the names you have utilized are merely labels, after all, and in effect, what you have actually done is to introduce C, with an implication that A and C should maybe be considered partially dependent variables). If you insist on maximal ignorance about that, as well, then you can, I suppose, assume 50% independence, which would be something like splitting the difference between the die and the coin. And there's an argument to be made there, in that you have, in fact, implied that they should maybe be considered partially dependent variables - but this comes down to trying to interpret what you have said, rather than trying to understand the nature of probability itself.
I tried potassium supplementation. The very first thing I noticed is that a significant portion of hunger was immediately converted into thirst; to be specific, where normally at time X I would be hungry, instead at time X I was thirsty instead. There was an immediate and overall reduction of calories in.
This suggests to me that I had a slight potassium deficiency which my body was compensating for by increasing the amount of food I was consuming.
Cursory research suggests potassium content in fresh foods has declined ~20% over the past century - which is not particularly surprising, if you think about modern farming methodologies. Additionally, it appears that lithium consumption (tying into SMTM's hypothesis) may deplete the body's potassium reserves (which could conceivably be the mechanism by which lithium causes weight gain, in slight contradiction to SMTM's hypothesis). Additionally additionally, and most importantly - low potassium content appears to be correlated with something like a 20% increase in caloric consumption, second only to protein deficiency in terms of increasing "natural" caloric intake. (All of this is cursory internet research, and should not be taken too seriously, but it is all pointing in a particular direction very suggestively).
I think the "sodium intake" is a red herring, sort of (if your body needs potassium, but can't distinguish between potassium and sodium in food intake, it may result in people salting their food more - another change that occurred after I began supplementing potassium is that I didn't need as much salt to make food taste like anything - so sodium intake may be a symptom of potassium deficiency).
Supposing we're all slightly potassium deficient at a healthy level of food consumption, and we cover the potassium intake gap by simply eating more food (thus getting the necessary levels of potassium), then potassium supplementation could quite reasonably decrease caloric consumption without any effort. And if we're substituting table salt for potassium, because our bodies struggle to tell the difference at intake, we might expect other health issues to arise from that.
If this was the case, then we should expect that all diets that work for most people who try them, while improving overall health, and without requiring exceptional willpower, should be high in potassium. Pondering this, I checked, and, indeed, meats are high in potassium. Ketogenic diets could, then, operate on the same principles as the potato diet.
(Indeed, I have done a combination ketogenic+rabbit starvation diet a few times, at 500 calories a day, with basically no carbohydrates or fats. A key component of this diet is very lean meat, and it required no willpower; I can run on this diet for months, and the only issue I run into is the monotony, which personally isn't a big deal. It is actually personally easier to be on that diet that to try to maintain a "normal" diet while restraining my food consumption to a level that prevents weight gain - but my normal diet is relatively light on anything with high levels of potassium, including meat.)
I submit that potassium is indeed very important, and is, plausibly, given declining potassium food content, the answer to the question of modern obesity.
Instead of further elaborations on my crackpot nonsense, something short:
I expect that there is some distance from a magnetic source between 10^5 meters and 10^7 meters at which there will be magnetic anomalies; in particular, there will be a phenomenon by which the apparent field strength drops much faster than expected and passes through zero into the negative (reversed polarity).
I specifically expect this to be somewhere in the vicinity of 10^6 meters, although the specific distance will vary with the mass of the object.
There should be a second magnetic anomaly somewhere in the vicinity of 10^12 m (So between 10^11 and 10^13), although I suspect at that distance it will be too faint to detect.
More easily detected, because there is a repulsive field at work at these distances - mass should be scarce at this distance from the dominant "local" masses, a scarcity that should continue up to about 10^18 m (between 10^17 and 10^19, although errors really begin to compound here); at 10^18 m, I expect an unusually dense distribution of matter; this value in the vicinity of 10^18 m should be the most common distance between objects in the galaxy.
It should be possible to find large masses (say, black holes) orbiting each other at, accounting for relativistic changes in distance, 10^18m, which we might otherwise expect to fall into one another - that is, there should be unexplainable orbital mechanics between large masses that are this distance apart.
I expect that there is some radius between 10^22 meters and 10^26 meters (vicinity of 10^24) which marks the largest possible size of a galaxy, and some radius between 10^28 and 10^32 (vicinity of 10^30) which marks the most common distance between galaxies.
Galaxies which are between the vicinity of 10^24 and the vicinity of 10^30 meters from one another should be moving apart, on average; galaxies which are greater than the vicinity of 10^30 meters from one another should be falling into one another on average.
Galaxies which are approximately 10^30 meters apart should be orbiting one another - neither moving towards nor away.
Yes, but then it sounds like those who have no such altruistic desire are equally justified as those who do. An alternative view of obligation, one which works very well with utilitarianism, is to reject personal identity as a psychological illusion. In that case there is no special difference between "my" suffering and "your" suffering, and my desire to minimize one of these rationally requires me to minimize the other. Many pantheists take such a view of ethics, and I believe its quasi-official name is "open individualism".
I think this requires an assumption that there exists on obligation to end our own suffering; I find that a curious notion, because it presupposes that there is only one valid way to exist.
You would prefer that we had the ethical intuitions and views of the first human beings, or perhaps of their hominid ancestors?
What bearing do their ethical intuitions have on me?
(What bearing do my ethical intuitions have on future hominids?)
Where you see neutrality, he would see obligation.
In what sense is it an obligation? By what mechanism am I obligated? Do I get punished for not living up to it?
You use that word, but the only meaningful source of that obligation, as I see it, is the desire to be a good person. Good, not neutral.
I disagree, and I think that you are more of a relativist than you are letting on. Ethics should be able to teach us things that we didn't already know, perhaps even things that we didn't want to acknowledge.
This is a point of divergence, and I find that what ethical systems "teach us" is an area full of skulls. (However, I am, in fact, far LESS of a relativist than I am letting on; I am in fact a variant of absolutist.)
As for someone who murders fewer people than he saves, such a person would be superior to me (who saves nobody and kills nobody) and inferior to someone who saves many and kills nobody.
Question: Would a version of yourself who did not believe in your ethics, and saw "neutral" as a perfectly valid thing to be, be happier than the version of yourself that exists?
Utility, as measured, is necessarily relative. By this I don't mean that it is theoretically impossible to have an objective measure of utility, only that it is practically impossible; in reality / in practice, we measure utility relative to a baseline. When calculating the utility of doing something nice for somebody, it is impractical to calculate their current utility, which would include the totality of their entire experience as summed in their current experience.
Rule utilitarianism operates in the same fashion much more straightforwardly, considering utility from an act as the average deviation from a relative position, which I think it is safe to call "normal".
Once we observe that utility is measured from a relative baseline, a normal, then it is meaningful to talk about acts which are of negative utility; the meaningful comparison is not to an absolute number, but to a relative measure, which any given act can fall beneath.
Insofar as we treat utilitarianism as having an absolute number which cannot be measured by which is the important criteria, "badness" itself is meaningless; badness compared to what? Now, you might say that the correct point of measurement is to the highest-positive-utility act; that utilitarianism says that all acts are measured relative to this. But this is not a position I believe is universally supported; certainly Karl Popper argued against this view of utilitarianism, proposing the framework of negative utilitarianism (I think he invented it?) as a solution to problems he saw with this worldview. Negative utilitarianism measures two distinct values, prioritizing one over the other.
Assuming we do stick to a relative measure, once we observe that acts, or the failure to act, can be of negative, neutral, or positive value, with respect to that relative "normal", it is meaningful to talk about negative-utility acts as distinct from an inaction relative to a positive-utility act. We can then call negative-utility acts "evil", neutral acts "bad" (I think a better term here is "suboptimal"), and good acts "good", and in doing so, recover an important part of the way most human beings experience ethics, which includes a component we call "evil", as distinct from "good", which itself is distinct from "neutral".
Or, to put all this another way - our moral intuitions do not in fact say that goodness and evil are fungible, and in particular, that somebody who murders somebody for every seven lives he saves is anything like a good person, and insofar as utilitarianism doesn't acknowledge this, it fails to actually be a good representation of human ethics, which is to say, the ethics we actually care about. It should add up to normality, after all.
I suspect there might be a qualia differential.
What is your internal experience of morality?
The tax should, in fact, cause some landlords / landowners to just abandon their land. This is a critical piece of Georgism; the idea that land is being underutilized, in particular as an investment which is expected to pay off in terms of higher land values / rents later, but also in terms of things like parking lots, where the current value of the use of the land may exceed the current taxes (which include only a portion of the value of the land and the improvements combined) while being lower than the Georgist taxes (which include the entire value of the land and none of the value of the improvements). In particular Georgist taxes should eliminate current incentive structures which "reward" things like keeping an apartment building empty of tenants.
The point of indifference is not the point of indifference in the current economic system, it is the point of indifference in the Georgist economic system.
Related: https://www.lesswrong.com/posts/57sq9qA3wurjres4K/ruling-out-everything-else
I do not think the linked post goes anywhere near far enough. In particular, it imagines that people share a common concept-space. The totality to which thought is arbitrary is, basically, complete.
I'm a crackpot.
Self-identifiably as so. Part of the reason I self-identify as a crackpot is to help create a kind of mental balance, a pushback against the internal pressure to dismiss people who don't accept my ideas: Hey, self, most people who have strong beliefs similar to or about the thing you have strong beliefs about are wrong, and the impulse to rage against the institution and people in it for failing to grasp the obvious and simple ideas you are trying to show them is exactly the wrong impulse.
The "embitterment" impulse can be quite strong; when you have an idea which, from your perspective, is self-evident if you spend any amount of time considering it, the failure of other people to adopt that idea can look like a failure to even consider anything you have said. Or it can look like everybody else is just unimaginative or unintelligent or unwilling to consider new ideas; oh, they're just putting in their 9-5, they don't actually care anymore.
Framing myself as a crackpot helps anticipate and understand the reactions I get. Additionally, framing myself as a crackpot serves as a useful signal to somebody reading; first, that if they have no patience for these kinds of ideas, that they should move on. And second, that if they do have the patience for these kinds of ideas, that I have self-awareness of exactly what kind of idea it is, and am unlikely to go off on insane rants against them for interpreting me as a crackpot, and also that having warned them in advance I am also aware that this may be an imposition and I am not taking their time for granted. (Also higher level meta signaling stuff that is harder to explain.)
Are you a crackpot? I have no idea. However, when people start talking about existential safety, my personal inclination is to tune them out, because they do pattern match to "Apocalyptic Thinkers". The AI apocalypse basically reads to me like so many other apocalypse predictions.
Mind, you could be right; certainly I think I'm right, and I'm not going to be casting any stones about spreading ideas that you think are correct and important.
However, my personal recommendation is to adopt my own policy: Self awareness.
Why are you using what I presume is your real name here?
I'm not actually interested in whether or not it is your real name, mind; mostly I'd like to direct your attention to the fact that the choice of username was in fact a choice. That choice imparts information. By choosing the username that you did, you are, deliberately or not, engaging in a kind of signaling.
In particular, from a particular frame of reference, you are engaging in a particular kind of costly signaling, which may serve to elevate your relative local status, by tying any reputational hits you may suffer as a result of mis-steps here to your real identity. You are saying "This is me, I am not hiding behind false identities." The overall effect of this is a costly signal which serves to elevate your status with the tribe here.
If it isn't your real name, why are you using a false identity that looks like a real identity?
Hang up, though. Let us say instead that you, instead, see false identities as a form of dishonesty; this isn't signaling, this is sticking to principles that are important to you.
Well, if that is the case, another question: Would you use this identity to say something that does have strong reputational costs associated to your real identity? Let us say that you would, you just don't have any such things to say.
Well, it is convenient for you, some might observe, that you are willing to stand up for principles that don't cost you anything. (Hence some part of why signaling tends to be costly; it avoids this problem.)
I will observe there is an important political dispute about anonymity on the internet, which has major communal aspects. The fewer users who insist on privacy, the more that commercial websites can exclude those who do. Oh, you don't want us tracking you? You don't get to use our website anymore. Observe the trend in websites, such as Twitter, of becoming increasingly user-unfriendly to those who are not logged in, or of excluding them altogether.
"Everything is political" is an observation that this phenomenon is, basically, universal.
Once we observe that there -is- a political implication in your choice of username, we must ask whether you -ought- to do anything about it; a lot of people like to skip this question, but it is an important question. Do you "owe" it to the people who prefer anonymity, to yourself remain anonymous? The pro-anonymity side would be really well served if everybody was forced to be anonymous; they are certainly better served if the choice is explicitly served (hence the EU rules on website cookies) instead of anonymity being opt-in instead of opt-out.
However, there are also people who don't want to be anonymous, or who don't want to interact with anonymous people; certainly there's the potential for some power imbalances there.
We've happened upon some kind of uneasy mostly-truce, where anonymity is contextual, and violating another person's anonymity is seen as a violation of the cultural norms of the internet. This truce is eroding; as fewer and fewer people choose to be anonymous, a higher and higher proportion of anonymous actions are those which would impose costs on the speaker if the speaker chose not to be anonymous, which makes anonymity an increasingly sinister-looking choice.
Imagine being a writer in a group of blogs with a common user system, moderating comments. To begin with, all the blogs allow anonymous comments. However, after one too many abusive comments, a blog bans anonymous commenters; some percentage of previously-anonymous commenters value commenting there enough to create accounts, reducing the number of "legitimate" anonymous comments in the ecosystem as a whole. This makes anonymous comments look worse, prompting the next blog to turn them off, then the next.
Look, the pro-anonymity people say, you're making a choice to oppose an anonymous internet; you're against us.
Well, there's definitely an "is" there. What's missing is the "ought", the idea that the political implications of an act create individual responsibility. There's a very deep topic here, relating to the way certain preferences are also natural attractor states, whose satisfaction rules out opposing preferences, but this comment is already long enough.
I get the impression, reading this and the way you and commenters classify people, that the magnitude of days is to some extent just equivalent to an evaluation of somebody's intellectual ability, and the internal complexity of their thoughts.
So if I said your article "Ruling Out Everything Else" is the 10-day version of a 10000-day idea, you might agree, or you might disagree, but I must observe that if you agree, it will be taken as a kind of intellectual humility, yes? And as we examine the notion of humility in this context, I think it should be noticed the implication of superiority; a 10000 day idea is a superior idea to a 10 day idea. (Otherwise, what would there to be humble about?) And if you felt like it was a more profound article than that, you'd find it somewhat offensive, I think.
...
Except that it is equally plausible that none of that is actually true, and that you're pointing at something else, and this interpretation is just one that wasn't ruled out. If another equally plausible interpretation is correct: A 10-day monk is wrong more often than a 1-day monk, yes? A 100-day monk is wrong more often than a 10-day monk? The number of days matter; when other commenters point out that you need to bounce an idea off reality to avoid being wrong, are they criticizing your point, or glimpsing a piece of it? Is it accurate to say that a significant piece of the idea represented here is that the number of days is in some sense equivalent to a willingness to be wrong about more?
It isn't the thing that the KL divergence is measuring, it is an analogy for it. The KL divergence is measuring the amount of informational entropy; strictly speaking, zipping a file has no effect in those terms.
However, we can take those examples more or less intact and place them in informational-entropy terms; the third gets a little weird in the doing, however.
So, having an intuition for what the ZIP file does, the equivalent "examples":
Example 1: KLE(Reference optimizer output stage, ineffective optimizer output) is 0; KLE(Reference final stage, ineffective optimizer final stage) is also 0. Not appropriate as a way of thinking about this, but helpful to frame the next two examples.
Example 2: KLE(Reference optimizer output stage, antieffective optimizer output) is -N; KLE(Reference final stage, antieffective optimizer final stage) is 0. Supposing an antieffective optimizer and an optimizer can both exist such that a future optimizer optimizes away the inefficiency introduced by the antieffective optimizer, we may observe a violation of the conclusion that, for N steps, adding an optimizer cannot result in a situation such that the KL distance of N+1 must be less than or equal to the KL distance of N, relative to their reference cases.
Example 3: The central conceit to this example is harder to translate; the basic idea is that one optimization can make a future optimization more efficient. Critically for this, the future optimization can vary in effectiveness depending on the input parameters. Suppose, for a moment, a state-reduction algorithm on a set of states n, using something like the opening example:
For n < 64, each state mapping n maps to the state {ceiling (n / 4)} [2 bits of optimization]
For n > 64, each state mapping n maps to the state {ceiling (n / 2)} [1 bit of optimization]
Then, for a probability distribution such that the number of states is greater than 64 but less than 128, a precursor optimization which halves the number of states (1 bit of optimization) will create an additional bit of optimization at this second optimizer.
Or, for a very contrived example, the first "optimizer" does minimal optimization, and mostly just encodes a control sequence into the output which enables the second optimizer to actually optimize.
There's something here about process-entropy versus data-entropy which I think merits examination, however.
[Also, an observation: Arithmetic is an optimizer. Consider the number of input states and the number of output states of addition.]
Stepping into a real-world example, consider a text file, and three cases, illustrating different things:
First case: Entirely ineffective ZIP compression, (some processes), effective ZIP compression. If we treat the ineffective ZIP compression as "the optimizer", then it is clear that some compression happens later in the sequence of processes; the number of bits of optimization increased. However, the existence or non-existence of the first ineffective ZIP compression has no effect on the number of bits of optimization, so maybe this isn't quite appropriate as a way of thinking about this.
Second case: Anti-effective ZIP compression, (some processes), effective ZIP compression. The anti-effective ZIP compression, instead of compressing the file, maybe just copies the file twenty times. Then the effective ZIP compression takes that, and possibly compresses it nearly as effectively as the first case. The existence or non-existence of the anti-effective ZIP compression does matter in this case - in particular, if we compare the state of its output compared to the final state, the anti-effective ZIP compression creates a wider optimization discrepancy; it would appear to "optimize at a distance."
Third case: Garbage data cleanup, (some processes), effective ZIP compression. Here we enter an interesting case, because, if we treat the garbage data cleanup as our optimizer, it both optimizes the immediate output, and, in the event that the garbage is significantly higher entropy than the rest of the data, might make the effective ZIP compression more effective.
I think the third case really matters here, but in general, once you introduce a second optimizer, you can create emergent optimizations which don't strictly belong to either optimizer. Additionally, once we submit the existence of emergent optimizations, it may entirely be possible for an optimizer to optimize "at a distance", even without a specific second optimization taking place. Consider any given optimization as a sequence of operations each of which is necessary; if we treat the first operation as "the optimizer", then the optimization as a whole happens "at a distance", given that without the first operation, the optimization fails, and it is only optimized at the final step in the process (given that each step is necessary).
Is this assuming all optimization happens at "the optimizer" (and thus implicitly assuming that no optimization can take place in any intermediate step?)
I notice that my ethics, my morality, my beliefs, differ in many ways from those of the past; I expect these things to differ in many ways from my own, in the future. I notice the relationship between these two concepts is reciprocal.
My grandfather talked to me, several times, about how he knew I had my own life, and that I wouldn't always want to spend a lot of time with him; he was explicitly giving me permission, I think, to do something that he himself regretted in his youth, but understood better with age. He was telling me to live unfettered by that particular obligation. I didn't see him much after I started working, and then he passed away.
If I have grandchildren some day, I know what I will tell them.
I consider the alternative grandfather who I didn't have, who was jealous of my time and attention, and how it would have changed things.
What do we owe the future?
I think maybe we owe the future the freedom to choose for themselves, unfettered by the chains of our expectations. The freedom to choose means nothing if it does not include the freedom to make what we think are the wrong choices. The conservationist wanted to restrain the choices of the future, to limit them. And while I think the cuteness-maximizers are wrong, we owe them the right to be wrong, just as we act in so many ways our ancestors would have regarded as wrong.
The idea of a god-AI enshrining some particular group of people's values forever doesn't look very appealing to me; indeed, it looks like an abdication of our fundamental responsibility to the future.
What do I owe the past, therefore?
Gratitude, that I have more choices than they did.
What does it look like, when the optimization power is turned up to 11 on something like the air conditioner problem?
I think it looks exactly like it does now; with a lot of people getting very upset that local optimization often looks un-optimized from the global perspective.
If I needed an air-conditioner for working in my attic space, which is well-insulated from my living space and much, much hotter than either my living space or the outside air in the summer, the single-vent model would be more efficient. Indeed, it is effectively combining the modern technology of an air conditioner, with the much older technology of what is sometimes called a "house fan" - a literal giant fan that pulled air from the outside, inside, which obviously only works when the interior is hotter than the exterior, conditions under which a single-vent AC should actually outperform the double-hose model.
I know somebody who uses one of the one-hose models. It is run in an otherwise entirely un-air-conditioned space, which is hotter than the outside air when the AC is not in action. Like my attic space, it is more efficient than the two-hose model would be, because the infiltration effect -cools the space-, owing to the fact that interior spaces here are generally hotter than exterior spaces.
Now, I have a room AC unit; it's a two-hose model, because it is air conditioning a single room in an air-conditioned space. However, I have been tempted to install an AC in a different room once before - a room which, owing to a combination of poor insulation and airflow, was considerably warmer than the rest of the house. From the perspective of cooling -that room-, the infiltration effect would be a net positive, as the infiltrating air is cooler than the "natural tendency" of the room; from the perspective of cooling -the entire house-, it would be a net negative, relative to a double-hose model. Does this mean a single-hose model is always worse for this application? Well - that depends on the relative efficiency of the small AC unit compared to the central air conditioning working in the rest of the house. Assuming the central AC is more efficient at cooling air than the room AC, which seems like a reasonable bet, then infiltration could, as with the case for environments which are hotter than the exterior, actually result in a net increase in efficiency under some operating conditions.
Now, just as your post is kind of an analogy about optimization failures, this response is kind of an analogy about optimization failures - namely, the kind of optimization failure where you optimize for a single situation, and end up making things worse.
And namely, I trust local optimizations more than I trust global optimizations, which means in an important sense that I think alignment, as a form of global optimization, may be significantly worse in important ways than "non-aligned" intelligence. Fortunately for this perspective, I expect "aligned" intelligence is necessarily worse at intelligence than non-aligned intelligence, and indeed general-purpose intelligence to always be worse than specific-purpose intelligence, and by such a factor that I think we gravely miscalculate how hard general intelligence is, because we base our predictions on how successful we are at creating specific-purpose intelligence.
There's a kind of general relationship here between "general purpose" and "global", and "specific purpose" and "local", which I believe should hold up under scrutiny.
ETA:
One thing to pay particular attention to, in the efficiency assumptions, is the following question: Is the device expected to be "always on"? There's a huge difference between a device whose primary purpose is effectively to "Cool off a relatively hot space, and then maintain that temperature for some duration of time" and a device whose primary function is effectively to "Maintain an existing temperature differential indefinitely (with a negligible consideration for the ramp-up period)". Of note, basically everybody I know who owns such a device is not keeping it on indefinitely, and the ramp-up period likely dominates the energy consumption of the device.
You have a simplification in your "black swan awareness" column which I don't think it is appropriate to carry over; in particular you'd need to rewrite the equation entirely to deal with an anti-Taleb, who doesn't believe in black swans at all. (It also needs to deal with the issue of repricocity; if somebody doesn't hang out with you, you can't hang out with them.)
You probably end up with a circle, the size of which determines what trends Taleb will notice; for the size of the apparent circle used for the fan, I think Taleb will notice a slight downward trend with 100-120 IQ people, followed by a general upward trend - so being slightly smart would be negatively correlated, but being very smart would be positively correlated. Note that the absolute smartest people - off on the far right of the distribution - will observe a positive correlation, albeit a weaker one. The people absolutely most into black swan awareness - generally at the top - likewise won't tend to notice any strong trends, but it will tend to be a weaker positive correlation. The people who are both very into black swan and awareness, and also smart, will notice a slight downward correlation, but not that strong. People who are unusually black swan un-aware, and higher-but-not-highest IQ, whatever that means, will instead notice an upward correlation.
The net effect is that a randomly chosen "smart person" will notice a slight upward correlation.
There's a phenomenon in multidimensional motion called "gimbal locking", in which the number of effective dimensions decrease over time under motion owing to local correlations between the dimensions, which I believe may be relevant here.
Yes, it does depend on the selection model; my point was that the selection model you were using made the same predictions for everybody, not just Taleb. And yes, changing the selection model changes the results.
However, in both cases, you've chosen the selection model that supports your conclusions, whether intentionally or accidentally; in the post, you use a selection model that suggests Taleb would see a negative association. Here, in response to my observation that that selection model predicts -everybody- would see a negative association, you've responded with what amounts to an implication that the selection model everybody else uses produces a positive association. I observe that, additionally, you've changed the labeling to imply that this selection model doesn't apply to Taleb, and "smart people" generally, but rather their fanboys.
However, if Taleb used this selection model as well, the argument presented in the main post, based on the selection model, collapses.
Do you have an argument, or evidence, for why Taleb's selection model should be the chosen selection model, and for why people who aren't Taleb should use this selection model instead?
So ... smart people are worse than average at the task of evaluating whether or not smart people are worse than average at some generic task which requires intellectual labor to perform, and in fact smart people should be expected to be better than average at some generic task which requires intellectual labor to perform?
Isn't the task of evaluating whether or not smart people are worse than average at some generic task which requires intellectual labor to perform, itself a task which requires intellectual labor to perform? So shouldn't we expect them to be better at it?
ETA:
Consider the hypothetical collider from the perspective of somebody in the middle; people in the upper right quadrant have cut them off, and they have cut people in the lower left quadrant. That is, they should observe exactly the same phenomenon in the people they know. Likewise, a below average person. Thus, the hypothetical collider should lead every single person to observe the same inverted relationship between IQ and black swan awareness; the effect isn't limited to the upper right quadrant. That is, if smart people are more likely to believe IQ isn't particularly important than less-smart people, this belief cannot arise from the hypothetical collider model, which predicts the same beliefs among all groups of people.
Another term for this pattern of behavior is "the script"; this terminology, and the related narrative-oriented way of framing the behavior, seems particularly common as arising from LSD usage, dating back something like sixty years at this point to an individual whose name I can't quite recall.
In this framing, people see themselves as characters living out a story; the grayed-out options are simply those things that are out of character for them. Insofar as your character is "agent of chaos", as another commenter alludes to, you still have grayed-out options. They're the things that you wouldn't ever do; say, go to college, get a degree as an accountant, marry an average woman, get a steady 9-5 job, get an affordable car, have three kids, live in a boring neighborhood, and keep your lawn average and tidy while hosting semiannual barbecues for the next forty years until you retire into a life of obscurity.
Granted that's a bit of a long-term plan; the grayed out options might just be the boring sequence of events that lead to enacting and maintaining that plan. But note that this should be on the table, at all, if you are truly acting chaotically; deliberate chaos is surprisingly orderly.
But even that isn't really addressing "the script", the "grayed out options" - they're not just the things you wouldn't ever choose to do, but much more importantly, the things you'd never even consider doing in the first place. For example, you could sew a blue dress and burn it to symbolize the death of your youth. But you see the thing - now you consider doing it. It's not just the things that are in or out of character at a surface level, but the things that are in or out of character at a habitual level, the kinds of thoughts you think, the kinds of ideas you come up with.
Charlie, playing the Wildcard in It's Always Sunny, had approximately one idea; everything else was grayed out. Even though he was the wildcard in the show, and came up with random ideas constantly - as soon as he was actually playing the wildcard, his vision narrowed.
Similarly for us. For most of us, indeed, the older we get, the more "character development" and "personal history" we have, the narrower the range of behavior we consider engaging in; even breaking the monotony of the routine becomes its own kind of predictable routine, the sort of thing "we" would do, so the sort of thing we find ourselves doing.
What's remarkable is that it isn't any kind of constraint on our actual ability to write a character outside our own script; pretty much everybody can think of an acquaintance who does things we ourselves would not, and imagine the kind of thing that person would do.
The topic question is "Why is Toby Ord's likelihood of human extinction due to AI so low?"
My response is that it isn't low; as a human-extinction event, that likelihood is very high.
You ask for a comparison to MIRI, but link to EY's commentary; EY implies a likelihood of human extinction of, basically, 100%. From a Bayesian updating perspective, 10% is closer to 50% than 100% is to 99%; Ord is basically in line with everybody else, it is EY who is entirely off the charts. So the question, why is Ord's number so low, is being raised in the context of a number which is genuinely unusually high; the meaningful question isn't what differentiates Ord from EY, but what distinguishes EY from everybody else. And honestly, it wouldn't surprise me if EY also thought the risk was 10%, and thought that a risk of 10% justifies lying and saying the risk is 100%, and that's the entirety of the discrepancy.
As for potential reasons, any number - for example, maybe superintelligence of the sort that would reliably be capable of wiping humanity out just isn't possible and what we could create would only succeed 10% of the time, or maybe it isn't possible on hardware we'll have available in the next century, or maybe there's a lot more convergence in what we might think of as morality-space than we currently have reason to expect, or maybe there is a threshold of intelligence where acausal negotiation is standard and any given superintelligence will limit particular kinds of actions - not to mention the many possibilities where the people developing the superintelligence get it horribly wrong but in a way that doesn't lead to human extinction. We're basically guessing at unknown unknowns.
From my perspective, I think intelligence is a lot more complicated than most people think, and the current batch of people are doing the intelligence-construction-equivalent of trying to build a house by randomly nailing boards to other boards, and thinking they're onto something when they manage to create something that manages to behave like a roof in that it can be used to keep the rain off your head if you hold it just right; I think even a .01% risk of human extinction is giving AI development a lot of credit.
(Also, I think people greatly underestimate how difficult it will be, once they get the right framework to enable intelligence, to get that framework to produce anything useful, as opposed to a superstitious idiot / internet troll.)
Remember that mathematics is something we make up; mathematics isn't fundamental to, prior to, or indeed related to existence itself at all; mathematics is the process of formalizing rules and seeing what happens. You can invent whatever rules you want, although the interesting stuff generally doesn't really happen unless the rules are consistent / satisfiable with respect to one another.
The fact that mathematics happen to be useful in describing reality doesn't imply that reality is fundamentally mathematical, except in the sense that reality does something like follow a consistent set of rules, and there may be a deep isomorphy between all sets of self-consistent rules - but it's also entirely possible that the mathematics we invent, we invent for a reason, and those reasons have an inherent relationship with the rules the universe itself follows. Personally I lean towards "deep isomorphy", which would imply that there's only one "mathematical universe".
Let's consider inconsistent mathematics for a moment, however, because I will observe that your paradox does not depend upon the rules of the "mathematical universe" actually being consistent / satisfiable - the paradox doesn't depend on the idea that the universe being described can exist, only on the idea that an entity described by the mathematical framework can be aware of its own existence. Suppose for a moment that there exists a mathematical system with one or more contradictions, which is still capable of "running" a dynamical system for a given set of limited parameters such as to give rise to an entity in that system which is "aware of its existence" (suppose for the sake of argument that the contradictions do not present a problem for that limited set of parameters, which includes a finite extent of time). Does that entity "exist" for your purposes?
Note, as you consider this question, that one of the central claims of a religion which focuses largely on examining oneself is that you do not, in fact, exist. Granted I think this claim is misleading, and I'd say the proper claim is something more like "The you that you think of as yourself is more like a mental image of yourself and is wholly imaginary", except that is also misleading, and "You don't exist" is actually somewhat closer to the true claim being made. However, I think it is particularly applicable here, because the entity that makes the claim "Cogito ergo sum" is, in a particular sense, not actually real; or at least is real in the same sense that a mathematical entity which is examined to see whether or not it thinks it exists, which itself is real in the same sense as these words are real.
Are these words real? They're embedded in physical hardware somewhere. The act of running the dynamical system to see whether or not an entity thinks it exists, is also the act of embedding the entity being examined in physical hardware somewhere. Unrolling the function to see whether or not an entity thinks it exists is equivalent to making that entity exist.
What if we could prove such an entity exists in a mathematical framework, without instantiating the specific entity by actually running the function? Well, I suspect such a proof is impossible, but supposing it isn't; does that entity actually think that it exists? It would have to exist in order to do so, no? This no longer seems particularly paradoxical; I would analogize to a human being whose existence is contingent on my having sex with a particular person at a particular time. They would think they exist, if they come to exist; this doesn't imply existence.
But supposing the paradox is still unresolved, I'd add the following considerations: Is the proof that such an entity would exist, if the function were run, itself a proof that the entity exists regardless of whether or not the function is run? Does the proof cause the entity to exist, or does it exist regardless of whether or not any such proof is attempted? (Do entities exist in the infinite possible encodings of the digits of irrational numbers?)
Suppose astronomers detect an asteroid, and suggest a 10% chance of it hitting the Earth on a near-pass in 2082. Would you regard this assessment of risk as optimistic, or pessimistic? How many resources would you dedicate to solving the problem?
My understanding is that 10% isn't actually that far removed from what many people who are deeply concerned about AI think (or, for that matter, people who aren't that concerned about AI think - it's quite remarkable how differently people can see that 10%); they just happen to think that a 10% chance of total extinction is a pretty bleak thing, and ought to get our full attention. Indeed, I'd bet there's somebody around here who is deeply concerned about AI risk who assesses the risk as 1%. Remember that that risk of total human annihilation is greater than the risk of COVID to any one individual, and our society suffered massive upheaval to limit the risks there.
Which is to say - I don't think FHI or Toby Ord are significantly more optimistic than people who are deeply concerned about AI risk.
I think this is a useful abstraction.
But I think the word you're looking for is "god". In the "Bicameral Consciousness" sense - these egregores you refer to are gods that speak to us, whose words we know. There's another word, zeitgeist, that refers to something like the same thing.
If you look in your mind, you can find them; just look for what you think the gods would say, and they will say it. Pick a topic you care about. What would your enemy say about that topic? There's a god, right there, speaking to you.
Mind, in a sense, they are not a civilization-spanning entity; they're instantiations, individual, unique, and varying. Just as there isn't a single English Language, but rather hundreds of millions of subtle variations on the same structure, inhabiting hundreds of millions of minds. They don't coordinate nearly as well as you'd think; frequently people with different instantiations of what we might think of as the same overarching creature will branch, and start fighting with each other.
They're each part of the myriad simulations each of us run, constantly, of how the world works.
Also - they rely on concurrence, on feedback. If we collectively stop thinking about them for an hour, many of them will easily die. And we have something like an immune system against them - boredom, the search for novelty, something else - we get tired of zeitgeists, we move on to another. Like a pandemic, they have a shelf life, at the end of which they either end, or turn into something harmless.
"Anti-Slavic" is a bit reductionist and slightly skew of the truth, but basically, anti-Slavic, in the same way that an overly reductionist version of the Western perspective on what "Nazi" stands for would be "Anti-Semitic".
Note that the Russian perspective on what "Nazi" represents doesn't necessarily look the same as your perspective of what "Nazi" represents.
If we consider the extra dimension(s) on which the amplitude of the wave function given to the Schrodinger Equation, the wave function instead defines a topology (or possibly another geometric object, depending on exactly what properties end up being invariant.)
If the topology can be evaluated over time by some alternative mathematical construct, that alternative mathematical construct may form the basis for a more powerful (in the sense of describing a wider range of potential phenomena) physics, because it should be constructable in such a way as to not possess the limitations of the Schrodinger Equation that the function returns a value for the entire dimensional space under consideration. (That is, observe that for y=sin(x), the waveform cannot be evaluated in terms of y, because it isn't defined for all of y.)
Additionally, the amplitude of quantum waves are geometrically limited in a way that a geometric object possessing the dimension(s) of amplitude shouldn't be; quantum waves have an extent from 0 to the amplitude, whereas a generalized geometric object should permit discontinuous extents, or extents which do not include the origin, or extents which cross the origin. If we treat the position but not the measure of the extent of the dimension(s) of amplitude as having topological properties, then with the exception of discontinuous extents / amplitudes, many of these geometries may be homotopic with the wave-function itself; however, there may be properties that can be described in terms of a geometric object / topology that cannot be described in terms of the homotopic wave function.
Suppose for a moment your washing machine is broken.
You have some options; you could ignore the problem. You could try to fix it yourself. You could call somebody to fix it. This isn't intended to be a comprehensive list of options, mind, these are cached thoughts.
Each of these options in turn produce new choices; what to do instead, what to try to do to fix it, who to call.
Let's suppose for a moment that you decide to call somebody. Who do you call? You could dial random numbers into your phone, but clearly that's not a great way of making that decision. You could look up a washing machine repair company on the internet; let's suppose you do this.
How do you decide which repair company to call? There are reviews - these are choices other people have made about how they liked the service. But before you even get there, what site do you use to get reviews? That's a choice. Maybe you let Google make that choice for you - you just pick whatever is the first listed site. The search engine is making choices for you; the review site algorithm is making choices for you; the people who posted reviews are making choices for you. Out of a vast space of options, you arrive at only a few.
Notice all the choices other people are making on your behalf in that process, however. You're not calling a car mechanic to repair your washing machine, yet that is, in fact, an option.
---
Suppose you need to drive to a grocery store in a new city. What choices are you making, and what choices do you ask your cell phone navigation application to make for you? Are you making more or less choices than your parents would? What about your grandparents? What is the difference in the kind and quantity of choices being made?
Are there differences in the quality of choices being made? Who benefits from the choices we make now?
What are you calling the "framing" of a decision? Is it something other than a series of decisions about what qualities with regard to the results of that decision that you care about?
The point is that meaningful labor is increasingly "selection effort", the work involved in making a decision between multiple competing choices, and some starter thoughts about how society can be viewed once you notice the idea of making choices as meaningful labor (maybe even the only meaningful form of labor).
The idea of mapping binary strings to choices is a point that information is equivalent to a codification of a sequence of choices; that is, the process of making choices is in fact the process of creating information. For a choice between N options, the options can be considered a series of binary gates, whose value can be 0 or 1, and thus the choice between those options produces a binary string; information. Or a number, if you prefer to think of it that way. That is, making decisions is an information-producing activity.
Suppose you have a list of choices a selection must be made from, and that the decision theory axioms of orderability and transitivity apply.
It should then be possible to construct a binary tree representing this list of choices, such that a choice can be represented as a binary string.
Likewise, a binary string, in a certain sense, represents a choice.
In this specific sense, what computers automate is the process of selection, of choice. Noticing this, and noticing that computers have automated away considerable amounts of "work", we must notice that "work", in the occupational sense, is to a significant extent the process of making selections. The process of mechanization has been the process of converting physical labor into selective labor, and in some cases the creation of physical heuristics that substantially solve selective problems - a vacuum operates on a physical heuristic that things of certain sizes and weights are undesirable to have on/in carpeting, for instance.
Noticing that the information age has largely been an ongoing project of automating selection efforts, one notable exception does crop up - cases of crowdsources selection efforts. Upvotes and downvotes and retweets and the various other crowdsourced mechanisms by which selective pressure is created are selective labor. We tend to think of this process as being to our own benefit, but I will observe the massive amount of monetary value that is extracted by the hosting platforms in the process - value that the hosting platforms enable, but does not create.
There are additionally individuals who create value - and, I would hope, a livelihood - based purely on selective labor we might not notice as labor. Curated musical lists, for example. Additionally, I notice an increasing trend of corporate entities performing selective effort on behalf of individual clients; when you get down to it, housing renting versus buying is a tradeoff between selection power (the ability to make selections) versus selection effort (the requirement to do so). And I notice the cost of renting is increasing relative to the cost of buying, and yet people I know who could buy, are still choosing to rent, and those who do buy, are increasingly buying housing which limits their exposure to selection effort (such as buying condos, duplexes, and in HOAs).
Other ways in which it looks like society is increasingly outsourcing selective effort: Political beliefs, truth-deciding (science), investment, maintenance of household items, movie selection, book selection, food selection. Anything where a company sends people a box of preselected items on a regular basis, where that is supplanting a previous personal selection effort.
The combination of these two things, to me, is interesting, combined with other observations of society. Because the high degree of selective effort we undertake for free in some situations, combined with what I can only describe as an increasingly widespread social resistance to other forms of selective effort, looks like fatigue of executive function. We spend considerable effort making decisions for the financial benefit of social media corporations, and have little selective energy left to make decisions about our own lives.
---
This situation might be a problem, might not. It looks odd to me, to be certain, and I'm increasingly dubious over the structure of ownership of commons whose value is not created by those who extract value from them.
However, I think it's more important than I'm suggesting here. I'll return to the idea that mechanization has been the process of eliminating all work except selective effort: This suggests that the effectiveness of a corporation is entirely in how effectively selective efforts take place.
The problem for a corporation, government, or other large impersonal entity is compounded, because the first selective effort, is selecting people to make selective efforts, who in turn will be selecting people to make selective efforts. A structure must be created to maintain a consistent form of selective effort - bureaucracy.
This begins to look a lot like the alignment problem, because, indeed, that's basically what it is. And it is perhaps illlustrative that thousands of years of social development have only really come up with one functional solution to the problem of alignment corruption: Competition, such that structures whose alignment becomes corrupted are dismantled or destroyed. Which is to say, yet another form of selection.
Would you willingly go back in time and re-live your life from the beginning, with all the knowledge you have now? Say, knowing what stocks to purchase, what cryptocurrencies are worth buying and when, being able to breeze through education and skip ahead in life, and all the other advantages you would have?
If the answer to that is yes, then observe that this is exactly the same thing.
The point of this being that you don't actually think of past-you, present-you, and future-you as you in the same sense. You'll happily overwrite past-you with present-you, but you'd see it as a problem if future-you overwrote present-you, so far as to be equatable to dying.
What, exactly, does it even mean for "you" to exist for 100k years?
Is the "you" from yesterday "you"? Would you be comfortable with your conscious mind being replaced with the conscious mind of that entity? What about the "you" from tomorrow"? What about the "you" from 100k years in the future? If that's still "you", should it be a problem for your mind to be erased, and that mind to be written in its place?
What does it mean, for a thing to move?
First, what phenomenon are we even talking about? It's important to start here. I'm going to start somewhat cavalierly: Motion is a state of affairs in which, if we measure two variables, X and T, where X is the position on some arbitrary dimension relative to some arbitrary point using some arbitrary scale, and T is the position in "time" as measured by a clock (also arbitrary), we can observe that X varies with T.
Notice there are actually two distinct phenomena here: There is the fact that "X" changed, which I am going to talk about. Then there is the fact that "T" changed, which I will talk about later. For now, it is taken as a given that "time passes". In particular, the value "T" refers to the measurement of a clock whose position is given by X.
Taking "T" as a dimension for the purposes of our discussion here, what this means is that motion is a state of affairs in which a change in position in time additionally creates a change in position in space; that is, motion is a state of affairs in which a spacial dimension, and the time dimension, are not independent variables. Indeed, special relativity gives us precisely the degree to which they are dependent on one another.
Consider geometries which are consistent with this behavior; if we hold motion in time as a given - that is, if we assume that the value on the clock will change - then the geometry can be a simple rotation.
However, suppose we don't hold motion in time as a given. What geometry are we describing then? I think we're looking at a three-dimensional geometry for this case, rather than a four-dimensional geometry.
Well, if my crackpot physics is right, it actually kind of reduces the probability I'd assign to the world I inhabit being "real". Seriously, the ideas aren't complicated, somebody else really should have noticed them by now.
But sure it makes predictions. There should be a repulsive force which can be detected when the distance between two objects is somewhere between the radius of the solar system and the radius of the smallest dwarf galaxy. I'd guess somewhere in the vicinity of 10^12 meters.
Also electrical field polarity should invert somewhere between 1 and 10^8 meters. That is, if you have an electrical field, and you measure it to be positive or negative, if you move some distance away, it should invert to be negative or positive.
Are these predictions helpful? Dunno.
Either way, however, it doesn't really say anything about whether the world is internal or external.
Does any of that make an observable difference.
Not really, no. And that's sort of the point; the claim that the world is external is basically an empty claim.
I think one of the more consistent reports of those who connect with that voice is that they lose that fear.
Because it's expensive, slow, and orthogonal to the purpose the AI is actually trying to accomplish.
As a programmer, I take my complicated mirror models, try to figure out how to transform them into sets of numbers, try to figure out how to use one set of those numbers to create another set of those numbers. The mirror modeling is a cognitive step I have to take before I ever start programming an algorithm; it's helpful for creating algorithms, but useless for actually running them.
Programming languages are judged as helpful in part by how well they do at pretending to be a mirror model, and efficient by how well they completely ignore the mirror model when it comes time to compile/run. There is no program which is made more efficient by representing data internally as the objects the programmers created; efficiency gains are made in compilers by figuring out how to reduce away the unnecessary complexity the programmers created for themselves so they could more easily map their messy intuitions to cold logic.
Why would an AI introduce this step in the middle of its processing?