Posts
Comments
Some psychiatry textbooks classify “overvalued ideas” as distinct from psychotic delusions.
Depending on how wide you make the definition, a whole rag-bag of diagnoses from the DSM V are overvalued ideas (e.g, anorexia nervosa over valuing being fat).
Possibly similar dilemma with e.g. UK political parties, who generally have a rule that publicly supporting another party’s candidate will get you expelled.
An individual party member, on the other hand, may well support the party’s platform in general, but think that that one particular candidate is an idiot who is unfit to hold political office - but is not permitted to say so,
(There is a joke about the Whitby Goth Weekend that everyone thinks half the bands are rubbish, but there is no consensus on which half that is. Something similar seems to hold for Labour Party supporters.)
An organisation such as the Catholic Church primarily wants to perpetuate its own existence, so of course the official doctrine is that they are The One True Church.
An individual Catholic, on the other hand, might genuinely believe that the benefits of religion are also available from other suppliers.
COVID-19 killed, idk, tens of millions worldwide rather than hundreds of millions.
But consider that an example of a (biological) virus takeoff of the order of months.
So the question for AGI takeoff .. death rate growing more rapidly than COVID19 pandemic, or slower?
Takeoff speed could be measured by e.g. the time between the first mass casualty incident that kills thousands of people vs the first mass casualty incident that kills hundreds of millions.
(This bit isn't serious) "i mean, a days long takeoff leaves you will loads of time for the hypersonic missiles to destroy all of Meta's datacenters."
Minutes long takeoff...
[By comparison, I forget the reference but there is a paper estimating how quickly a computer virus could destroy most of the Internet. About 15 minutes, if I recall correctly.]
e.g. After the mass casuality incident...
"You told the government that you had a shutdown procedure, but you didnt, and hundreds of people died because you knowingly lied to the government."
My personal view on how it might help:
- Meta will probably carry on being as negligent as ever, even with sb1047
- When/if the first mass casualty incident happens, sb1047 makes it easier for Meta to be successfully sued
- After that.AI companies become more careful.
On the one hand, we encounter a lot of arguments about gender that seem, to me, to be philosophically bad. Maybe a good source of reasoning fallacies you might be able to spot in other contexts too,
On the other hand, the more I think about it, the less I care about the object level isue. It seems inevitable that there are going to be various sorts of statistical outliers and hard to classify cases, and really is it that big a deal?
I know one person who is intersex, and I know because they're involved in right activism and they told me, Probably couldnt tell otherwise. Could well be xome other people I know are intersex and haven't told me.Maybd they dont even know themselves, as it appears that this information was frequently witheld by doctors. Shrug.
Also: if you take gender as chromosomal sex, then (a) tye aforementioed person is totally genuinely both xx and xy because they have mosaic chromosomes; and (b) it seems really strange for your gender to sometimes be something that you, yourself, do not know,.
A financial conflict of interest is a wonderous thing...
"Okay, Beatrice. There was no alien, and the flash of light you saw in the sky wasn't a UFO. Swamp gas from a weather balloon was trapped in a thermal pocket and refracted the light from Venus -- Men in Black
The TV Series "Dark Skies" .. in which the US Government is orchestrating a coverup about the involvement of giant prawns from outer space in the Roswell incident, the JFK assassination, the shootdown of Gary Power's US spyplane, erc.
I agree that Vernor Vinge's A Deepness in the Sky is an example.
Almost but not quite an example: Edmund Cooper's The Overman Culture. It is obvious to the reader from the outset that the characters cannot be when and where they think the are (evacuated from London during World War 2).Maybe not enough deceiver's perspective to count.
Also not quite: Gene Wolfe's The Book of the New Sun.
This is pretty much why many people thought that the term "Open Source" was a betrayal of the objectives of the Free Software movement,
"Free as in free speech, not free bewr" has implication that "well, you can read the source" lacks.
Yeah, many of the issues are the same:
*RLHF can be jail broken with prompts, so you can get it to tell you a sexy story or a recipe for methamphetamine. If we ever get to a point where LLMs know truly dangerous things, they'll tell you those, too.
*Open source weights are fundamentally insecure, because you can finetune out the guardrails. Sexy stories, meth, or whatever.
The good thing about the War on Horny
- probably doesnt really matter, so not much harm done when people get LLMx to write porn
- Turns out, lots of people want to read porn (surprise! who would have guessed?) so there are lots of attackers trying to bypass the guardrails
- This gives us good advance warning that the guardrails are worthless
Also note that Open Source precludes doing this ...
The basic Open SOurce deal is that absolutely anyone can take the product and do whatever they like with it, without paying the supplier anything.
So
- The vendor cannot prevent the customer doing something bad with the product (If there is a line of code that says "dont do this bad thing", then the customer can just delete it
- The vendor also cannot charge the customer an insurance premium base on how likely the customer is to do something ba with the product
... which would suggest that Open Source is only viable in areas where there isn't much third party liability.
With a nod to the recent Crowdstrike incident .... if your AI is sending out packets to other people;s Windows systems, and bricking them about as fast it can send packets through its ethernet interface, your liability may be expanding rapidly. An additional billion dollars for each hour you dont shut it down sounds possible.
If your AI is doing something that's causing harm to third parties that you are legally liable for .. chances are, whatever it is doing, it is doing it at Internet speeds, and even small delays are going to be very, very expensive.
I am imagining that all the people who got harmed after the first minute or so after the AI went rogue are going to be pointing at SB1047 to argue that you are negligent, and therefore liable for whatever bad thing it did.
If quantum computers really work, for more than 3 qbits, then I think I will believe in infinite worlds interpretation.
On the other hand, if there turns out to be some fundamental reason why quantum omputers with many qbits cant exist then maybe not.
The version where you only have 3 qbits is kind of unsatisfactory (look, there are exactly 8 parallel universes and no more...)
If American citizens who can vote from Trump are arguing over whether hr's a bad guy, there is arguably a point to it .. though I can also the case against.
But Swedish guys ... who arent even allowed to vote for Trump if they want to .. aruing over Trump? What are you doing?
(Presumably, if there is any point at all, the argument is not against Trump specifically but the global phenomenon he is part of, and whoever the Swedish Trump equivalent is).
To be truly dangerous, an AI would typically need to have (a) lack of alignment (b) be smart enough to cause harm
Lack of alignment is now old news. The warning shot is, presumably, when an example of (b) happens and we realise that both component pieces exist.
I am given to understand that in firearms training, they say "no such thing as a warning shot".
By rough analogy - envisage an AI warning shot as being something that only fails to be lethal because the guy missed.
Maybe: it's easier to capture a whole load of diverse stuff if you don't care about numerically quantifying it, and dont care about statistical significance tests, multiple testing, etc. Once you have a a candidate list of qualitative features that might be interesting, you can then ask: ok, how do I numerically measure this thing?
A possible justification of qualitative research: you do this first, before you even know which hypotheses you want to test quantitatively.
RE: autism. we might also add sensory issues/only able to concentrate on one sense at a time, and the really strange one: having fluent knowledge of the literal meanings of words, but difficulty with metaphors.
a) Are these part of the same symptom cluster as the "theory of mind" aspects of autism?
b) If so, why? Why on Earth would we expect metaphorical use of language to (i) be somehow processed by different metal modules from literal usage (ii) be somehow related to reasoning about other minds?
I actually personally know a couple of people who have the metaphors one. They tell me the issuer is that the literal meaning is just way more salient than the literal one.
AS a purely anecdotal data point, my mother had COVID19 (again) a couple of weeks ago.
We appear to be in the "nearly everybody will keep getting it regularly, for ever" phase of COVID19.
Chronic Fatigue Syndrome is, maybe - maybe - someth8ing of that kind.
Unfortunately, (a) the clinical trials of Cognitive Behavioral Therapy in this area, and as a result (b) we have a strong suspicion that it is ineffective as a treatment, and maybe is actually harmful relative to no treatment at all.
So, eg. does exercizing break you out of the bad equilibrium, or does it tighten the noose around your your neck tighter, making you more sick, disabled, and (potentially) closer to death?
So if I see verified reports of AI causing a mass casualty incident with more that %500 million in damage (or whatever the threshold in the California bill is), I shall consider that evidence on a par to seeing Lake-Town get toasted by Smaug, and update accordingly.
This analogy might not work for all the things "dragons" is standing in for in this thread ... but if I have a good statistical bound on the risk posed by dragons being low (but cannot, strictly speaking, rule out their existence entirely) I may conclude that a residual 1E-5 chance of running in to one to be a acceptable risk.
Example chosen because (a) I have really, absolutely no idea what the optimal action is here; (b) I have reason to belive that the risk is kind of low, anyway. (c) most commentors here wont have a strong opinion, so we wont have a flame war over what the right answer might be. Let it stand in for other cases where it is very, very unclear what the optimal action is.
Some, at least, of these highly politically partisan hot-button issues have the property that most people don;t have a reason for caring whether they're true or not. In which cases, shrug might be the reasonable response.
Possibly the idea of this thread is that we're not supposed to mention any real examples, go avoid gettin g caught up in culture wars.
I can think of examples where even if I am going to do 9or not do) something based on whether the claim is trur of not... (a) the risk of doing X if claim is true seems small'(b) the cost of doing X if the claim is false is small (c) evidence for or against the claim looks really weak. So, shrug.
e.g. (true story) I am in the emergency room with tachycardia caused by graves disease. One of the ER docs and the endocrinologist have a really deeply technical argument over whether administration of Hartmann;s solutipn via IV is a plus or minus in my situation. Listening go this, I gather that, really, there is not much in it. Nurse would like to know if she can go ahead and stick the IV drip in my arm. Shrug. Whatever. You have my patient consent for that procedure, yes.
I guess you could view that random number in [0,1] as a choice sequence (cf. intuitionism) and you're allowed to see any finite number of bits of it by flipping coins to see what those bits are, but you don't know the answer to any question that would require seeing infitely many bits...
At the other end of the formality scale ... for one reason or another, I am looking at the Casimir Effect in quantum field theory. Gah, zeta summation. These guys are playing fast and loose here, mathematically. Like, no way in hell would I dare write down a formalization of what's going here here.
r.g. recently I was working on a completely formalized proof (in the HOL4 theorem prover) of a routine to print out the value of an IEEE 754 floating point number.
So, OK, the program itself I can write in maybe 30 minutes. Difficulty: here we want a fully formalized proof of correctness in HOL4 (i.e. proof starts from logic and works its way up, a la Russell and Whitehead's Principia Mathematica).
Let's see: the definitions I'm going to need here are mostly obvious, in that they're formalization of stuff that's in the IEEE 754 standard: things like is_zero(), is_nan()...
Theorems are moderately obvious, in that what I want to say here is that the way you do it in the C runtime's math.h works .. except, of course, we're not doing this in C we're doing it in ML, for provabilioty reasons.
On to the proofs. OMG, I need to prove some really stupid lemmas here; in that the lemma is Obviously True, but annoying to prove.
On to the proofs... well, I guess. case split on what type of floating point value we're talking about (infinity? nan? zero? finite?), algebraically simplify...and we're done.
In "Proofs and Refutations", Imre Lakatos talks about "monster barring".
Sometimes theorems have "fine print" that rules out weird cases where the theorem would be false or meaningless, and it can be quite hard to understand why it's there/
e.g. in probability theory, the looming problem is that if I choose a point uniformly at random in [0,1] and ask what is the probability that it falls in a set S, there might be no such probability. At which point, a whole load of stuff about Borel sigma algebras appears in the statement of the theorem to make the problem go away.
A Deepness in the Sky feels like the author know he can't write female characters, but knows that women ought to feature in the plot, so is going really out of the way to avoid showing their viewpoint ... at least, in a direct way.
There are a lot of surprise plot twists, so its hard to expand on this without plot spoliers.
Oh (not a spoiler) the second narrator is obviously not being entirely truthful. The book gets a lot better when you realise they;re supposed to be read as an unreliable narrator. (So, sure, its communications intercept of an alien species approximately rendered into English by an intelligence analyst who has been enslaved by Space Nazis ... someone, somewhere, might be lying here...)
The poor naive viewer thought Neon Genesis Evangelion was just going to be about giant robots fighting, but, oh no, there seems to be a Religious Allusion.
The Quest for the Holy Grail is, in origin at least, religious.
So, maybe, the LLM is picking up on a textual similarity between (a) religious allegory; (b) Dungeons and Dragons type adventures that retain some of the trappings of religious allegory.
Could be a type of Waluigi effect: once you've started making it a religious allegory, you're stuck continuing with it, and at each stage there is some chance that your amoral and nihilistic adventurers (in th efashion of Fritz Lieber's Lankmar) will suddenly discover Spiritual Signifcance.
Serious answer: government already makes requests to e.g. DNS hosting providers to shut things down.
Non-serious answer: Turing Police, like in the William Gibson novels.
I've lost count of how many gurus have been involved in some sort of sex scandal...
(Seriously, care would seem to be advisable when choosing a yoga sex cult.)
Thanks for this post. I think that these kind of bad experience reports are valuable.
- Is meditation-induced psychosis really a thing? Sure looks like at this point, as we have a significant nuber of well-documented case histories.
- Well. how risky is meditation then? I am currently unsure on this point; the sheer number of bad experience reports, is somewhat concerning. By way of comparison: I am currently taking carbimazole for Graves disease, a drug with a maybe 1E-3 (ish) probability of a side effect that is kind of bad (agranulocytosis) ... well, I might be willing to take a 1 in 1000 risk for the benefit of the drug.
- Were there particular risk factors in these case reports, that indicate meditation is contra-indicated for some type of people? Also, not currently clear to me.
In Buddhist philosophy, there's a type of argument along the lines of ... suppose that you actually attained enlightenment. yeah, sure, sounds unlikely, as enlightenment seems kind of hard. but .. just hypothetically supposing you did ... and given what we believe "enlightenment" is supposed to be like. Then, would that "you" still be "you"?
This sounds like a terrible idea.
Though, if you're going to be put under sedation in hospital for some legit medical reason, you could have in mind a cool experiment to try when you're coming around in the recovery room.
i was sedated for endoscopy about 10 years ago,
they tell you not to drive afterwards (really, don't try and drive afterwards)
and to have a friend with you for the rest of the day to look after you
i was somewhat impaired for the rest of the day (like, even trying to cook a meal was difficult and potentially risky ... e.g. be careful not to accidentally burn yourself when cooking)
I drew a bunch of sketches after coming round to see how it affected my ability to draw.
Alternative theory (which, to be clear, I dont actually believe, but offer for consideration)
- Many of the high iq people are too autistic to be successful
- but female hormones protects against the autism somehow, without impacting iq too much
- so the successful high iq people tend to be trans more often on average
I think its more likely its the transgender - autism correlation....
- some forms of autism come with higher iq (and other forms, really really dont)
- and there's the transgender autism correlation
which together would seem to predict transgender high iq people
(and also transgender low iq that you arent seeing due to ascertainment bias)
And the really funny bit is NIST deliberately subverted the standard so that an organization who knew the master key (probably NSA) could break the security of the system. And then, in actualt implementation, the master key was changed so that someone else could break into everyone's system And, officially at least, we have no idea who that someone is. Probably Chinese government. Could be organized crime, though probably unlikely.
The movie Sneakers had this as its plots years ago.. US government puts a secret backdoor in everyone's computer system .. and, then, uh,, someone steals the key to that backdoor;
But anyway, yes, it is absolutely NISTs fault that they unintentionally gave the Chinese government backdoor access into US government computers.
https://cacm.acm.org/research/technical-perspective-backdoor-engineering/
for example. Although that paper is more about, "Given that NIST has deliberately subverted the standard, how did actual products also get subverted to exploit the weakness that NIST introduced."
While I was typing this, quetzal_rainbow made the same point
Ascertainment bias, of course, because we only see the cases where this did not work, and do not know exactly how many members of e.g. Delta Force were originally in doubt as to their gender. We can know it doesnt work sometimes.