LessWrong 2.0 Reader

I’m not sure if you intended the allusion to “the tendentious assumption in the other comment thread that courts are maximally adversarial processes bent on on misreading legislation to achieve their perverted ends”, but if it was aimed at the thread I commented on… what? IMO it is fair game to call out as false the claim that

It only counts if the $500m comes from "cyber attacks on critical infrastructure" or "with limited human oversight, intervention, or supervision....results in death, great bodily injury, property damage, or property loss."

even if deepfake harms wouldn’t fall under this condition. Local validity matters.

I agree with you that deepfake harms are unlikely to be direct triggers for the bill’s provisions, for similar reasons as you mentioned.

so basically, we could make an AI that wants to maximize a variable called Utility

Oh, maybe this is the confusion.  It's not a variable called Utility.  It's the actual true goal of the agent.  We call it "utility" when analyzing decisions, and VNM-rational agents act as if they have a utility function over states of the world, but it doesn't have to be external or programmable.

I'd taken your pseudocode as a shorthand for "design the rational agent such that what it wants is ...".  It's not literally a variable, nor a simple piece of code that non-simple code could change.

Notwithstanding the tendentious assumption in the other comment thread that courts are maximally adversarial processes bent on on misreading legislation to achieve their perverted ends, I would bet that the relevant courts would not in fact rule that a bunch of deepfaked child porn counted as "Other grave harms to public safety and security that are of comparable severity to the harms described in subparagraphs (A) to (C), inclusive", where those other things are "CBRN > mass casualties", "cyberattack on critical infra", and "autonomous action > mass casualties".  Happy to take such a bet at 2:1 odds.

But there are some simpler reason that particular hypothetical fails:

• Image models are just not nearly as expensive to train, so it's unlikely that they'd fall under the definition of a covered model to begin with.
• Even if someone used a covered multimodal model, existing models can already do this.

See:

(2) “Critical harm” does not include any of the following:

(A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative.

If you’re late call me at 510 998 4771

If you're saying we should work on enlightenment before working on AGI x-risk, I disagree.

We may well not have the time.

I am very aware that we may not have time. 

But sometimes people will make an argument "unless we figure out X our attempts at resolving x-risk are really really doomed." 

Lots of different people, for lots of different versions of X, actually: 

• solving agent foundations, 
• scaling human enlightenment (so that we stop flailying making things worse all the time), 
• building a culture that can talk about the fact that we can't see or talk about conflict (and so our efforts to do good in the world get predictably coopted by extractive forces).
• etc.

I definitely want to know if any of those statements are true, for any particular version of X, even if we don't have time to do X before the deadline.

Not having time doesn't have any b

If things go wrong[1] then our neural net will conclude that it has high status despite all evidence to the contrary. We have programmed schizophrenia.

No, you've programmed grandiose delusions - a lot more goes wrong with schizophrenia than just that.

I don’t need a “theory” to explain how a “hypothetical” learning algorithm can build a generative model that can represent this kind of information in its latent variables, and draw appropriate inferences.

Sure, but we would still need a separate explanation if we want to understand how representation/reference works in a model (or in the brain) itself. If we are interested in that, of course. It could be interesting from the standpoint of philosophy of mind, philosophy of language, linguistics, cognitive psychology, and of course machine learning interpretability.

If, when you run these algorithms, you wind up with all kinds of edge cases where it’s unclear what is “about” what, (and you do), then that’s a sign that you should not be treating “aboutness” as a bedrock principle in the first place.

I don't think we did run into any edge cases of representation so far where something partially represents or is partially represented, like chess is partially sport-like. Representation/reference/aboutness doesn't seem a very vague concept. Apparently the difficulty of finding an adequate definition isn't due to vagueness.

That being said, it's clearly not necessary for your theory to cover this topic if you don't find it very interesting and/or you have other objectives.

The "one weird trick" to getting the right answers is to discard all stuck, fixed points. Discard all priors and posteriors. Discard all aliefs and beliefs. Discard worldview after worldview. Discard perspective. Discard unity. Discard separation. Discard conceptuality. Discard map, discard territory. Discard past, present, and future. Discard a sense of you. Discard a sense of world. Discard dichotomy and trichotomy. Discard vague senses of wishy-washy flip floppiness. Discard something vs nothing. Discard one vs all. Discard symbols, discard signs, discard waves, discard particles. 

All of these things are Ignorance. Discard Ignorance.

They don't seem like ignorance to me! Many of them seem distinctly like knowledge!

You probably don't understand what I just said. 

It does seem that way. : P

That's fine. 

Ok then, what would be a polite and appropriate way to respond to your speech acts like these? Should I state that they sound wrong to me? Should I ignore the content them and treat them as your artistic expression? 
 

Yes, but with a very different description of the subjective experience -- kind of like getting a sunburn on your back feels very different than most other types of back pain.

No worldview will be able to output the best answer in every circumstance. This is not a matter of compute.

Why not? Or why can't you have a worldview that computes the best answer to any given "what should I do" question, to arbitrary but not infinite precision?

Is this something that you think you know mainly because of your personal experience with pervious worldviews failing you? Some other way?

Is it something that your reader should be able to infer from this post, or from their own experience of life (assuming they're paying attention?)