LessWrong 2.0 Reader

an intuition for avoiding motivated cognition in reflecting on moral principles.

if we were starting over, new to the world but with all the technology we have now, would we recreate this practice?

example: we start and out and there's us, and these innocent fluffy creatures that can't talk to us, but they can be our friends. we're just learning about them for the first time. would we, at some point, spontaneously choose to kill them and eat their bodies, despite the fact we have huel and supplements? to me, the answer seems obviously not.

The research on humans in 0 g is only relevant if you want to send humans to mars. And such a mission is likely to end up being an ISS on mars. Or a moon landings reboot. A lot of newsprint and bandwidth expended talking about it. A small amount of science that could have been done more cheaply with a robot. And then everyone gets bored, they play golf on mars and people look at the bill and go "was that really worth it?"

Oh and you would contaminate mars with earth bacteria. 

A substantially bigger, redesigned space station is fairly likely to be somewhat more expensive. And the point of all this is still not clear. 

Current day NASA also happens to be in a failure mode where everything is 10 to 100 times more expensive than it needs to be, projects live or die based on politics not technical viability, and repeating the successes of the past seems unattainable. They aren't good at innovating, especially not quickly and cheaply.

The energy stored within the nitrogen triple bond, one of the strongest common bonds in chemistry, is ~10 eV which is a bit more than 15*10^-19 J. This is considered *very stable*. It is quite the feat for some biological processes to be able to break this bond. Now, an average human punch confers around 150 J of energy. So, if you had some very strange means of directing that energy, punching the air could split around 10^20 nitrogen molecules which is around 4 ml (4.65 mg) at room temperature.

I can't find any off the top of my had, but I'm pretty sure the LW/Lightcone salary question has been asked and answered before, so it might help to link to past discussions?

Random thoughts. You can relatively simply get a global phase factor at each timestep if you want. I don;t think a global phase factor at each step really counts as meaningfully different though. Anyway, as an example of this:

$f(x_{k-1},x_k,x_{k+1})=x_{k-1}$

So that, at each (classical) timestep every single element of the CA tape just moves one step to the right. (So any patterns of 1's and 0's just orbit the tape in circles forever, unchanging.). Its quite a boring CA, but a simple example.

We can take the quantum CA that is exactly the same, but with some complex phase factor:

$f_q(x_{k-1},x_k,x_{k+1},y_k)={\delta }_{x_{k-1}}^{y_k}\exp \left(i\theta \right)$

Where the delta function is saying "1 iff $y_k=x_{k-1}$ , else 0."

This is exactly the same as the old classical one (everything moves on step to the right), but this time we also have a global phase factor applied to the total system. The total phase factor is $\exp \left(i\theta N\right)$, where N is the total number of cells on the tape.

Tiny bit more interesting:

$f_q(x_{k-1},x_k,x_{k+1},y_k)={\delta }_{x_{k-1}}^{y_k}\exp \left(i\theta x_{k-1}\right)$

Now we only gain phase factors on values of 1, so the global phase depends on the total number of 1's on the tape, rather than its length.

To get proper quantum stuff we need phase factors that are not global. (IE some relative phases). I feel like this equation below is a reasonable kind of place to start, but I have run out of time for now so might return to this later.

$f_q(x_{k-1},x_k,x_{k+1},y_k)=\left(1/\sqrt{2}\right)\left[{\delta }_{x_{k-1}}^{y_k}\exp \right(i\theta )+{\delta }_{x_{k+1}}^{y_k}\exp (-i\theta \left)\right]$

Norms can accomplish this as well - I wrote about this a couple weeks ago.

Yes. Thanks. Good explanation.

Are you familiar with Davidad's program working on compositional world modeling? (The linked notes are from before the program was launched, there is ongoing work on the topic.)

The reason I ask is because embedded agents and agents in multi-agent settings should need compositional world models that include models of themselves and other agents, which implies that hierarchical agency is included in what they would need to solve. 

It also relates closely to work Vanessa is doing (as an "ARIA Creator") in learning theoretic AI, related to what she has called "Frugal Compositional Languages [LW · GW]" and see this work [LW · GW] by @alcatal [LW · GW] - though I understand both are not yet addressing on multi-agent world models, nor is it explicitly about modeling the agents themselves in a compositional / embedded agent way, though those are presumably desiderata.

Really interesting work! I have two questions:

1. In the model organisms of misalignment -section it is stated that AI companies might be nervous about researching model organisms because it could increase the likelihood of new regulation, since it would provide more evidence on concerning properties in AI system. Doesn't this depend on what kind of model organisms the company expects to be able to develop? If it's difficult to find model organisms, we would have evidence that alignment is easier and thus there would be less need for regulation.  

2. Why didn't you listed AI control work as one of the areas that may be slow to progress without efforts from outside labs? According to your incentives analysis it doesn't seem like AI companies have many incentives to pursue this kind of work, and there were zero papers on AI control.

Thanks for the comment Steven.

Your alternative wording of practical CF is indeed basically what I'm arguing against (although, we could interpret different degrees of the simulation having the "exact" same experience, and I think the arguments here don't only argue against the strongest versions but also weaker versions, depending on how strong those arguments are).

I'll explain a bit more why I think practical CF is relevant to CF more generally.

Firstly, functionalist commonly say things like

Computational functionalism: the mind is the software of the brain. (Piccinini [LW(p) · GW(p)])

Which, when I take at face value, is saying that there is actually a program being implemented by the brain that is meaningful to point to (i.e. it's not just a program in the sense that any physical process could be a program if you simulate it (assuming digital physics etc)). That program lives on a level of abstraction above biophysics.

Secondly, computational functionalism, taken at fact value again, says that all details of the conscious experience should be encoded in the program that creates it. If this isn't true, then you can't say that conscious experience is that program because the experience has properties that the program does not.

Putnam advances an opposing functionalist view, on which mental states are functional states. (SEP)

He proposes that mental activity implements a probabilistic automaton and that particular mental states are machine states of the automaton’s central processor. (SEP)

the mind is constituted by the programs stored and executed by the brain (Piccinini [LW(p) · GW(p)])

I can accept the charge that this still is a stronger version of CF that a number of functionalists subscribe to. Which is fine! My plan was to address quite narrow claims at the start of the sequence and move onto broader claims later on.

I'd be curious to hear which of the above steps you think miss the mark on capturing common CF views.