MakoYass's Shortform

Maybe it's some legal hack, like maybe in some situations you can't dismiss unethical research, but you can dismiss fraudulent research... and a research where people were forced to falsely write that their participation was voluntary, is technically fraudulent.

The reason is to prevent the voluntary participant from later claiming that their participation was involuntary and telling that to the IRB.

'Well if your participation was involuntary, why did you sign this document?'

It kind of limits the arguments someone could make attacking the ethics of the study. The attacker would have to allege coercion on the order of people being forced to lie on forms under threat.

If someone explicitely writes into their consent forms "my participation was entirely voluntary" and the participation isn't voluntary it might be easier to attack the person running the trial later. 

The voluntary consent of the human subject is absolutely essential. 

This means that the person involved should have legal capacity to give consent; should be so situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion; and should have sufficient knowledge and comprehension of the elements of the subject matter involved, as to enable him to make an understanding and enlightened decision. This latter element requires that, before the acceptance of an affirmative decision by the experimental subject, there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonably to be expected; and the effects upon his health or person, which may possibly come from his participation in the experiment. 

The duty and responsibility for ascertaining the quality of the consent rests upon each individual who initiates, directs or engages in the experiment. It is a personal duty and responsibility which may not be delegated to another with impunity

Important to remember and stand by the Nuremberg Code in these contexts. 

The obvious bad consequence is a false sense of security leading people to just get BCIs instead of trying harder to shape (e.g. delay) AI development.

" You can't make horses competitive with cars by giving them exoskeletons. " <-- this reads to me like a separate argument, rather than a restatement of the one that came before.

I agree that BCI seems unlikely to be a good permanent/long-term solution, unless it helps us solve alignment, which I think it could. It could also just defuse a conflict between AIs and humans, leading us to gracefully give up our control over the future light cone instead of fighting a (probably losing) battle to retain it.

...Your post made me think more about my own (and others') reasons for rejecting Neuralink as a bad idea... I think there's a sense of "we're the experts and Elon is a n00b". This coupled with feeling a bit burned by Elon first starting his own AI safety org and then ditching it for this... overall doesn't feel great.

The true thing that Sagan's line might be interpreted to mean is "A claim which is very unlikely on priors needs very strong evidence to end up with a posterior probability close to 1."  "Extraordinary evidence" would ideally have been stated as "extraordinarily strong evidence", but that makes the line a bit clunkier.  Unfortunately, there is often a tradeoff between accuracy and pithiness.  Many pithy sayings require a bit of interpretation/reconstruction to get the correct underlying idea.  I think anyone who invokes a catchphrase should be aware of this, though I don't know how many people share this perspective.

Are there in fact a significant number of people who take it at the face value of "extraordinary evidence" and think it must mean it was obtained via super-advanced technology or something?

Some extraordinary claims established by ordinary evidence:

Stomach ulcers are caused by infection with Helicobacter Pylori.  It was a very surprising discovery that was established by a few simple tests.

The correctness of Kepler's laws of planetary motion was established almost entirely by analyzing historical data, some of it dating back to the ancient Greeks.

Special relativity was entirely a reinterpretation of existing data.  Ditto Einstein's explanation of the photoelectric effect, discovered in the same year.  

I assume you mean, by "stamp collectors", people on the biology/chemistry/materials science side of things, rather than on the math/theoretical physics side of things, and by "extraordinary claims" you mean something along the lines of "claims that a specific simple model makes good predictions in a wide variety of circumstances", and by "ordinary evidence" you mean something along the lines of "some local pieces of evidence from one or a few specific experiments". So with that in mind:

1. Biology:
   1. Cell theory ("If you look at a tissue sample from a macroscopic organism, it will be made of cells.")
   2. Homeostasis ("If you change the exterior environment of an organism, its responses will tend to keep its internal state within a certain range in terms of e.g. temperature, salinity, pH, etc).
   3. DNA->RNA->protein pipeline ("If you look at an organism's DNA, you can predict the order of the amino acid residues in the proteins it expresses,, and every organism uses pretty much the same codon table which is blah blah")
2. Chemistry:
   1. Acid-base chemistry 
   2. Bond geometry and its relation to orbitals (e.g. "bond angles will tend to be ~109º for things attached to a carbon that has only single bonds, because that's the angle that two vertices of a tetrahedron make across the center").
   3. Bond energy (i.e. "you can predict pretty well how much energy a given reaction will produce just by summing the bond energy of each individual bond before and after")
   4. Resonance/delocalization
   5. Law of Mass Action: (i.e. "for every chemical reaction, there is an equilibrium ratio of reactants to products at a constant temperature. That equilibrium is computable based on the number of molecules in the reactants and products, and the energy contained within those molecules")
   6. For organic chemistry, literally hundreds of "if you put a molecule with this specific structure in with these specific reagents in these specific conditions, you will get a molecule that is transformed in this one specific way with no other important changes". For a concrete example: if you have a Grignard Reagent RMgX, and an aldehyde R'HO, you can combine them to form R-CH(OH)-R'. Individually, these "laws" are perhaps not so satisfying, but in combination they say "for pretty much any organic compound, you can synthesize that compound from relatively cheap inputs by using some combination of these reactions".
3. Misc other fields
   1. The photovoltaic effect, demonstrated in 1839, and its relation to the band gap -- the fact that some materials have energy levels that are "forbidden" to electrons led to unexplained empirical observations all the way back in 1839, and understanding the phenomenon (and tinkering a whole bunch, because analytical and computational methods don't even come close to being good enough) paved the way to the information age.
   2. Fourier Transforms aren't directly a physical phenomenon, but the fact that you can convert a series of values of any complex periodic system down into a sum of simple sine waves, knowing only the input frequencies but not the input amplitudes, meant that you could e.g. mechanically predict the future tides for a location based only on the past tides for that location.

I'm not so sure how well these examples will demonstrate that "collecting buckets of examples is not as useful as being able to deeply interpret and explain the examples that you have", but also I'm pretty sure that's just false a lot of the time -- you may have a deep theory of everything which is in principle sufficient, but that doesn't mean your deep theory of everything is computationally tractable for solving the specific problem you have in front of you.

Probably has something to do with the fact that a catastrophe is an event, and safety is an absence of something. It's just inherently harder to point at a thing and say that it caused fewer catastrophes to happen. Show me the non-catastrophes. Bring them to me, put them on my table. You can't do it.

Some biotech contributes to existential risk but others doesn't. A lot of vaccine technology doesn't increase existential risk but reduces it because of reduced danger from viruses. Phage therapy is the same for reducing the risk from infectious bacteria.

LessWrong itself is a form of technology that's intended to lead to existential risk reduction by facillitating a knowledge community to exist that otherwise wouldn't. 

The general idea of CFAR is that social technology they developed like double crux helps people to think more clearly and thus reduce existential risk. 

Yes, but the hardest part of that turns out to be generating high quality model alternatives under limited compute lmao

Yes, but doing that correctly requires a lot of preparation.

Well, yes, but nobody's going to get paid (in status) by putting it that way. It needs to be much more obfuscated and have a lot of math that requires mental contortions to apply to actual human choices.

More seriously, yes, but most of those words need multiple books worth of exploration to fully understand.  "basically just": what are the limits, edge cases, and applicability of exceptions.  "comparing": on what dimensions, and how to handle uncertainty that often overwhelms the simple calculations.  "alternatives": there are a near-infinite number, how to discover and filter the ones worth thinking about.  "picking": what does this even mean? how does choice work?  "best": what does THIS even mean?  How would one know if an alternative is better than another?

Oh, and I guess "right?": what would "wrong" look like, and how would you know which it is?

Not quite, in my opinion. In practice, humans tend to be wrong in predictable ways (what we call a "bias") and so picking the best option isn't easy.

What we call "rationality" tends to be the techniques / thought patterns that make us more likely to pick the best option when comparing alternatives.

Yes, this seems straightforwardly true, although I don't think it's especially significant unless I'm failing to think of some relevant context about why you think indexical claims matter so much (but then I don't spend a lot of time thinking very hard about semantics in a formal context, so maybe I'm just failing to grasp what all is encompassed by "indexical").

Hmm. It appears to me that Qualia are whatever observations affect indexical claims, and anything that affects indexical claims is a qualia

I don't think so, here is a counter-example:

Alice and Bob start talking in a room. Alice has an identical twin, Alex. Bob doesn't know about the twin and thinks he's talking to Alex. Bob asks: "How are you today?". Before Alice responds, Alex walks in.

Bob's observation of Alex will surprise him, and he'll quickly figure out that something's going on. But more importantly: Bob's observation of Alex alters the indexical 'you' in "How are you today?" (at least compared to Bob's intent, and it might change for Alice if she realises Bob was mistaken, too).

I don't think this is anything close to describing qualia. The experience of surprise can be a quale, the feeling of discovering something can be a quale (eureka moments), the experience of the colour blue is a quale, but the observation of Alex is not.

Do you agree with this? (It's from https://plato.stanford.edu/entries/indexicals/)

An indexical is, roughly speaking, a linguistic expression whose reference can shift from context to context. For example, the indexical ‘you’ may refer to one person in one context and to another person in another context.

Btw, 'qualia' is the plural form of 'quale'

I can see how this might result from confusing consciousness qua phenomenonality with consciousness qua personal identity.

Qualia are whatever observations affect indexical claims

Why would that be the case?

Can you expand on what you mean by "demonic"?  Is it a shorthand for "indicative of broken cognition, because it's both cruel and unnecessary", or something else?  I THINK what you're wondering about is whether these techniques/behaviors are ever actually optimal when dealing with misaligned agents who you nonetheless consider to be moral patients.  Is that close?

I think that both questions are related to uncertainty about the other agent(s).  Bargaining implies costly changes to future behaviors (of both parties).  Which makes signaling of capability and willingness important.  Bargainers need to signal that they will change something in a meaningful way based on whatever agreement/concession is reached.  In repeated interaction (which is almost all of them), actual follow-through is the strongest signal.

So, actual torture is the strongest signal of willingness and ability to torture.  Building a torturizer shows capability, but only hints at willingness.  Having materials that could build a torturizer or an orgasmatron is pretty weak, but not zero.  Likewise with strength and wealth - it's shows capability of benefit/reduced-harm from cooperation, which is an important prerequisite.

I don't think you can assert that threats are never carried out, unless you somehow have perfect mutual knowledge (and then, it's not bargaining, it's just optimization). Thomas Schelling won a Nobel for his work in bargaining under uncertainty, and I think most of those calculations are valid, no matter how adavnced and rational the involved agents are, when their knowledge is incomplete and they're misaligned in their goals.

Fun factoid: it is claimed that some South American apes used fire before human came and kill them: https://evolbiol.ru/document/915 

That sounds like punishing any usage of common resources which is likely undesireable. 

Good policy to manage individual commons requires to think through how their usage is best managed. Elinor Ostrom did a lot of research into what works for setting up good systems.

Sounds like auctioning the usage of the common.

I can imagine a few technical problems, like determining what level of usage is optimal (you don't want people to overfish the lake, but you don't know exactly how many fish are there), or the costs of policing. But it would be possible to propose a few dozen situations where this strategy could be used, and address these issues individually; and then perhaps only use the strategy in some of them. Or perhaps by examining individual specific cases, we would discover a common pattern why this doesn't work.

I think that's happened with a friend of mine who went into social work. They are in a field where that makes sense.