Dominance is (a certain kind of) nonlinearity on a single locus, epistasis is nonlinearity across different loci.

I feel like for observables it's more intuitive for them to be (0, 2) tensors (bilinear forms) whereas for density matrices it's more intuitive for them to be (2, 0) tensors. But maybe I'm missing something about the math that makes this problematic, since I haven't done many quantum calculations.

The way I (computer scientist who dabbles in physics, so YMMV I might be wrong) understand the physics here:

• Feynmann diagrams are basically a Taylor expansion of a physical system in terms of the strength of some interaction,
• To avoid using these Taylor expansions for everything, one tries to modify the parameters of the model to take a summary of the effects into account; for instance one distinguishes between the "bare mass", which doesn't take various interactions into account, versus the "effective mass", which does,
• Sometimes e.g. the Taylor series don't converge (or some integrals people derived from the Taylor expansions don't converge), but you know what the summary parameters turn out to be in the real world, and so you can just pretend the calculations do converge into whatever gives the right summary parameters (which makes sense if we understand the model is just an approximation given what's known and at some point the model breaks down).

Meanwhile, for ML:

• Causal scrubbing is pretty related to Taylor expansions, which makes it pretty related to Feynmann diagrams,
• However, it lacks any model for the non-interaction/non-Taylor-expanded effects, and so there's no parameters that these Taylor expansions can be "absorbed into",
• While Taylor expansions can obviously provide infinite detail, nobody has yet produced any calculations for causal scrubbing that fail to converge rather than simply being unreasonably complicated. This is partly because without the model above, there's not many calculations that are worth running.

I've been thinking about various ideas for Taylor expansions and approximations for neural networks, but I kept running in circles, and the main issue I've ended up with is this:

In order to eliminate noise, we need to decide what really matters and what doesn't really matter. However, purely from within the network, we have no principled way of doing so. The closest we get is what affects the token predictions for the network, but even that contains too may unimportant parameters, because if e.g. the network goes off on a tangent but then returns to the main topic, maybe that tangent didn't matter and we're fine with the approximation discarding it.

As a simplified version of this objection, consider that the token probabilities are not the final output of the network, but instead the tokens are sampled and fed back into the network, which means that really the final layer of the network is connected back to the first layer through a non-differentiable function. (The non-differentiability interferes with any interpretability method based on derivatives....)

What we really want to know is the impacts of the network in real-world scenarios, but it's hard to notice main consequences of the network, and even if we could, it's hard to set up measurable toy models of them. Once we had such toy models, it's unclear whether we'd even need elaborate techniques for interpreting them. If for instance Claude is breaking a generation of young nerds by praising any nonsensical thing they say by responding "Very insightful!", that doesn't really need any advanced interpretability techniques to be understood.

Oh I meant a (2, 0) tensor.

Why view the density matrix as an operator instead of as a tensor? Like I think of it as kinda similar to a covariance matrix (except not mean-standardized and also with a separate dimension for each configuration instead of being restricted to one dimension for each way the configurations could vary), with the ordinary quantum states being kinda similar to mean vectors.

Observation: for a variable that is positive and often close to zero (e.g. approximately lognormal), the L infty minimizer would be half of the maximum, which is isomorphic to literally just using the maximum. This seems like the quantitative version of focusing terrorism discussions on 9/11 and focusing wealth inequaility discussions on Elon Musk and so on.

I don't think consumers demand authentic AI friends because they already have authentic human friends. Also it's not clear how you imagine the AI companies could train the AIs to be more independent and less superficial; generally training an AI requires a differentiable loss, but human independence does not originate from a differentiable loss and so it's not obvious that one could come up with something functionally similar via gradient descent.

On straightforward extrapolation of current technologies, it kind of seems like AI friends would be overly pliable and lack independent lives. One could obviously train an AI to seem more independent to their "friends", and that would probably make it more interesting to "befriend", but in reality it would make the AI less independent because its supposed "independence" would actually arise from a constraint generated by its "friends"'s perception, rather than from an attempt to live independently. This seems less like a normal friendship and more like a superstimulus simulating the appearance of a friendship for entertainment value. It seems reasonable enough to characterize it as non-authentic.

Do you disagree? What do you think would lead to a different trajectory?

I'm not really suggesting something as convoluted as Hitler killing someone much worse than him. More like, maybe Elon Musk started supporting Republicans because he learned something very bad about Democrats and maybe eventually he's going to realize Republicans have something very bad too and then maybe he does a project that solves both bad things at once.

This seems to require working tightly enough with Republicans for long enough to understand why they are so bad, so it could be compared to Elon Musk working on EVs to understand how to scale up EV production.

What does it mean to assess it at any point, as distinct from in the long run? And was he really ever good for humanity if assessed through your one-point method? (E.g. climate impacts seems intrinsically a long-run thing...)

It's impossible to know until he is done/defeated, because the things he experiences due to his actions now could cause huge swings in his impacts on the future.

Very powerful AIs may very well be created in order to defend the current capitalist system.

Like the most plausible proposal for what distinguishes bounded tool AI vs dangerous AI is that dangerous AI does adversarial/minimax-like reasoning whereas bounded tool AI mostly just assumes the world will allow it to do whatever it tries, so it doesn't need to try very hard.

This means the main people who will be forced to create dangerous AI are the ones working in hardcore adversarial contexts, which will especially be the military and police (as well as their opponents, including rogue states and gangsters). But the military and police have as their primary goal to maintain the current system.

Wait, no.

The obvious objection to my comment would be, what if people who are really obese are obese for different reasons than the reason obesity has increased over time? (With the latter being what I assume jimrandomh is trying to figure out.)

I had thought of that counter but dimissed it because, AFAIK the rate of severe obesity has also increased a lot over time. So it seems like severe obesity would have the same cause as the increase over time.

But, we could imagine something like, contaminant -> increase in moderate obesity -> societal adjustment to make obesity more feasible (e.g. mobility scooters) -> increase in severe obesity.

Wouldn't it be much cheaper and easier to take a handful of really obese people, sample from the various things they eat, and look for contaminants?

Since it's the FDA that's doing the regulating, they could pick the investigator. Completely ungameable.

The best way is probably to have an excellent investigator rank the research subjects by their quality of life. If you've got a good idea about what a high-quality life is, you could probably do the ranking of them yourself.

The reason I suggest making it filter-in is because it seems to me that it's easier to make a meaningful filter that accurately detects a lot of sensitive stuff than a filter that accurately detects spam, because "spam" is kind of open-ended. Or I guess in practice spam tends to be porn bots and crypto scams? (Even on LessWrong?!) But e.g. truly sensitive talk seems disproportionately likely to involve cryptography and/or sexuality, so trying to filter for porn bots and crypto scams seems relatively likely to have reveal sensitive stuff.

The filter-in vs filter-out in my proposal is not so much about the degree of visibility. Like you could guard my filter-out proposal with the other filter-in proposals, like to only show metadata and only inspect suspected spammers, rather than making it available for everyone.

I would distinguish two variants of this. There's just plain inertia, like if you have a big pile of legacy code that accumulated from a lot of work, then it takes a commensurate amount of work to change it. And then there's security, like a society needs rules to maintain itself against hostile forces. The former is sort of accidentally surreal, whereas the latter is somewhat intentionally so, in that a tendency to re-adapt would be a vulnerability.

I wonder if you could also do something like, have an LLM evaluate whether a message contains especially-private information (not sure what that would be... gossip/reputationally-charged stuff? sexually explicit stuff? planning rebellions? doxxable stuff?), and hide those messages while looking at other ones.

Though maybe that's unhelpful because spambot authors would just create messages that trigger these filters?

I had at times experimented with making LLM commentators/agents, but I kind of feel like LLMs are always (nearly) "in equillibrium", and so your comments end up too dependent on the context and too unable to contribute with anything other than factual knowledge. It's cute to see your response to this post, but ultimately I expect that LessWrong will be best off without LLMs, at least for the foreseeable future.

On the object level, this is a study about personality, and it majorly changed the way I view some personality traits:

• I now see conservatism/progressivism as one of the main axes of personality,
• It further cemented my perception that "well-being", or "extraversion minus neuroticism", is the strongest of the traditional personality dimensions, and that maybe also this raises questions about what personality even means (for instance, surely well-being is not simply a biological trait),
• I'm now much more skeptical about how "real" many personality traits are, including traits like "compassion" that were previously quite central to my models of personality.

I think my study on the EQ-SQ model follows in the footsteps of this, rethinking much of what I thought I knew about differential psychology.

However, I actually view the fundamental contribution of the post quite different from this. Really, I'm trying to articulate and test theories of personality, as well as perform exploratory analyses, and I hope that I will inspire others to do so, as well as that I will become better at doing so over time. If this interests you, I would suggest you join Rationalist Psychometrics, a small discord server for this general topic.

In terms of methodology, this study is heavily focused on factor analysis. At the time of writing the post, I thought factor analysis was awesome and underrated. I still think it's great for testing the sorts of theories discussed in the post, and since such theories take up a lot of space in certain groups' discussion of differential psychology, I still think factor analysis is quite underrated.

But factor analysis is not everything. My current special interest is Linear Diffusion of Sparse Lognormals, which promises to do much better than factor analysis ... if I can get it to work. As such, while the post (and psychometrics in general) focuses quite heavily on factor analysis, I cannot wholeheartedly endorse that aspect of the post.

Prediction markets are good at eliciting information that correlates with what will be revealed in the future, but they treat each piece of information independently. Latent variables are a well-established method of handling low-rank connections between information, and I think this post does a good job of explaining why we might want to use that, as well as how we might want to implement them in prediction markets.

Of course the post is probably not entirely perfect. Already shortly after I wrote it, I switched from leaning towards IRT to leaning towards LCA, as you can see in the comments. I think it's best to think of the post as staking out a general shape for the idea, and then as one goes to implementing it, one can adjust the details based on what seems to work the best.

Overall though, I'm now somewhat less excited about LVPMs than I was at the time of writing it, but this is mainly because I now disagree with Bayesianism and doubt the value of eliciting information per se. I suspect that the discourse mechanism we need is not something for predicting the future, but rather for attributing outcomes to root causes. See Linear Diffusion of Sparse Lognormals for a partial attempt at explaining this.

Insofar as rationalists are going to keep going with the Bayesian spiral, I think LVPMs are the major next step. Even if it's not going to be the revolutionary method I assumed it would be, I would still be quite interested to see what happens if this ever gets implemented.

How about geology, ecology and history? It seems like you are focused on mechanisms rather than contents.

That said, I'm using "quantum mechanics" to mean "some generalization of the standard model" in many places.

I think this still has the ambiguity that I am complaining about.

As an analogy, consider the distinction between:

• Some population of rabbits that is growing over time due to reproduction
• The Fibonacci sequence as a model of the growth dynamics of this population
• A computer program computing or mathematician deriving the numbers in or properties of this sequence

The first item in this list is meant to be analogous to the quantum mechanics qua the universe, as in it is some real-world entity that one might hypothesize acts according to certain rules, but exists regardless. The second is a Platonic mathematical object that one might hypothesize matches the rules of the real-world entity. And the third are actual instantiations of this Platonic mathematical object in reality. I would maybe call these "the territory", "the hypothetical map" and "the actual map", respectively.

In practice, the actual experimental predictions of the standard model are something like probability distributions over the starting and ending momentum states of particles before and after they interact at the same place at the same time, so I don't think you can actually run a raw standard model simulation of the solar system which makes sense at all. To make my argument more explicit, I think you could run a lattice simulation of the solar system far above the Planck scale and full of classical particles (with proper masses and proper charges under the standard model) which all interact via general relativity, so at each time slice you move each particle to a new lattice site based on its classical momentum and the gravitational field in the previous time slice. Then you run the standard model at each lattice site which has more than one particle on it to destroy all of the input particles and generate a new set of particles according to the probabilistic predictions of the standard model, and the identities and momenta of the output particles according to a sample of that probability distribution will be applied in the next time slice. I might be making an obvious particle physics mistake, but modulo my own carelessness, almost all lattice sites would have nothing on them, many would have photons, some would have three quarks, fewer would have an electron on them, and some tiny, tiny fraction would have anything else. If you interpreted sets of sites containing the right number of up and down quarks as nucleons, interpreted those nucleons as atoms, used nearby electrons to recognize molecules, interpreted those molecules as objects or substances doing whatever they do in higher levels of abstraction, and sort of ignored anything else until it reached a stable state, then I think you would get a familiar world out of it if you had the utterly unobtainable computing power to do so.

Wouldn't this fail for metals, quantum computing, the double slit experiment, etc.? By switching back and forth between quantum and classical, it seems like you forbid any superpositions/entanglement/etc. on a scale larger than your classical lattice size. The standard LessWrongian approach is to just bite the bullet on the many worlds interpretation (which I have some philosophical quibbles with, but those quibbles aren't so relevant to this discussion, I think, so I'm willing to grant the many worlds interpretation if you want).

Anyway, more to the point, this clearly cannot be done with the actual map, and the hypothetical map does not actually exist, so my position is that while this may help one understand the notion that there is an rule that perfectly constrains  the world, the thought experiment does not actually work out.

Somewhat adjacently, your approach to this is reductionistic, viewing large entities as being composed of unfathomably many small entities. As part of LDSL I'm trying to wean myself off of reductionism, and instead take large entities to be more fundamental, and treat small entities as something that the large entities can be broken up into.

This is tangential to what I'm saying, but it points at something that inspired me to write this post. Eliezer Yudkowsky says things like the universe is just quarks, and people say "ah, but this one detail of the quark model is wrong/incomplete" as if it changes his argument when it doesn't. His point, so far as I understand it, is that the universe runs on a single layer somewhere, and higher-level abstractions are useful to the extent that they reflect reality. Maybe you change your theories later so that you need to replace all of his "quark" and "quantum mechanics" words with something else, but the point still stands about the relationship between higher-level abstractions and reality.

My in-depth response to the rationalist-reductionist-empiricist worldview is Linear Diffusion of Sparse Lognormals. Though there's still some parts of it I need to write. The main objection I have here is that "single layer" is not so much the true rules of reality so much as it is the subset of rules that are unobjectionable due to applying everywhere and every time. It's like the minimal conceivable set of rules.

The point of my quantum mechanics model is not to model the world, it is to model the rules of reality which the world runs on.

I'd argue the practical rules of the world are determined not just by the idealized rules, but also by the big entities within the world. The simplest example is outer space; it acts as a negentropy source and is the reason we can assume that e.g. electrons go into the lowest orbitals (whereas if e.g. outer space was full of hydrogen, it would undergo fusion, bombard us with light, and turn the earth into a plasma instead). More elaborate examples would be e.g. atmospheric oxygen, whose strong reactivity leads to a lot of chemical reactions, or even e.g. thinking of people as economic agents means that economic trade opportunities get exploited.

It's sort of conceivable that quantum mechanics describes the dynamics as a function of the big entities, but we only really have strong reasons to believe so with respect to the big entities we know about, rather than all big entities in general. (Maybe there are some entities that are sufficiently constant that they are ~impossible to observe.)

Quantum mechanics isn't computationally intractable, but making quantum mechanical systems at large scales is.

But in the context of your original post, everything you care about is large scale, and in particular the territory itself is large scale.

That is a statement about the amount of compute we have, not about quantum mechanics.

It's not a statement about quantum mechanics if you view quantum mechanics as a Platonic mathematical ideal, or if you use "quantum mechanics" to refer to the universe as it really is, but it is a statement about quantum mechanics if you view it as a collection of models that are actually used. Maybe we should have three different terms to distinguish the three?

Couldn't one say that a model is not truly a model unless it's instantiated in some cognitive/computational representation, and therefore since quantum mechanics is computationally intractable, it is actually quite far from being a complete model of the world? This would change it from being a map vs territory thing to more being a big vs precise Pareto frontier.

(Not sure if this is too tangential to what you're saying.)

This also kind of reveals why bad faith is so invalidating. If the regulatory commission can trust others to outsource its investigations, then it might be able to save resources. However, that mainly works if those others act in sufficiently good faith that they aren't a greater resource sink than investigating it directly and/or just steamrolling the others with a somewhat-flawed regulatory authority.

Neither full-contact psychoanalysis nor focusing on the object-level debate seems like a good way to proceed in the face of a regulatory commission. Instead, the regulatory commission should just spend its own resources checking what's true, and maybe ask the parties in the debate to account for their deviances from the regulatory commission's findings. Or if the regulatory commission is a sort of zombie commission that doesn't have the capacity to understand reality, each member in the conflict could do whatever rituals best manipulate the commission to their own benefit.

One thing to consider is that until you've got an end-to-end automation of basic human needs like farming, the existence of other humans remains a net benefit for you, both to maintain these needs and to incentivize others to share what they've done.

Automating this end-to-end is a major undertaking, and it's unclear whether LLMs are up to the task. If they aren't, it's possible we will return to a form of AI where classical alignment problems apply.

There might be humans who set it up in exchange for power/similar, and then it continues after they are gone (perhaps simply because it is "spaghetti code").

The presence of the regulations might also be forced by other factors, e.g. to suppress AI-powered frauds, gangsters, disinformation spreaders, etc..

Not if the regulation is sufficiently self-sustainably AI-run.

These aren't the only heavy tails, just the ones with highest potential to happen quickly. You could also have e.g. people regulating themselves to extinction.

I think this is a temporary situation because no sufficiently powerful entity has invested sufficiently much in AI-based defence. If this situation persists without any major shift in power for long enough, then it will be because the US and/or China have made an AI system to automatically suppress AI-powered gangs, and maybe also to automatically defend against AI-powered militaries. But the traditional alignment problem would to a great degree apply to such defensive systems.

She also frequently compared herself to Glaistig Uaine and Kyubey.

Reminder not to sell your soul(s) to the devil.

What I don't get is, why do you have this impulse to sanewash the sides in this discussion?

Is this someone who has a parasocial relationship with Vassar, or a more direct relationship? I was under the impression that the idea that Michael Vassar supports this sort of thing was a malicious lie spread by rationalist leaders in order to purge the Vassarites from the community. That seems more like something someone in a parasocial relationship would mimic than like something a core Vassarite would do.

I have been very critical of cover ups in lesswrong. I'm not going to name names and maybe you don't trust me. But I have observed this all directly. If you are let people toy with your brain while you are under the influence of psychedelics you should expect high odds of severe consequences. And your friends mental health might suffer as well.

I would highlight that the Vassarite's official stance is that privacy is a collusion mechanism created to protect misdoers, and so they can't consistently oppose you sharing what they know.

all FSAs are equivalent

??????

I think one underused trick for training LLMs is to explicitly "edit" them. That is, suppose they generate some text X in response to prompt Y, and it has some error or is missing something. In that case you can create a text X' that fixes this problem, and do a gradient update to increase log P(X'|Y)/P(X|Y).

For example, if we generate virtual comments in the style of certain LW users, one could either let those users browse the virtual comments that have been created in their style and correct them, or one could let the people who receive the virtual comments edit them to remove misunderstanding or similar.

If we think of the quantified abilities as the logarithms of the true abilities, then taking the log has likely massively increased the correlations by bringing the outliers into the bulk of the distribution.

Bayesianism was a mistake.

Your post is an excellent example of how the supposedly-reasonable middle ground tends to be so clueless as to be plausibly worse than the extremes.

Like, e.g. Blanchard doesn’t think trans men have AGP

You mean AAP here, right?

He accepts autohomoeroticism, which is close enough to AAP that the difference doesn't matter. The real problem here is Michael Bailey who has a sort of dogmatic denial of AAP.

doesn’t think trans women who are attracted to men have AGP

That's pretty common in people's second-hand version; the real issue here is that this is sometimes wrong and some androphiles are AGP.

Oversimplification 2: Bisexuals exist. Many trans women report their sexual orientation changing when they start taking hormones. The correlation between having AGP and being attracted to women can’t be as 100% as Blanchard appears to believe it is.

Blanchard explicitly measured that some trans women identified as bisexual, and argued that they were autogynephilic and not truly bisexual. There's some problems with that assertion, but uncovering those problems really ought to engage with more of the nuances than what you imply here.

Oversimplification 4: Do heterosexual cisgender women have AGP? (Cf. Comments by Aella, eigenrobot etc.) if straight cisgender women also like being attractive in the same way as (some) trans women do, it becomes somewhat doubtful that it’s a pathology.

According to qualitative studies I've done, around 15% of women are at least somewhat AGP (though I think it correlates with being bi/lesbian), but the assertion that this implies it's not a pathology for males seems like magical thinking. E.g. ~100% of women have breasts, but this does not mean that developing breasts would not be considered a pathology for males.

If you consider the "true ability" to be the exponential of the subtest scores, then the extent to which the problem I mention applies depends on the base of the exponential. In the limiting case where the base goes to infinity, only the highest ability matter, whereas in the limiting case where the base goes to 1, you end up with something basically linear.

As for whether it's a crux, approximately nobody has thought about this deeply enough that they would recognize it, but I think it's pretty foundational for a lot of disagreements about IQ.

The analogy that I'm objecting to is, if you looked at e.g. the total for a ledger or a budget, it is an index that sums together expenses in a much more straightforward way. For instance if there is a large expense, the total is large.

Meanwhile, IQ scores are more like the geometric mean of the entries on such an entity. The geometric mean tells you whether the individual items tend to be large or small, which gives you broad-hitting information that distinguishes e.g. people who live in high-income countries from people who live in low-income countries, or large organizations from individual people; but it won't inform you if someone got hit by a giant medical bill or if they managed to hack themselves to an ultra-cheap living space. These pretty much necessarily have to be low-rank mediators (like in the g model) rather than diverse aggregates (like in the sum model).

(Well, a complication in this analogy is that a ledger can vary not just in the magnitude of the transfers but also qualitatively in the kinds of transfers that are made, whereas IQ tests fix the variables, making it more analogous to a standardized budget form (e.g. for tax or loan purposes) broken down by stuff like "living space rent", "food", "healthcare", etc..)

That's part of the problem, often the bad middle ground looks superficially plausible, so it's very sticky and hard to get rid of, because it's not exactly that people get told the wrong things but rather that they spontaneously develop the wrong ideas.

The three basic issues with this viewpoint are:

1. IQ test batteries do not measure even close to all cognitive abilities and realistically could never do that.
2. Many of the abilities that IQ scores weight highly are practically unimportant.
3. Differential-psychology tests are in practice more like log scales than like linear scales, so "sums" are more like products than like actual suns; even if you are absurdly good at one thing, you're going to have a hard time competing with someone in IQ if they are moderately better at many things.

Ever since the situation with Blanchardianism, I've become extremely bearish on the possibility on this, considering how everyone on all sides including rationalists on all sides of the debate just massively failed on it.

With IQ realism, you also get insane stuff where the supposedly reasonable middle ground tends to have skepticism about the g factor and thinks of IQ as an index that sums together cognitive abilities.

I haven't thought of this in relation to wild animal welfare or birthrates but I don't immediately see the argument that we can outperform the abysmal track record seen in these two other cases.

A possible model is that while good startups have an elevation in the "cult-factor", they have an even greater elevation in the unique factor related to the product they are building. Like SpaceX has cult-like elements but SpaceX also has Mars and Mars is much bigger than the cult-like elements, so if we define a cult to require that the biggest thing going on for them is cultishness then SpaceX is not a cult.

This is justified by LDSL (I really should write up the post explaining it...).

I feel like one should use a different term than vitalism to describe the unpredictability, since Henri Bergson cane up with vitalism based on the idea that physics can make short-term predictions about the positions of things but that by understanding higher powers one can also learn to predict what kinds of life will emerge etc..

Like let's say you have a big pile of grain. A simple physical calculation can tell you that this pile will stay attached to the ground (gravity) and a more complex one can tell you that it will remain ~static for a while. But you can't use Newtonian mechanics, relativity, or quantum mechanics to predict the fact that it will likely grow moldy or get eaten by mice, even though that will also happen.

Singapore and the US both have a military, a police, and taxation. This seems much more clear-cut to me than "cults" do.

I think maybe one could treat "cult" more like a pronoun than like a theoretical object. Like when one is in the vicinity of one of the groups Ben Pace mentioned, it makes sense to have a short term to talk about the group, and "the cult" is useful for disambiguating the cult from other groups that might be present.

I like vaccines and suspect they (or antibiotics) account for the majority of the value provided by the medical system. I don't usually see discussion of what can be done to promote or improve vaccines, so I don't know much about it, but the important part is they remain available and get improved and promoted in whatever ways are reasonable.

Beyond that, a major health problem is obesity and here semaglutide seems like it would help a lot.

I think there's something to this. Also since making the OP, I've been thinking that human control of fire seems important. If trees have the majority of the biomass, but humans can burn the trees for energy or just to make space, then that also makes humans special (and overlaps a lot with what you say about energy controlled).

This also neatly connects human society to the evolutionary ecology since human dominance hierarchies determine who is able to control what energy (or set fire to what trees).