Why does generalization work? 2024-02-20T17:51:10.424Z
Natural abstractions are observer-dependent: a conversation with John Wentworth 2024-02-12T17:28:38.889Z
The lattice of partial updatelessness 2024-02-10T17:34:40.276Z
Updatelessness doesn't solve most problems 2024-02-08T17:30:11.266Z
Sources of evidence in Alignment 2023-07-02T20:38:34.089Z
Quantitative cruxes in Alignment 2023-07-02T20:38:18.534Z
Why are counterfactuals elusive? 2023-03-03T20:13:48.981Z
Martín Soto's Shortform 2023-02-11T23:38:29.999Z
The Alignment Problems 2023-01-12T22:29:26.515Z
Brute-forcing the universe: a non-standard shot at diamond alignment 2022-11-22T22:36:36.599Z
A short critique of Vanessa Kosoy's PreDCA 2022-11-13T16:00:45.834Z
Vanessa Kosoy's PreDCA, distilled 2022-11-12T11:38:12.657Z
Further considerations on the Evidentialist's Wager 2022-11-03T20:06:31.997Z
Enriching Youtube content recommendations 2022-09-27T16:54:41.958Z
An issue with MacAskill's Evidentialist's Wager 2022-09-21T22:02:47.920Z
General advice for transitioning into Theoretical AI Safety 2022-09-15T05:23:06.956Z
Alignment being impossible might be better than it being really difficult 2022-07-25T23:57:21.488Z
Which one of these two academic routes should I take to end up in AI Safety? 2022-07-03T01:05:23.956Z


Comment by Martín Soto (martinsq) on Martín Soto's Shortform · 2024-02-28T22:38:10.179Z · LW · GW

Marginally against legibilizing my own reasoning:     

When taking important decisions, I spend too much time writing down the many arguments, and legibilizing the whole process for myself. This is due to completionist tendencies. Unfortunately, a more legible process doesn’t overwhelmingly imply a better decision!

Scrutinizing your main arguments is necessary, although this looks more like intuitively assessing their robustness in concept-space than making straightforward calculations, given how many implicit assumptions they all have. I can fill in many boxes, and count and weigh considerations in-depth, but that’s not a strong signal, nor what almost ever ends up swaying me towards a decision!

Rather than folding, re-folding and re-playing all of these ideas inside myself, it’s way more effective time-wise to engage my System 1 more: intuitively assess the strength of different considerations, try to brainstorm new ways in which the hidden assumptions fail, try to spot the ways in which the information I’ve received is partial… And of course, share all of this with other minds, who are much more likely to update me than my own mind. All of this looks more like rapidly racing through intuitions than filling Excel sheets, or having overly detailed scoring systems.

For example, do I really think I can BOTEC the expected counterfactual value (IN FREAKING UTILONS) of a new job position? Of course a bad BOTEC is better than none, but the extent to which that is not how our reasoning works, and the work is not really done by the BOTEC at all, is astounding. Maybe at that point you should stop calling it a BOTEC.

Comment by Martín Soto (martinsq) on CFAR Takeaways: Andrew Critch · 2024-02-24T22:16:08.904Z · LW · GW

This is pure gold, thanks for sharing!

Comment by Martín Soto (martinsq) on Why does generalization work? · 2024-02-21T06:16:14.716Z · LW · GW

Didn't know about ruliad, thanks!

I think a central point here is that "what counts as an observer (an agent)" is observer-dependent (more here) (even if under our particular laws of physics there are some pressures towards agents having a certain shape, etc., more here). And then it's immediate each ruliad has an agent (for the right observer) (or similarly, for a certain decryption of it).

I'm not yet convinced "the mapping function/decryption might be so complex it doesn't fit our universe" is relevant. If you want to philosophically defend "functionalism with functions up to complexity C" instead of "functionalism", you can, but C starts seeming arbitrary?

Also, a Ramsey-theory argument would be very cool.

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-20T21:33:03.480Z · LW · GW

- Chan master Yunmon

Comment by Martín Soto (martinsq) on Why does generalization work? · 2024-02-20T21:29:14.710Z · LW · GW

Yep! Although I think the philosophical point goes deeper. The algorithm our brains themselves use to find a pattern is part of the picture. It is a kind of "fixed (de/)encryption".

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-20T20:39:23.519Z · LW · GW

Thank you, habryka!

As mentioned in my answer to Eliezer, my arguments were made with that correct version of updatelessness in mind (not "being scared to learn information", but "ex ante deciding whether to let this action depend on this information"), so they hold, according to me.
But it might be true I should have stressed this point more in the main text.

Comment by Martín Soto (martinsq) on The lattice of partial updatelessness · 2024-02-20T20:36:13.489Z · LW · GW

Yep! I hadn't included pure randomization in the formalism, but it can be done and will yield some interesting insights.

As you mention, we can also include pseudo-randomization. And taking these bounded rationality considerations into account also makes our reasoning richer and more complex: it's unclear exactly when an agent wants to obfuscate its reasoning from others, etc.

Comment by Martín Soto (martinsq) on The lattice of partial updatelessness · 2024-02-20T20:34:03.455Z · LW · GW

First off, that  was supposed to be , sorry.

The agent might commit to "only updating on those things accepted by program ", even when it still doesn't have the complete infinite list of "exactly in which things does  update" (in fact, this is always the case, since we can't hold an infinite list in our head). It will, at the time of committing, know that  updates on certain things, doesn't update on others... and it is uncertain about exactly what it does in all other situations. But that's okay, that's what we do all the time: decide on an endorsed deliberation mechanism based on its structural properties, without yet being completely sure of what it does (otherwise, we wouldn't need the deliberation). But it does advise against committing while being too ignorant.

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-20T20:29:30.067Z · LW · GW

Is it possible for all possible priors to converge on optimal behavior, even given unlimited observations?

Certainly not, in the most general case, as you correctly point out.

Here I was studying a particular case: updateless agents in a world remotely looking like the real world. And even more particular: thinking about the kinds of priors that superintelligences created in the real world might actually have.

Eliezer believes that, in these particular cases, it's very likely we will get optimal behavior (we won't get trapped priors, nor commitment races). I disagree, and that's what I argue in the post.

I'm also surprised that dynamic stability leads to suboptimal outcomes that are predictable in advance. Intuitively, it seems like this should never happen.

If by "predictable in advance" you mean "from the updateless agent's prior", then nope! Updatelessness maximizes EV from the prior, so it will do whatever looks best from this perspective. If that's what you want, then updatelessness is for you! The problem is, we have many pro tanto reasons to think this is not a good representation of rational decision-making in reality, nor the kind of cognition that survives for long in reality. Because of considerations about "the world being so complex that your prior will be missing a lot of stuff". And in particular, multi-agentic scenarios are something that makes this complexity sky-rocket.
Of course, you can say "but that consideration will also be included in your prior". And that does make the situation better. But eventually your prior needs to end. And I argue, that's much before you have all the necessary information to confidently commit to something forever (but other people might disagree with this).

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-20T20:23:09.050Z · LW · GW

Is this consistent with the way you're describing decision-making procedures as updateful and updateless?

Absolutely. A good implementation of UDT can, from its prior, decide on an updateful strategy. It's just it won't be able to change its mind about which updateful strategy seems best. See this comment for more.

"flinching away from true information"

As mentioned also in that comment, correct implementations of UDT don't actually flinch away from information: they just decide ex ante (when still not having access to that information) whether or not they will let their future actions depend on it.

The problem remains though: you make the ex ante call about which information to "decision-relevantly update on", and this can be a wrong call, and this creates commitment races, etc.

Comment by Martín Soto (martinsq) on Natural abstractions are observer-dependent: a conversation with John Wentworth · 2024-02-20T20:12:16.679Z · LW · GW

I'm not sure we are in disagreement. No one is negating that the territory shapes the maps (which are part of the territory). The central point is just that our perception of the territory is shaped by our perceptors, etc., and need not be the same. It is still conceivable that, due to how the territory shapes this process (due to the most likely perceptors to be found in evolved creatures, etc.), there ends up being a strong convergence so that all maps represent isomorphically certain territory properties. But this is not a given, and needs further argumentation. After all, it is conceivable for a territory to exist that incentivizes the creation of two very different and non-isomorphic types of maps. But of course, you can argue our territory is not such, by looking at its details.

Where “joint carvy-ness” will end up being, I suspect, related to “gears that move the world,” i.e., the bits of the territory that can do surprisingly much, have surprisingly much reach, etc.

I think this falls for the same circularity I point at in the post: you are defining "naturalness of a partition" as "usefulness to efficiently affect / control certain other partitions", so you already need to care about the latter. You could try to say something like "this one partition is useful for many partitions", but I think that's physically false, by combinatorics (in all cases you can always build as many partitions that are affected by another one). More on these philosophical subtleties here: Why does generalization work?

Comment by Martín Soto (martinsq) on OpenAI's Sora is an agent · 2024-02-16T19:12:15.980Z · LW · GW

Guy who reinvents predictive processing through Minecraft

Comment by Martín Soto (martinsq) on The Commitment Races problem · 2024-02-15T18:58:50.509Z · LW · GW

I agree most superintelligences won't do something which is simply "play the ordinal game" (it was just an illustrative example), and that a superintelligence can implement your proposal, and that it is conceivable most superintelligences implement something close enough to your proposal that they reach Pareto-optimality. What I'm missing is why that is likely.

Indeed, the normative intuition you are expressing (that your policy shouldn't in any case incentivize the opponent to be more sophisticated, etc.) is already a notion of fairness (although in the first meta-level, rather than object-level). And why should we expect most superintelligences to share it, given the dependence on early beliefs and other pro tanto normative intuitions (different from ex ante optimization)? Why should we expect this to be selected for? (Either inside a mind, or by external survival mechanisms)
Compare, especially, to a nascent superintelligence who believes most others might be simulating it and best-responding (thus wants to be stubborn). Why should we think this is unlikely?
Probably if I became convinced trapped priors are not a problem I would put much more probability on superintelligences eventually coordinating.

Another way to put it is: "Sucks to be them!" Yes sure, but also sucks to be me who lost the $1! And maybe sucks to be me who didn't do something super hawkish and got a couple other players to best-respond! While it is true these normative intuitions pull on me less than the one you express, why should I expect this to be the case for most superintelligences?

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-15T05:37:26.541Z · LW · GW

Thank you for engaging, Eliezer.

I completely agree with your point: an agent being updateless doesn't mean it won't learn new information. In fact, it might perfectly decide to "make my future action A depend on future information X", if the updateless prior so finds it optimal. While in other situations, when the updateless prior deems it net-negative (maybe due to other agents exploiting this future dependence), it won't.

This point is already observed in the post (see e.g. footnote 4), although without going deep into it, due to the post being meant for the layman (it is more deeply addressed, for example, in section 4.4 of my report). Also for illustrative purposes, in two places I have (maybe unfairly) caricaturized an updateless agent as being "scared" of learning more information. While really, what this means (as hopefully clear from earlier parts of the post) is "the updateless prior assessed whether it seemed net-positive to let future actions depend on future information, and decided no (for almost all actions)".

The problem I present is not "being scared of information", but the trade-off between "letting your future action depend on future information X" vs "not doing so" (and, in more detail, how exactly it should depend on such information). More dependence allows you to correctly best-respond in some situations, but also could sometimes get you exploited. The problem is there's no universal (belief-independent) rule to assess when to allow for dependence: different updateless priors will decide differently. And need to do so in advance of letting their deliberation depend on their interactions (they still don't know if that's net-positive).
Due to this prior-dependence, if different updateless agents have different beliefs, they might play very different policies, and miscoordinate. This is also analogous to different agents demanding different notions of fairness (more here). I have read no convincing arguments as to why most superintelligences will converge on beliefs (or notions of fairness) that successfully coordinate on Pareto optimality (especially in the face of the problem of trapped priors i.e. commitment races), and would be grateful if you could point me in their direction.

I interpret you as expressing a strong normative intuition in favor of ex ante optimization. I share this primitive intuition, and indeed it remains true that, if you have some prior and simply want to maximize its EV, updatelessness is exactly what you need. But I think we have discovered other pro tanto reasons against updatelessness, like updateless agents probably performing worse on average (in complex environments) due to trapped priors and increased miscoordination.

Comment by Martín Soto (martinsq) on The Commitment Races problem · 2024-02-15T05:27:58.880Z · LW · GW

The normative pull of your proposed procedure seems to come from a preconception that "the other player will probably best-respond to me" (and thus, my procedure is correctly shaping its incentives).

But instead we can consider the other player trying to get us to best-respond to them, by jumping up a meta-level: the player checks whether I am playing your proposed policy with a certain notion of fairness $X (which in your case is $5), and punishes accordingly to how far their notion of fairness $Y is from my $X, so that I (if I were to best-respond to his policy) would be incentivized to adopt notion of fairness $Y.

It seems clear that, for the exact same reason your argument might have some normative pull, this other argument has some normative pull in the opposite direction. It then becomes unclear which has stronger normative pull: trying to shape the incentives of the other (because you think they might play a policy one level of sophistication below yours), or trying to best-respond to the other (because you think they might play a policy one level of sophistication above yours).

I think this is exactly the deep problem, the fundamental trade-off, that agents face in both empirical and logical bargaining. I am not convinced all superintelligences will resolve this trade-off in similar enough ways to allow for Pareto-optimality (instead of falling for trapped priors i.e. commitment races), due to the resolution's dependence on the superintelligences' early prior.

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-13T03:18:02.406Z · LW · GW

(Sorry, short on time now, but we can discuss in-person and maybe I'll come back here to write the take-away)

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-11T04:25:57.226Z · LW · GW

To me it feels like the natural place to draw the line is update-on-computations but updateless-on-observations.

A first problem with this is that there is no sharp distinction between purely computational (analytic) information/observations and purely empirical (synthetic) information/observations. This is a deep philosophical point, well-known in the analytic philosophy literature, and best represented by Quine's Two dogmas of empiricism, and his idea of the "Web of Belief". (This is also related to Radical Probabilisim.)
But it's unclear if this philosophical problem translates to a pragmatic one. So let's just assume that the laws of physics are such that all superintelligences we care about converge on the same classification of computational vs empirical information.

A second and more worrying problem is that, even given such convergence, it's not clear all other agents will decide to forego the possible apparent benefits of logical exploitation. It's a kind of Nash equilibrium selection problem: If I was very sure all other agents forego them (and have robust cooperation mechanisms that deter exploitation), then I would just do like them. And indeed, it's conceivable that our laws of physics (and algorithmics) are such that this is the case, and all superintelligences converge on the Schelling point of "never exploiting the learning of logical information". But my probability of that is not very high, especially due to worries that different superintelligences might start with pretty different priors, and make commitments early on (before all posteriors have had time to converge). (That said, my probability is high that almost all deliberation is mostly safe, by more contingent reasons related to the heuristics they use and values they have.)
You might also want to say something like "they should just use the correct decision theory to converge on the nicest Nash equilibrium!". But that's question-begging, because the worry is exactly that others might have different notions of this normative "nice" (indeed, no objective criterion for decision theory). The problem recurs: we can't just invoke a decision theory to decide on the correct decision theory.

Am I missing something about why logical counterfactual muggings are likely to be common?

As mentioned in the post, Counterfactual Mugging as presented won't be common, but equivalent situations in multi-agentic bargaining might, due to (the naive application of) some priors leading to commitment races. (And here "naive" doesn't mean "shooting yourself in the foot", but rather "doing what looks best from the prior", even if unbeknownst to you it has dangerous consequences.)

if it comes up it seems that an agent that updates on computations can use some precommitment mechanism to take advantage of it

It's not looking like something as simple as that will solve, because of reasoning as in this paragraph:

Unfortunately, it’s not that easy, and the problem recurs at a higher level: your procedure to decide which information to use will depend on all the information, and so you will already lose strategicness. Or, if it doesn’t depend, then you are just being updateless, not using the information in any way.

Or in other words, you need to decide on the precommitment ex ante, when you still haven't thought much about anything, so your precommitment might be bad.
(Although to be fair there are ongoing discussions about this.)

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-09T22:24:52.538Z · LW · GW

It seems like we should be able to design software systems that are immune to any infohazard

As mentioned in another comment, I think this is not possible to solve in full generality (meaning, for all priors), because that requires complete updatelessness and we don't want to do that.

I think all your proposed approaches are equivalent (and I think the most intuitive framing is "cognitive sandboxes"). And I think they don't work, because of reasoning close to this paragraph:

Unfortunately, it’s not that easy, and the problem recurs at a higher level: your procedure to decide which information to use will depend on all the information, and so you will already lose strategicness. Or, if it doesn’t depend, then you are just being updateless, not using the information in any way.

But again, the problem might be solvable in particular cases (like, our prior).

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-09T22:19:16.547Z · LW · GW

The motivating principle is to treat one's choice of decision theory as itself strategic.

I share the intuition that this lens is important. Indeed, there might be some important quantitative differences between
a) I have a well-defined decision theory, and am choosing how to build my successor
b) I'm doing some vague normative reasoning to choose a decision theory (like we're doing right now),
but I think these differences are mostly contingent, and the same fundamental dynamics about strategicness are at play in both scenarios.

Design your decision theory so that no information is hazardous to it

I think this is equivalent to your decision theory being dynamically stable (that is, its performance never improves by having access to commitments), and I'm pretty sure the only way to attain this is complete updatelessness (which is bad).

That said, again, it might perfectly be that given our prior, many parts of cooperation-relevant concept-space seem very safe to explore, and so "for all practical purposes" some decision procedures are basically completely safe, and we're able to use them to coordinate with all agents (even if we haven't "solved in all prior-independent generality" the fundamental trade-off between updatelessness and updatefulness).

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-09T02:05:25.477Z · LW · GW

I agree that the situation is less
"there is a theoretical problem which is solvable but our specification of Updatelessness is not solving"
and more
"there is a fundamental obstacle in game-theoretic interactions (at least the way we model them)".

Of course, even if this obstacle is "unavoidable in principle" (and no theoretical solution will get rid of it completely and for all situations), there are many pragmatic and realistic solutions (partly overfit to the situation we already know we are actually in) that can improve interactions. So much so as to conceivably even dissolve the problem into near-nothingness (although I'm not sure I'm that optimistic).

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-09T02:01:28.314Z · LW · GW

You're right that (a priori and on the abstract) "bargaining power" fundamentally trades off against "best-responding". That's exactly the point of my post. This doesn't prohibit, though, that a lot of pragmatic and realistic improvements are possible (because we know agents in our reality tend to think like this or like that), even if the theoretical trade-off can never be erased completely or in all situations and for all priors.

Your latter discussion is a normative one. And while I share your normative intuitions that best-responding completely (being completely updateful) is not always the best to do in realistic situations, I do have quibbles with this kind of discourse (similar to this). For example, why would I want to go Straight even after I have learned the other does? Out of some terminal valuation of fairness, or counterfactuals, more than anything, I think (more here). Or similarly, why should I think sticking to my notion of fairness shall ex ante convince the other player to coordinate on it, as opposed to the other player trying to pull out some "even more meta" move, like punishing notions of fairness that are not close enough to theirs? Again, all of this will depend on our priors.

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-09T00:36:49.817Z · LW · GW

This is exactly the kind of procedure which might get hindered by commitment races, because it involves "thinking more about what the other agents will do", and the point of commitment races is that sometimes (and depending on your beliefs) this can seem net-negative ex ante (that is, before actually doing the thinking).

Of course, this doesn't prohibit logical handshakes from being enacted sometimes. For example, if all agents start with a high enough prior on others enacting their part of , then they will do it. More realistically, it probably won't be as easy as this, but if it is the case that all agents feel safe enough thinking about  (they deem it unlikely this backfires into losing bargaining power), and/or the upshot is sufficiently high (when multiplied by the probability and so on), then all agents will deem it net-positive to think more about  and the others, and eventually they'll implement it.

So it comes down to how likely we think are priors (or the equivalent thing for AIs) which successfully fall into this coordination basin, opposed to priors which get stuck in some earlier prior without wanting to think more. And again, we have a few pro tanto reasons to expect coordination to be viable (and a few in the other direction). I do think out of my list of statements, logical handshakes in causal interactions might be one of the most likely ones.

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-08T19:41:14.596Z · LW · GW

Another coarse, on-priors consideration that I could have added to the "Other lenses" section:

Eliezer says something like "surely superintelligences will be intelligent enough to coordinate on Pareto-optimality (and not fall into something like commitment races), and easily enact logical or value handshakes". But here's why I think this outside-view consideration need not hold. It is a generally good heuristic to think superintelligences will be able to solve tasks that seem impossible to us. But I think this stops being the case for tasks whose difficulty / complexity grows with the size / computational power / intelligence level of the superintelligence. For a task like "beating a human at Go" or "turning the solar system into computronium", the difficulty of the task is constant (relative to the size of the superintelligence you're using to solve it). For a task like "beat a copy of yourself in Go", that's clearly not the case (well, unless Go has a winning strategy that a program within our universe can enact, which would be a ceiling on difficulty). I claim "ensuring Pareto-optimality" is more like the latter. When the intelligence or compute of all players grows, it is true they can find more clever and sure-fire ways to coordinate robustly, but it's also true that they can individually find more clever ways of tricking the system and getting a bit more of the pie (and in some situations, they are individually incentivized to do this). Of course, one might still hold that the first will grow much more than the latter, and so after a certain level of intelligence, agents of a similar intelligence level will easily coordinate. But that's an additional assumption, relative to the "constant-difficulty" cases.

Of course, if Eliezer believes this it is not really because of outside-view considerations like the above, but because of inside-views about decision theory. But I generally disagree with his takes there (for example here), and have never found convincing arguments (from him or anyone) for the easy coordination of superintelligences.

Comment by Martín Soto (martinsq) on Updatelessness doesn't solve most problems · 2024-02-08T19:24:48.929Z · LW · GW

You're right there's something weird going on with fix-points and determinism: both agents are just an algorithm, and in some sense there is already a mathematical fact of the matter about what each outputs. The problem is none of them know this in advance (exactly because of the non-terminating computation problem), and so (while still reasoning about which action to output) they are logically uncertain about what they and the other outputs.

If an agent believes that the others' action is completely independent of their own, then surely, no commitment race will ensue. But say, for example, they believe their taking action A makes it more likely the other takes action B. This belief could be justified in a number of different ways: because they believe the other to be perfectly simulating them, because they believe the other to be imperfectly simulating them (and notice, both agents can imperfectly simulate each other, and consider this to give them better-than-chance knowledge about the other), because they believe they can influence the truth of some mathematical statements (EDT-like) that the other will think about, etc.

And furthermore, this doesn't solely apply to the end actions they choose: it can also apply to the mental moves they perform before coming to those actions. For example, maybe an agent has a high enough probability on "the other will just simulate me, and best-respond" (and thus, I should just be aggressive). But also, an agent could go one level higher, and think "if I simulate the other, they will probably notice (for example, by coarsely simulating me, or noticing some properties of my code), and be aggressive. So I won't do that (and then it's less likely they're aggressive)".

Another way to put all this is that one of them can go "first" in logical time (at the cost of having thought less about the details of their strategy).

Of course, we have some reasons to think the priors needed for the above to happen are especially wacky, and so unlikely. But again, one worry is that this could happen pretty early on, when the AGI still has such wacky and unjustified beliefs.

Comment by Martín Soto (martinsq) on A Shutdown Problem Proposal · 2024-01-22T11:19:12.741Z · LW · GW

Brain-storming fixes:

  • Each subagent's bargaining power is how much compute they can use. This makes everything more chaotic, and is clearly not what you had in mind with this kind of idealized agents solution.
  • Probabilistic vetos, such that those of some subagents are less likely to work. I think this breaks things in your proposal and still has the game-theoretic problems.
  • We ensure the priors of each subagent (about how the others respond) are such that going for risky game-theoretic stuff is not individually rational. Maybe some agents have more optimistic priors, and others less optimistic, and this results in the former controlling more, and the latter only try to use their veto in extreme cases (like to ensure the wrong successor is not built). But it'd be fiddly to think about the effect of these different priors on behavior, and how "extreme" the cases are in which veto is useful. And also this might mess up the agent's interactions with the world in other ways: for example, dogmatically believing that algorithms that look like subagents have "exactly this behavior", which is sometimes false. Although of course this kind of problem was already present in your proposal.
Comment by Martín Soto (martinsq) on A Shutdown Problem Proposal · 2024-01-22T11:12:47.646Z · LW · GW

Then there’s the problem of designing the negotiation infrastructure, and in particular allocating bargaining power to the various subagents. They all get a veto, but that still leaves a lot of degrees of freedom in exactly how much the agent pursues the goals of each subagent. For the shutdown use-case, we probably want to allocate most of the bargaining power to the non-shutdown subagent, so that we can see what the system does when mostly optimizing for u_1 (while maintaining the option of shutting down later).

I don't understand what you mean by "allocating bargaining power", given already each agent has true veto power. Regardless of the negotiation mechanism you set up for them (if it's high-bandwidth enough), or whether the master agent says "I'd like this or that agent to have more power", each subagent could go "give me my proportional (1/n) part of the slice, or else I will veto everything" (and depending on its prior about how other agents could respond, this will seem net-positive to do).

In fact that's just the tip of the iceberg of individually rational game-theoretic stuff (that messes with your proposal) they could pull off, see Commitment Races.

Comment by Martín Soto (martinsq) on Martín Soto's Shortform · 2024-01-20T10:41:06.085Z · LW · GW

The Singularity

Why is a rock easier to predict than a block of GPUs computing? Because the block of GPUs is optimized so that its end-state depends on a lot of computation.
[Maybe by some metric of “good prediction” it wouldn’t be much harder, because “only a few bits change”, but we can easily make it the case that those bits get augmented to affect whatever metric we want.]
Since prediction is basically “replicating / approximating in my head the computation made by physics”, it’s to be expected that if there’s more computation that needs to be finely predicted, the task is more difficult.
In reality, there is (in the low level of quantum physics) as much total computation going on, but most of it (those lower levels) are screened off enough from macro behavior (in some circumstances) that we can use very accurate heuristics to ignore them, and go “the rock will not move”. This is purposefully subverted in the GPU case: to cram a lot of useful computation into a small amount of space and resources, the micro computations (at the level of circuitry) are orderly secured and augmented, instead of getting screened off due to chaos.

Say we define the Singularity as “when the amount of computation / gram of matter (say, on Earth) exceeds a certain threshold”. What’s so special about this? Well, exactly for the same reason as above, an increase in this amount makes the whole setup harder to predict. Some time before the threshold, maybe we can confidently predict some macro properties of Earth for the next 2 months. Some time after it, maybe we can barely predict that for 1 minute.

But why would we care about this change in speed? After all, for now (against the backdrop of real clock time in physics) it doesn’t really matter whether a change in human history takes 1 year or 1 minute to happen.
[In the future maybe it does start mattering because we want to cram in more utopia before heat death, or because of some other weird quirk of physics.]
What really matters is how far we can predict “in terms of changes”, not “in terms of absolute time”. Both before and after the Singularity, I might be able to predict what happens to humanity for the next X FLOP (of total cognitive labor employed by all humanity, including non-humans). And that’s really what I care about, if I want to steer the future. The Singularity just makes it so these FLOP happen faster. So why be worried? If I wasn’t worried before about not knowing what happens after X+1 FLOP, and I was content with doing my best at steering given that limited knowledge, why should that change now?
[Of course, an option is that you were already worried about X FLOP not being enough, even if the Singularity doesn’t worsen it.]

The obvious reason is changes in differential speed. If I am still a biological human, then it will indeed be a problem that all these FLOP happen faster relative to clock time, since they are also happening faster relative to me, and I will have much less of my own FLOP to predict and control each batch of X FLOP made by humanity-as-a-whole.

In a scenario with uploads, my FLOP will also speed up. But the rest of humanity/machines won’t only speed up, they will also build way more thinking machines. So unless I speed up even more, or my own cognitive machinery also grows at that rate (via tools, or copies of me or enlarging my brain), the ratio of my FLOP to humanity’s FLOP will still decrease.

But there’s conceivable reasons for worry, even if this ratio is held constant:

  • Maybe prediction becomes differentially harder with scale. That is, maybe using A FLOPs (my cognitive machinery pre-Singularity) to predict X FLOPs (that of humanity pre-Singularity) is easier than using 10A FLOPs (my cognitive machinery post-Singularity) to predict 10X FLOPs (that of humanity post-Singularity). But why? Can’t I just split the 10X in 10 bins, and usea an A to predict each of them as satisfactorily as before? Maybe not, due to the newly complex interconnections between these bins. Of course, such complex interconnections also become positive for my cognitive machinery. But maybe the benefit for prediction from having those interconnections in my machinery is lower than the downgrade from having them in the predicted computation.

[A priori this seems false if we extrapolate from past data, but who knows if this new situation has some important difference.]

  • Maybe some other properties of the situation (like the higher computation-density in the physical substrate requiring the computations to take on a slightly different, more optimal shape [this seems unlikely]) lead to the predicted computation having some new properties that make it harder to predict. Such properties need not even be something absolute, that “literally makes prediction harder for everyone” (even for intelligences with the right tools/heuristics). It could just be “if I had the right heuristics I might be able to predict this just as well as before (or better), but all my heuristics have been selected for the pre-Singularity computation (which didn’t have this property), and now I don’t know how to proceed”. [I can run again a selection for heuristics (for example running again a copy of me growing up), but that takes a lot more FLOP.]
Comment by Martín Soto (martinsq) on Gentleness and the artificial Other · 2024-01-05T22:28:42.451Z · LW · GW

You might enjoy this analysis of the piece of sci-fi you didn't want to spoil.

There’s a movie that I’m trying not to spoil, in which an AI in a female-robot-body makes a human fall in love with her, and then leaves him to die, trapped and screaming behind thick glass. One of the best bits, I think, is the way, once it is done, she doesn’t even look at him."

Comment by Martín Soto (martinsq) on Gentleness and the artificial Other · 2024-01-05T12:07:16.903Z · LW · GW
Comment by Martín Soto (martinsq) on Gentleness and the artificial Other · 2024-01-05T12:04:45.230Z · LW · GW
Comment by Martín Soto (martinsq) on Meaning & Agency · 2023-12-21T21:35:37.798Z · LW · GW

My intuition says the natural thing would be to assume something about the experts not talking about each other (which probably means being independent, which sounds too strong). I feel like whenever they can talk about each other an example like this will exist. But not sure! Maybe you can have a relative endorsement definition that's more like "modulo the other information I'm receiving about you from the environment, I treat the additional bits you're giving me as the best information".

Comment by Martín Soto (martinsq) on Meaning & Agency · 2023-12-20T18:11:58.597Z · LW · GW

(I will not try to prove transitivity here, since my goal is to get the overall picture across; I have not checked it, although I expect it to hold.)

Transitivity doesn't hold, here's a counterexample.

The intuitive story is: X's action tells you whether Z failed, Y fails sometimes, and Z fails more rarely.

The full counterexample (all of the following is according to your beliefs ): Say available actions are 0 and 1. There is a hidden fair coin, and your utility is high if you manage to match the coin, and low if you don't. Y peeks at the coin, and takes the correct action, except when it fails, which has a 1/4 chance. Z does the same, but it only fails with a 1/100 chance. X plays 1 iff Z has failed.
Given X's and Y's action, you always go with Y's action, since X tells you nothing about the coin, and Y gives you some information. Given Z's and Y's actions, you always go with Z's, because it's less likely to have failed (even when they disagree). But given Z's and X's, there will be some times (1/100), in which you see X played 1, and then you will not play the same as Z.

The same counterexample works for beliefs (or continuous actions) instead of discrete actions (where you will choose a probability  to believe, instead of an action ), but needs a couple small changes. Now both Z and Y fail with 1/4 probability (independently). Also, Y outputs its guess as 0.75 or 0.25 (instead of 1 or 0), because YOU (that is, ) will be taking into account the possibility that it has failed (and Y better output whatever you will want to guess after seeing it). Instead of Z, consider A as the third expert, which outputs 0.5 if Z and Y disagree, 15/16 if they agree on yes, and 1/16 if they agree on no. X still tells you whether Z failed. Seeing Y and X, you always go with Y's guess. Seeing A and Y, you always go with A's guess. But if you see A = 15/16 and X = 1, you know both failed, and guess 0. (In fact, even when you see X = 0, you will guess 1 instead of 15/16.)

Comment by Martín Soto (martinsq) on OpenAI: Leaks Confirm the Story · 2023-12-13T10:47:32.642Z · LW · GW


AGI is the sweetest, most interesting, most exciting challenge in the world.

We usually concede this point and I don't even think it's true. Of course, even if I'm right, maybe we don't want to push in this direction in dialogue, because it would set the bad precedent of not defending ethics over coolness (and sooner or later something cool will be unethical). But let me discuss anyway.

Of course building AGI is very exciting, it incentivizes some good problem-solving. But doing it through Deep Learning and like OpenAI does has an indeleble undercurrent of "we don't exactly know what's going on, we're just stirring the linear algebra pile". Of course that can already be an incredibly interesting engineering problem, and it's not like you don't need a lot of knowledge and good intuitions to make these hacky things work. But I'm sure the aesthetic predispositions of many (and especially the more mathematically oriented) will line up way better with "actually understanding the thing". From this perspective Alignment, Deep Learning Theory, Decision Theory, understanding value formation, etc. feel fundamentally more interesting intellectual challenges. I share this feeling and I think many other people do. A lot of people have been spoiled by the niceness of math, and/or can't stand the scientific shallowness of ML developments.

Comment by Martín Soto (martinsq) on How LDT helps reduce the AI arms race · 2023-12-11T09:48:54.620Z · LW · GW

I think this has a fix-point selection problem: If one or both of them start with a different prior under which the other player punishes them for not racing / doesn't reward them enough (maybe because they have very little faith in the other's rationality, or because they think it's not within their power to decide that, and also there's not enough evidential correlation in their decisions), then they'll race.

Of course, considerations about whether the other player normatively endorses something LDT-like also enter the picture. And even if individual humans would endorse it (and that's already a medium-big if), I worry our usual decision structures (for example in AI labs) don't incentivize it (and what's the probability some convincing decision theorist cuts through them? not sure).

Comment by Martín Soto (martinsq) on Vegan Nutrition Testing Project: Interim Report · 2023-10-08T13:14:23.365Z · LW · GW

Just want to quickly flag that, based on my anecdotal experience, the vegan communities I was thinking of in which nutrition was thoroughly discussed didn't involve learning from vegan elders either. They were mostly students, and had learned about nutrition from the internet, books, memes and visiting professionals, and in fact I recall them as being more heavy on the nutrition talk than the older vegans I've met (even if the elders also supplemented etc.). I feel like the adequate seriousness with which they treated nutrition came more from a place of positive and optimistic "let's do things the right way" (to be a good example, to maintain some important boundaries that will allow us to help sustainably, etc.).

Another is that compulsive optimization doesn't lead people to neglect something as simple as iron.

I disagree, I think unfortunately this and worse can happen in some environments and mental spaces.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-10-08T13:04:00.836Z · LW · GW

(See also this new comment.)

First off, thanks for including that edit (which is certainly better than nothing), although that still doesn't neglect that (given the public status of the post) your summaries will be the only thing almost everyone sees (as much as you link to these comments or my original text), and that in this thread I have certainly just been trying to get my positions not misrepresented (so I find it completely false that I'm purposefully imposing an unnecessary tax, even if it's true that engaging with this misrepresentation debate takes some effort, like any epistemic endeavor).

Here's the two main reasons why I wouldn't find your proposal above fair:

  1. I expect most people who will see this post / read your summaries of my position to have already seen it (although someone correct me if I'm wrong about viewership dynamics in LessWrong). As a consequence, I'd gain much less from such a disclaimer / rethinking of the post being incorporated now (although of course it would be positive for me / something I could point people towards).
    Of course, this is not solely a consequence of your actions, but also of my delayed response times (as I had already anticipated in our clarifications thread).
    1. A second order effect is that most people who have seen the post up until now will have been "skimmers" (because it was just in frontpage, just released, etc.), while probably more of the people who read the post in the future will be more thorough readers (because they "went digging for it", etc.). As I've tried to make explicit in the past, my worry is more about the social dynamics consequences of having such a post (with such a framing) receive a lot of public attention, than with any scientific inquiry into nutrition, or any emphasis on public health. Thus, I perceive most of the disvalue coming from the skimmers' reactions to such a public signal. More on this below.
  2. My worry is exactly that such a post (with such a framing) will not be correctly processed by too many readers (and more concretely, the "skimmers", or the median upvoter/downvoter), in the sense that they will take away (mostly emotionally / gutturally) the wrong update (especially action-wise) from the actual information in the post (and previous posts).
    Yes: I am claiming that I cannot assume perfect epistemics from LessWrong readers. More concretely, I am claiming that there is a predictable bias in one of two emotional / ethical directions, which exists mainly due to the broader ethical / cultural context we experience (from which LessWrong is not insulated).
    Even if we want LessWrong to become a transparent hub of information sharing (in which indeed epistemic virtue is correctly assumed of the other), I claim that the best way to get there is not through completely implementing this transparent information sharing immediately in the hopes that individuals / groups will respond correctly. This would amount to ignoring a part of reality that steers our behavior too much to be neglected: social dynamics and culturally inherited biases. I claim the best way to get there is by implementing this transparency wherever it's clearly granted, but necessarily being strategic in situations when some unwanted dynamics and biases are at play. The alternative, being completely transparent ("hands off the simulacrum levels"), amounts to leaving a lot of instrumental free energy on the table for these already existing dynamics and biases to hoard (as they have always done). It amounts to having a dualistic (as opposed to embedded) picture of reality, in which epistemics cannot be affected by the contingent or instrumental. And furthermore, I claim this topic (public health related to animal ethics) is unfortunately one of the tricky situations in which such strategicness (as opposed to naive transparency) is the best approach (even if it requires some more efforts on our part).
    Of course, you can disagree with these claims, but I hope it's clear why I don't find a public jury is to be trusted on this matter.
    1. You might respond "huh, but we're not talking about deciding things about animal ethics here. We're talking about deciding rationally whether some comments were or weren't useful. We certainly should be able to at least trust the crowd on that?" I don't think that's the case for this topic, given how strong the "vegans bad" / "vegans annoying" immune reaction is for most people generally (that is, the background bias present in our culture / the internet).
    2. As an example, in this thread there are some people (like you and Jim) who have engaged with my responses / position fairly deeply, and for now disagreed. I don't expect the bulk of the upvotes / downvotes in this thread (or if we were to carry out such a public vote) to come from this camp, but more from "skimmers" and first reactions (that wouldn't enter the nuance of my position, which is, granted, slightly complex). Indeed (and of course based on my anecdotal experience on the internet and different circles, including EA circles), I expect way too many anonymous readers/voters to, upon seeing something like human health and animal ethics weighed off in this way, would just jump on the bandwagon of punishing the veganism meme for the hell of it.
      And let me also note that, while further engagement and explicit reasoning should help with recognizing those nuances (although you have reached a different conclusion), I don't expect this to eliminate some strong emotional reactions to this topic, that drive our rational points ("we are not immune to propaganda"). And again, given the cultural background, I expect these to go more in one direction than the other.

So, what shall we do? The only thing that seems viable close to your proposal would be having the voters be "a selected crowd", but I don't know how to select it (if we had half and half this could look too much like a culture war, although probably that'd be even better than the random crowd due to explicitly engaging deeply with the text). Although maybe we could agree on 2-3 people. To be honest, that's sounding like a lot of work, and as I mentioned I don't think there's that much more in this debate for me. But I truly think I have been strongly misrepresented, so if we did find 2-3 people who seemed impartial and epistemically virtuous I'd deem it positive to have them look at my newest, overly explicit explanation and express opinions.

So, since your main worry was that I hadn't made my explanation of misrepresentation explicit enough (and indeed, I agree that I hadn't yet written it out in completely explicit detail, simply because I knew that would require a lot of time), I have in this new comment provided the most explicit version I can muster myself to compose. I have made it explicit (and as a consequence long) enough that I don't think I have many more thoughts to add, and it is a faithful representation of my opinions about how I've been misrepresented.
I think having that out there, for you (and Jim, etc.) to be able to completely read my thoughts and re-consider whether I was misrepresented, and for any passer-by who wants to stop by to see, is the best I can do for now. In fact, I would recommend (granted you don't change your mind more strongly due to reading that) that your edit linked to this new, completely explicit version, instead of my original comment written in 10 minutes.

I will also note (since you seemed to care about the public opinions of people about the misrepresentation issue) that 3 people (not counting Slapstick here) (only one vegan) have privately reached out to me to say they agree that I have been strongly misrepresented. Maybe there's a dynamic here in which some people agree more with my points but stay more silent due to being in the periphery of the community (maybe because of perceived wrong-epistemics in exchanges like this one, or having different standards for information-sharing / what constitutes misrepresentation, etc.).

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-10-08T12:55:24.740Z · LW · GW

I think my position has been strongly misrepresented here.

As per the conclusion of this other comment thread, I here present a completely explicit explanation of where and how I believe my position to have been strongly misrepresented. (Slapstick also had a shot at that in this shorter comment.)

Misrepresentation 1: Mistaking arguments

Elizabeth summarizes

The charitable explanation here is that my post focuses on naive veganism, and Soto thinks that’s a made-up problem. He believes this because all of the vegans he knows (through vegan advocacy networks) are well-educated on nutrition.

It is false that I think naive veganism is a made-up problem, and I think Elizabeth is taking the wrong conclusions from the wrong comments.

Her second sentence is clearly a reference to this short comment of mine (which was written as a first reaction to her posts, before my longer and more nuanced explanation of my actual position):

I don't doubt your anecdotal experience is as you're telling it, but mine has been completely different, so much so that it sounds crazy to me to spend a whole year being vegan, and participating in animal advocacy, without hearing mention of B12 supplementation. Literally all vegans I've met have very prominently stressed the importance of dietary health and B12 supplementation.

As should be obvious, this is not contesting the existence of naive veganism ("I don't doubt your anecdotal experience"), but just contrasting it with my own personal anecdotal experience. This was part of my first reaction, and didn't yet involve a presentation of my actual holistic position.

Elizabeth arrives at the conclusion that, because of my anecdotal experience, I believe naive veganism doesn't exist (I don't trust the other anecdotal experiences reported by her or other commenters), and that's the reason why I don't agree with her framing. I think my longer explanation makes it evident that I'm not ignoring the existence of naive veganism, but instead quantitatively weighing against other consequences of Elizabeth's posts and framings. For example:

To the extent the naive transition accounts are representative of what's going on in rat/EA spheres, some intervention that reduces the number of transitions that are naive (while fixing the number of transitions) would be a Pareto-improvement. And an intervention that reduces the number of transitions that are naive, and decreases way less the number of transitions, would also be net-positive.


My worry, though, is that signaling out veganism for this is not the most efficient way to achieve this. I hypothesize that

  1. Naive transitions are more correlated with social dynamics about insuficient self-care not exclusive (nor close to exclusive) to veganism in rat/EA spheres.

Of course, these two excerpts are already present in the screenshots presented in the post (which indeed contain some central parts of my position, although leave out some important nuance), so I find this misrepresentation especially hard to understand or explain. I think it's obvious, when saying something like "Naive transitions are more correlated with social dynamics...", that I endorse their existence (or at least their possible existence).

Yet another example, in the longer text I say:

It feels weird for me to think about solutions to this community problem, since in my other spheres it hasn't arisen.

This explicitly acknowledges that this is a problem that exists in this community. Indeed, engaging with Elizabeth's writing and other anecdotal accounts in the comments has updated upwards my opinion of how many naive vegans exist in the rationalist community. But this is mostly independent of my position, as the next paragraph addresses.

You might worry, still, that my position (even if this is not stated explicitly) is motivated in reality by such a belief, and other arguments are rationalizations. First off I will note that this is a more complex inference, and while it is possible to believe it, it's clearly not a faithful representation of the explicit text, and should be flagged as such. But nonetheless, on the hopes of convincing you that this is not going on, I will point out that my actual position and arguments have to do mostly with social dynamics, epistemics, and especially the community's relationship to ethics. (See Misrepresentation 2: Mistaking claims for proof.)

Given, then, that the most important consequences of all this go through dynamics, epistemics and relationship to ethics (since this community has some chance of steering big parts of the future), I think it's clear that my position isn't that sensitive to the exact number of naive vegans. My position is not about ignoring those that exist: it is about how to come about solving the problem. It is about praxis, framing and optimal social dynamics.

Even more concretely, you might worry I'm pulling off a Motte and Bailey, trying to "quietly imply" with my text that the number of naive vegans is low, even if I don't say it explicitly. You might get this vibe, for example, from the following phrasing:

To the extent the naive transition accounts are representative of what's going on in rat/EA spheres,...

I want to make clear that this phrasing is chosen to emphasize that I think we're still somewhat far from having rigorous scientific knowledge about how prevalent naive veganism is in the community (for example, because your sample sizes are small, as I have mentioned in the past). That's not to neglect the possibility of naive veganism being importantly prevalent, as acknowledged in excerpts above.

I also want to flag that, in the short comment mentioned above, I said the following:

since most vegans do supplement [citation needed, but it's been my extensive personal and online experience, and all famous vegan resources I've seen stress this]

This indeed is expressing my belief that, generally, vegans do supplement, based on my anecdotal experience and other sources. This is not yet talking explicitly about whether this is the case within the community (I didn't have strong opinions about that yet), but it should correctly be interpreted (in the context of that short comment) as a vague prior I am using to be (a priori) doubtful of deriving strong conclusions about the prevalence in the community. I do still think this vague prior is still somewhat useful, and that we still don't have conclusive evidence (as mentioned above). But it is also true (as mentioned above) that this was my first reaction, written before my long text holistically representing my position, and since then I have updated upwards my opinion of how many naive vegans exist in the rationalist community. So it makes sense that this first short comment was more tinged by an implicit lower likelihood of that possibility, but this was superseded by further engaging with posts and comments, and that is explicitly acknowledged in my latter text, as the above excerpts demonstrate.

Finally, one might say "well, of course Elizabeth didn't mean that you literally think 0 naive vegans exist, it was just a way to say you thought too few of them existed, or you were purposefully not putting weight on them". First off, even if those had been my actual stated or implied positions, I want to note this use of unwarranted hyperbole can already tinge a summary with unwarranted implications (especially a short summary of a long text), and thus would find it an implicit misrepresentation. And this is indeed part of what I think is going on, and that's why I repeatedly mention my problem is more with framing and course of action than with informational content itself. But also, as evidenced by the excerpts and reasoning above, it is not the case that I think too few naive vegans exist, or I purposefully don't represent them. I acknowledge the possibility that naive veganism is prevalent amongst community vegans, and also imply that my worries are not too sensitive to the exact number of naive vegans, and are of a different nature (related to epistemics and dynamics).

In summary, I think Elizabeth went something like "huh, if Martín is expressing these complex worries, it probably is just because he thinks naive veganism is not a real problem, since he could be understood to have some doubts about that in his early comments". And I claim that is not what's going on, and it's completely missing the point of my position, which doesn't rely in any way on naive veganism not being a thing. On the contrary, it discusses directly what to do in a community where naive veganism is big. I hope the above excerpts and reasoning demonstrated that.

Misrepresentation 2: Mistaking claims

Elizabeth summarizes

I have a lot of respect for Soto for doing the math and so clearly stating his position that “the damage to people who implement veganism badly is less important to me than the damage to animals caused by eating them”.

This, of course, makes it very explicitly sound like in my text I only weigh two variables against each other: disvalue caused by naive veganism, and disvalue caused by animal exploitation.

This is missing a third variable that is very present in my long text, and to which many paragraphs are dedicated or make reference: the consequences of all this (posts, framing, actions, etc.) for social dynamics of the community, and the community's (and its individuals') relationship to ethics.

In fact, not only is this third variable very present in the text, but also in some places I explicitly say it's the most important variable of the three, so demonstrating that my arguments have mostly to do with it. Here's one excerpt making that explicit:

As an extreme example, I very strongly feel like financing the worst moral disaster of current times so that "a few more x-risk researchers are not slightly put off from working in our office" is way past the deontological failsafes. As a less extreme example, I strongly feel like sending a message that will predictably be integrated by most people as "I can put even less mental weight on this one ethical issue that sometimes slightly annoyed me" also is. And in both cases, especially because of what they signal, and the kind of community they incentivize.

Here's another one, even more clear:

But I am even more worried about the harder-to-pin-down communal effects, "tone setting", and the steering of very important sub-areas of the EA community into sub-optimal ethical seriousness (according to me), which is too swayed by intellectual fuzzies, instead of actual utilons.

And finally, in my response answering some clarificatory questions from Elizabeth (several days before this post was published), here's an even more explicit one:

Of course, I too don't optimize for "number of vegans in the world", but just a complex mixture including that as a small part. And as hinted above, if I care about that parameter it's mainly because of the effects I think it has in the community. I think it's a symptom (and also an especially actionable lever) of more general "not thinking about ethics / Sincerity in the ways that are correct". As conscious as the members of this community try to be about many things, I think it's especially easy (through social dynamics) to turn a blind eye on this, and I think that's been happening too much.

Indeed, one of Elizabeth's screenshots already emphasizes this, placing it as one of the central parts of my argument (although doesn't yet explicitly mention that it's, for me, the most important consequence):

Relatedly, incentivizing a community that's more prone to ignoring important parts of the holistic picture when that goes to the selfish benefit of individuals. (And that's certainly something we don't want to happen around the people taking important ethical decisions for the future.)

I do admit that, stylistically speaking, this point would have been more efficiently communicated had I explicitly mentioned its importance very near the top of my text (so that it appeared, for example, explicitly mentioned in her first screenshot).

Nonetheless, as the above excerpts show, the point (this third, even more important variable) was made explicit in some fragments of the text (even if the reader could have already understood it as implied by other parts that don't mention it explicitly). And so, I cannot help but see Elizabeth's sentence above as a direct and centrally important misrepresentation of what the text explicitly communicated.

You might worry, again, that there's some Motte and Bailey going on, of the form "explicitly mention those things, but don't do it at the top of the text, so that it seems like truly animal ethics is the only thing you care about, or something". While I'm not exactly sure what I'd gain from this practice (since anyway it's patent that many readers disagree with me ethically about the importance of animals, so I might as well downweigh its importance), I will still respond to this worry by pointing that, even if the importance of this third variable is only explicitly mentioned further down in the text, most of the text (and indeed, even parts of the screenshots) already deals with it directly, thus implying its importance and centrality to my position, and furthermore most of the text discussed / builds towards the importance of this third variable in a more detailed and nuanced way than just stating it explicitly (to give a better holistic picture of my thoughts and arguments).

In summary, not only does this representation neglect a central part of my text (and something that I explicitly mentioned was the most important variable in my argument), but also, because of that, attributes to me a statement that I haven't stated and do not hold. While I am uncertain about it (mostly because of remaining doubts about how prevalent naive veganism is), it is conceivable (if we lived in a world with high naive veganism) that, if we ignored all consequences of these posts/framings/actions except for the two variables Elizabeth mentions, attacking naive veganism through these posts is at least net-positive (even if, still, in my opinion, not the optimal approach). But of course, the situation completely changes when realistically taking into account all consequences.

How might Elizabeth have arrived at this misrepresentation? Well, it is true that at the start of my long text I mention:

And an intervention that reduces the number of transitions that are naive, and decreases way less the number of transitions, would also be net-positive.

It is clear how this short piece of text can be interpreted as implying that the only two important variables are the number of naive transitions and the number of transitions (even if I shortly later make clear these are not the only important variables, and most of the text is devoted to discussing this, and I even explicitly mention that is not the case). But clearly that doesn't imply that I believe "the damage to people who implement veganism badly is less important to me than the damage to animals caused by eating them". I was just stating that, under some situations, it can make sense to develop certain kinds of health-focused interventions (to make evident that I'm not saying "one should never talk about vegan nutrition", which is what Elizabeth was accusing me of doing). And indeed a central part of my position as stated in the text was that interventions are necessary, but of a different shape to Elizabeth's posts (and I go on to explicitly recommend examples of these shapes). But of course that's not the same as engaging in a detailed discussion about which damages are most important, or already taking into account all of the more complex consequences that different kinds of interventions can have (which I go on to discuss in more detail in the text).

Misrepresentation 3: Missing counter-arguments and important nuance

Elizabeth explains

There are a few problems here, but the most fundamental is that enacting his desired policy of suppressing public discussion of nutrition issues with plant-exclusive diets will prevent us from getting the information to know if problems are widespread. My post and a commenter’s report on their college group are apparently the first time he’s heard of vegans who didn’t live and breathe B12. 

But I can’t trust his math because he’s cut himself off from half the information necessary to do the calculations. How can he estimate the number of vegans harmed or lost due to nutritional issues if he doesn’t let people talk about them in public?

First off, the repeated emphasis on "My post and a commenter's report..." (when addressing this different point she's brought up) again makes it sound as if my position was affected or relied on a twisted perception of the world in which naive vegans don't exist. I have already addressed why this is not the case in Misrepresentation 1: Mistaking arguments, but I would like to call attention again to the fact that framing and tone are used to caricaturize my position, or make it seem like I haven't explicitly addressed Elizabeth's point here (and I haven't done so because of a twisted perception). I already find this mildly misleading, given I both had directly addressed that point, and that the content of the text clearly shows my position doesn't depend on the non-existence of naive veganism as a community problem. But of course, it's not clear (in this one particular sentence) where authorial liberties of interpretation should end. Maybe Elizabeth is just trying to psychoanalyze me here, finding the hidden motives for my text (even when the text explicitly states different things). First, I would have preferred this to be flagged more clearly, since the impression I (and probably most readers, who of course won't read my long comment) get from this paragraph is that of implying that my text showcased an obvious-to-all naiveté and didn't address these points. Second, in Misrepresentation 1: Mistaking arguments I have argued why these hidden motives are not real (and again, that is clear form the content of the long text).

Now on to Elizabeth's main point. In my response to Elizabeth's response to my long text (which was sent several days before the post's publication), I addressed some clarifications that Elizabeth had asked for. There, answering directly to her claims that (on her immediate experience) the number of "naive vegans turned non-naive" had been much greater than the number of "vegans turned non-vegan" (which, again, my holistic position doesn't too strongly quantitatively rely on), I said:

The negative effects of the kind "everyone treats veganism less seriously, and as a result less people transition or are vocal about it" will be much more diffused, hard-to-track, and not-observed, than the positive effects of the kind "this concrete individual started vegan supplements". Indeed, I fear you might be down-playing how easy it is for people to arrive (more or less consciously) at these rationalized positions, and that's of course based on my anecdotal experience both inside and outside this community.

Thus, to her claim that I have "cut myself off from half the information", I was already pre-emptively responding by noting that (in my opinion) she has cut herself off from the other half of the information, by ignoring these kind of more diluted effects (that, according to my position, have the biggest impact on the third and most important variable of social dynamics, epistemics, and ethical seriousness). Again, it is also clear in this excerpt that I am worrying more about "diluted effects on social dynamics" than about "the exact figure of how wide-spread naive veganism is".

Indeed (and making a more general diagnostic of the misrepresentation that has happened here), I think Elizabeth hasn't correctly understood that my holistic position, as represented in those texts (and demonstrated in the excerpts presented above), brought forth a more general argument, not limited to short-term interventions against naive veganism, nor sensitively relying on how widespread naive veganism is.
Elizabeth understands me as saying "we should ignore naive veganism". And then, of course, the bigger naive veganism is, the bigger a mistake I might have been making. But in reality my arguments and worries are about framing and tone, and comparing different interventions based on all of their consequences, including the "non-perfectly-epistemic" consequences of undesirably exacerbating this or that dynamic. Here's an excerpt of my original long text exemplifying that:

As must be clear, I'd be very happy with treating the root causes, related to the internalized optimizy and obsessive mindset, instead of the single symptom of naive vegan transitions. This is an enormously complex issue, but I a prior think available health and wellbeing resources, and their continuous establishment as a resource that should be used by most people (as an easy route to having that part of life under control and not spiraling, similar to how "food on weekdays" is solved for us by our employers), would provide the individualization and nuance that these problems require.

Even more clearly, here is one excerpt where I mention I'm okay with running clinical trials to get whatever information we might need to better navigate this situation:

Something like running small group analytics on some naive vegans as an excuse for them to start thinking more seriously about their health? Yes, nice! That's individualized, that's immediately useful. But additionally extracting some low-confidence conclusions and using them to broadcast the above message (or a message that will get first-order approximated to that by 75%) seems negative.

The above makes clear that my worry is not about obtaining or making available that information. It is about the framing and tone of Elizabeth's message, and the consequences it will have when naively broadcast (without accounting for a part of reality: social dynamics).

Finally, Elizabeth says my desired policy is "suppressing public opinion". Of course, that's already a value judgement, and it's tricky to debate what counts as "suppressing public opinion", and what as "acknowledging the existence of social dynamics, and not shooting yourself in the foot by doing something that seems bad when taking them into account". I'm confident that my explanations and excerpts above satisfactorily argue for my having advocated for the latter, and not the former. But then again, as with hidden motives mentioned above, arriving at different conclusions than I do about this (about the nature of what I have written) is not misrepresentation, just an opinion.
What I do find worrisome is how this opinion has been presented and broadcast (so, again, framing). If my position had been more transparently represented, or if Elizabeth had given up on trying to represent it faithfully in a short text, and Elizabeth had nonetheless mentioned explicitly that her interpretation of that text was that I was trying to suppress public discussion (even though I had explicitly addressed public discussion and when and how it might be net-positive), then maybe it would have been easier for the average reader to notice that there might be an important difference of interpretations going on here, and that they shouldn't update so hard on her interpretation as if I had explicitly said "we shouldn't publicly discuss this (under any framing)". And even then I would worry this over-represented your side of the story (although part of that is unavoidable).
But this was not the case. These interpretations were presented in a shape pretty indistinguishable from what would have been an explicit endorsed summary. Her summary looks completely the same as it would look had I not addressed and answered the points she brings up in any way, or explicitly stated the claims and attitudes she attributes to me.

In summary, although I do think this third misrepresentation is less explicitly evident than the other two (due to mixing up with Elizabeth's interpretation of things), I don't think her opinions have been presented in a shape well-calibrated about what I was and wasn't saying, and I think this has led the average reader to, together with Elizabeth, strongly misrepresent my central positions.

Thank you for reading this wall of overly explicit text.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T20:52:23.340Z · LW · GW

This only asserts that there's a mismatch; it provides no actual evidence of one

I didn't provide quotes from my text when the mismatch was obvious enough from any read/skim of the text. In this case, for example, even the screenshots of my text included in the post demonstrate that I do think naive transitions to veganism exist. So of course this is more a point about framing, and indeed notice that I already mentioned in another comment that this one example might not constitute a strong misrepresentation, as the other two do (after all, it's just an hyperbole), although it  still gives me worries about biased tone-setting in a vaguer way.

Pretty straightforwardly, if the pilot study results had only been sent through private communications, then they wouldn't have public discussion (ie, public discussion would be suppressed).

In the text I clearly address why

  1. My proposal is not suppressing public discussion of plant-based nutrition, but constructing some more holistic approach whose shape isn't solely focused on plant-based diets, or whose tone and framing aren't like this one (more in my text).
  2. I don't think it's true private communications "prevent us from getting the information" in important ways (even if taking into account the social dynamics dimension of things will always, of course, be a further hindrance). And also, I don't think public communications give us some of the most important information.

I hope it is now clear why I think Elizabeth's quoted sentence is a misrepresentation, since neither I push for suppressing public discussions of plant-based nutrition (only a certain non-holistic approach to this, more concretely, Elizabeth's approach), nor I ignored the possible worry that this prevents us from obtaining useful information (on the contrary, I addressed this). Of course we can object-level argue about whether these (my positions) are true (that's what I was trying to do with Elizabeth, although as stated she didn't respond to these two further points), but what's clear is that they are the ones represented in my text.

More generally, I think this is a kind of "community-health combating of symptoms" with many externalities for the epistemic and moral capabilities of our community (and ignoring them by ignoring the social dynamics at play in our community and society seems like wishful thinking, we are not immune to propaganda), and I think different actions will lead to a healthier and more robust community without the same externalities (all of this detailed in my text).

In any event, I will stop engaging now. I just wanted my name not to be associated with those positions in a post that will be read by so many people, but it's not looking like Elizabeth will fix that, and having my intentions challenged constantly so that I need to explain my each and every mental move is too draining.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T16:17:26.718Z · LW · GW

As I've object-level argued above, I believe these summaries fall into the category of misrepresentation, not just interpretation. And I don't believe an author should maintain such misrepresentations in their text in light of evidence about them.

In any event, certainly a link to my comment is better than nothing. At this point I'm just looking for any gesture in the direction of avoiding my name from being associated with positions I do not hold.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T13:13:39.298Z · LW · GW

As mentioned, my main objective in writing these comments is not have associated with a position I don't hold. Many people will see that part of the post, but not go into the comments, and even more will go into the comments but won't see my short explanation of how my position has been misrepresented, since it has been heavily down-voted (although I yet have to hear any object-level argument for why the misrepresentation actually hasn't happened or my conduct is epistemically undesirable). So why wouldn't I demand for the most public part of all this to not strongly misrepresent my position?

I get it this incurs some extra effort on Elizabeth (as does any change or discussion), but I'm trying to minimize that by offering myself to write the alternate summary, or even just remove that part of the text (which should take minimal effort). She's literally said things I didn't claim (especially the third example), and a demand to fix that doesn't seem so far-fetched to me that the first guess is I'm just trying to mud the waters.

Maybe we're still fresh from the Nonlinear exchange and are especially wary of tactical incurring of costs, but of course I don't even need to point out how radically different this situation is.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T11:24:11.789Z · LW · GW

It's true the boundary between interpretation and strong misrepresentation is fuzzy. In my parent comment I'm providing object-level arguments for why this is a case of strong misrepresentation. This is aggravated by the fact that this post will be seen by a hundred times more people than my actual text, by the fact that Elizabeth herself reached out for these clarifications (which I've spent time to compose), and by the fact that I offered to quickly review the write-up more than a week ago.

I'm not trying to incur any extra costs, Elizabeth is free to post her opinions even if I believe doing so is net-negative. I'm literally just commenting so that my name is not associated with opinions which are literally not held by me (this being completely explicited in my linked text, but of course this being too long for almost anyone to actually check first-hand).

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T11:17:52.451Z · LW · GW

This stood out to me as strange. Are you referring to this comment?

No, I was referring to this one, and the ones in that thread, all part of an exchange in which Elizabeth reached out to me for clarification.

In the one you quoted I was still not entering that much detail.

I'll answer your comment nonetheless.

It sounds like you're saying that the nutritional requirements of veganism are so complex that they require individualized professional assistance, that there is no one-page "do this and you will get all the nutrients you need" document that will work for a the vast majority of vegans.

No, what I was saying wasn't as extreme. I was just saying that it's good general practice to visit a nutritionist at least once, learn some of the nutritional basics and perform blood tests periodically (each 1 or 2 years). That's not contradictory with the fact that most vegans won't need to pour a noticeable amount of hours into all this (or better said, they will have to do that the first 1-2 months, but mostly not afterwards). Also, there is no one-page be-all end-all for any kind of nutrition, not only veganism. But there certainly exist a lot of fast and easy basic resources.

After reading your post, I feel like you are making a distinction without a difference here. You mention community dynamics, but they are all community dynamics about the ethical implications of veganism in the community, not the epistemic implications. It seems perfectly fair for Elizabeth to summarize your position the way she does.

Yes, of course, we were talking about veganism. But in the actual comment I was referring to, I did talk about epistemic implications, not only implications for animal ethics (as big as they already are). What I meant is "if there is something that worries me even more than the animal ethics consequences of this (which are big), it is breeding a community that shies away from basic ethical responsibility at the earliest possibility and rationalizes the choice (because of the consequences this can have for navigating the precipice)".

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T11:08:23.066Z · LW · GW

Yes, your quotes show that I believe (and have stated explicitly) that publishing posts like this one is net-negative. That was the topic of our whole conversation. That doesn't imply that I'm commenting to increase the costs of these publications. I tried to convince Elizabeth that this was net-negative, and she completely ignored those qualms, and that's epistemically respectable. I am commenting mainly to avoid my name from being associated with some positions that I literally do not hold.

I believe that her summaries are a strong misrepresentation of my views, and explained why in the above comment through object-level references comparing my text to her summaries. If you don't provide object-level reasons why the things I pointed out in my above comment are wrong, then I can do nothing with this information. (To be clear, I do think the screenshots are fairly central parts of my clarifications, but her summaries misrepresent and directly contradict other parts of them which I had also presented as central and important.)

I do observe that providing these arguments is a time cost for you, or fixing the misrepresentations is a time cost for Elizabeth, etc. So the argument "you are just increasing the costs" will always be available for you to make. And to that the only thing I can say is... I'm not trying to get the post taken down, I'm not talking about any other parts of the post, just the ones that summarize my position.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-30T08:39:27.243Z · LW · GW

Yes, I was referring to your written summaries of my position, which are mostly consistent with the shown screenshots, but not with other parts of my answers. That's why I kindly demand these pieces of text attached to my name to be changed to stop misrepresenting my position (I can provide written alternate versions if that helps), or at least removed while this is pending.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-29T16:11:46.933Z · LW · GW

This post literally strongly misrepresents my position in three important ways¹. And these points were purposefully made central in my answers to the author, who kindly asked for my clarifications but then didn't include them in her summary and interpretation. This can be checked by contrasting her summary of my position with the actual text linked to, in which I clarified how my position wasn't the simplistic one here presented.

Are you telling me I shouldn't flag that my position has been importantly misrepresented? On LessWrong? And furthermore on a post that will be seen by way more people than my original text?

¹ I mean the three latter in my above comment, since the first (the hyperbolic presentation) is worrisome but not central.

Comment by Martín Soto (martinsq) on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem · 2023-09-29T10:16:15.376Z · LW · GW

Hi Elizabeth, I feel like what I wrote in those long comments has been strongly misrepresented in your short explanations of my position in this post, and I kindly ask for a removal of those parts of the post until this has cleared up (especially since I had in the past offered to provide opinions on the write-up). Sadly I only have 10 minutes to engage now, but here are some object-level ways in which you've misrepresented my position:

The charitable explanation here is that my post focuses on naive veganism, and Soto thinks that’s a made-up problem.

Of course, my position is not as hyperbolic as this.

his desired policy of suppressing public discussion of nutrition issues with plant-exclusive diets will prevent us from getting the information to know if problems are widespread

In my original answers I address why this is not the case (private communication serves this purpose more naturally).

I have a lot of respect for Soto for doing the math and so clearly stating his position that “the damage to people who implement veganism badly is less important to me than the damage to animals caused by eating them”.

As I mentioned many times in my answer, that's not the (only) trade-off I'm making here. More concretely, I consider the effects of these interventions on community dynamics and epistemics possibly even worse (due to future actions the community might or might not take) than the suffering experienced by farmed animals murdered for members of our community to consume at present day.

I can’t trust his math because he’s cut himself off from half the information necessary to do the calculations. How can he estimate the number of vegans harmed or lost due to nutritional issues if he doesn’t let people talk about them in public?

Again, I addressed this in my answers, and argue that data of the kind you will obtain are still not enough to derive the conclusions you were deriving.

More generally, my concerns were about framing and about how much posts like this one can affect sensible advocacy and the ethical backbone of this community. There is indeed a trade-off here between transparent communications and communal dynamics, but that happens in all communities and ignoring it in ours is wishful thinking. It seems like none of my worries have been incorporated into the composition of this post, in which you have just doubled down on the framing. I think these worries could have been presented in a way healthier form without incurring in all of those framing costs, and I think its publication is net-negative due to the latter.

Comment by Martín Soto (martinsq) on UDT shows that decision theory is more puzzling than ever · 2023-09-25T14:51:04.534Z · LW · GW

Here's a link to the recording.

Here's also a link to a rough report with more details about our WIP.

Comment by Martín Soto (martinsq) on Vegan Nutrition Testing Project: Interim Report · 2023-09-22T13:07:04.940Z · LW · GW

Thank you, and sorry again for my delay!

As I understand it, our main disagreements seem to be:

  1. How many vegans, or more accurately how much general seriousness towards animal ethics, will be lost as a result of your kind of interventions.
  2. How bad it is for this to happen.

Let me start with 1.

I also think my posts deserve more responsibility for the tests and supplements than for the dietary changes. It would mean a great deal to me to have this empirical fact either explicitly disagreed with or acknowledged.

While I acknowledge these observed benefits, I do want to challenge that these observations are nearly enough to be sure the effects are net-positive, because of the following: The negative effects of the kind "everyone treats veganism less seriously, and as a result less people transition or are vocal about it" will be much more diffused, hard-to-track, and not-observed, than the positive effects of the kind "this concrete individual started vegan supplements". Indeed, I fear you might be down-playing how easy it is for people to arrive (more or less consciously) at these rationalized positions, and that's of course based on my anecdotal experience both inside and outside this community. But I am even more worried about the harder-to-pin-down communal effects, "tone setting", and the steering of very important sub-areas of the EA community into sub-optimal ethical seriousness (according to me), which is too swayed by intellectual fuzzies, instead of actual utilons.

Let me now turn to 2.

But you're right my posts probably weren't optimal for maximizing the number of vegans in the world.

Of course, I too don't optimize for "number of vegans in the world", but just a complex mixture including that as a small part. And as hinted above, if I care about that parameter it's mainly because of the effects I think it has in the community. I think it's a symptom (and also an especially actionable lever) of more general "not thinking about ethics / Sincerity in the ways that are correct". As conscious as the members of this community try to be about many things, I think it's especially easy (through social dynamics) to turn a blind eye on this, and I think that's been happening too much.

If you don't want omnivores running your vegan education, provide it yourself.

My experience is that vegans usually just have enough trouble fighting their way through people's cognitive dissonance, so that doing "public stunts" remarking health aspects is not the best ethical use of their public resources, and actually focusing on animal ethics is more positive. I do agree, of course, that in private communications (or publicly available places which are not the focus of stunts), the health measures recommended when transitioning should be explicited.

And, to be honest, that's what I've always experienced in private vegan circles, as I've mentioned before. For example, people are so conscious of B12, Iron and Omega-3 that they've just become a post-ironic meme. But it seems from your messages that this hasn't been the case in some important private spaces of the community in the past. Unfortunately, even if that is the case, I don't think it switches the balance so that "public interventions" start to be the ethically most positive. I think the most efficient way to repair those damages are still private interventions.

Meanwhile, for most existing naive vegans there's a $20 bill on the floor

And I think the most ethically positive way of picking them up is not public interventions about veganism that have other negative side-effects (that I claim are larger), but a mix of private interventions about veganism (enacted through a more effective net of support and help, for example by having specialists available to decrease the activation barrier of getting these important health actions started), and public interventions about health and care more in general. I'd be super excited to see things like that happen, but of course I know they're easier said than done, and I'm no expert in community health.

I see the omnivores I knew in 2016 who were actively pursuing information about animal suffering and reducitarianism/ameliatarianism, until bad vegan advocates ground that curiosity down.

Here I can only say "Wow, the difference in our reported experiences is big, and I don't know exactly how to account for that". My experience has been that I / friends have always had to "plant the idea" of animal exploitation maybe even being sometimes questionable. When someone comes at you having read stuff online about veganism, and interested in animal ethics, they have basically already gone past the first shallow arguments, they're already taking animal ethics seriously, and you just counsel them on practical matters (like indeed health). And not only have I found "we have always had to bootstrap this curiosity" (on people who we knew actually don't want to kill babies, so we just were presenting information or new perspectives to them), but also that vegan advocates correctly reward their way through the omnivores' well-meant advances (like reducitarianism) (while still stating, of course, that veganism is the moral baseline for them whenever it is attainable, which in my experience has been almost always).

E.g. you see all the places not offering vegan meals. I see vegan meals at Lightcone and Constellation.

To be clear, I wasn't complaining about places not serving vegan food. I was complaining about places serving meat. In my first long comment I give some thoughts on why I find this net-negative ("From the outside, I don't understand...").

Comment by Martín Soto (martinsq) on Where might I direct promising-to-me researchers to apply for alignment jobs/grants? · 2023-09-19T11:42:05.035Z · LW · GW

Center on Long-term Risk (CLR)

Comment by Martín Soto (martinsq) on Vegan Nutrition Testing Project: Interim Report · 2023-09-18T13:18:27.858Z · LW · GW

Thank your for your words.

I'd be down for a Dialogue, but indeed I cannot engage in that too soon (maybe only mid October).

But as explained in my message, I remain slightly worried about the memetic effect of your possibly publishing posts within the same framing. I completely understand you have no need to delay any publication, but because of that I'd be especially interested in ensuring that views like mine are correctly represented, and thus answering your clarifying questions seems important.

Would something like "you write the list of clarifying questions, and I write up my first-pass responses by this Saturday (even if not as deep as a Dialogue)" work for you? Completely understandable if it doesn't.

Also, I'm interested in hearing where you think the disagreement lies (out of the possibilities I've outlined above).