LessWrong 2.0 Reader

I agree with you that as stated the analogy is risking dangerous superficiality.

• The 'cognitive' work of evolution came from the billion years of evolution in the innumerable forms of life that lived, hunted and reproduced through the eons. Effectively we could see evolution-by-natural selection as a something like a simple, highly-parallel, stochastic, slow algorithm. I.e. a simple many-tape random Turing machine taking a very large number of timesteps.

• A way to try and maybe put some (vegan) meat on the bones of this analogy would be to look at conditional KT-complexity. KT-complexity is a version of Kolmogorov complexity that also accounts for the time- cost of running the generating program.
  - In KT-complexity pseudorandomness functions just like randomness. 

  - Algorithms may indeed be copied and the copy operation is fast and takes very little memory overhead.
  - Just as in Kolmogorov complexity we rejiggle and think in terms of an algorithmic probability.
  - a private-public key is trivial in a pure Kolmogorov complexity framework but correctly modelled in a KT-complexity framework.

To deepen the analogy with thermodynamics one should probably carefully read John Wentworth's generalized heat engines [LW · GW] and Kolmogorov sufficient statistics. 

If you're mobile (able to be in the UK) and willing to try a different lifestyle, consider going to the EA hotel aka CEEALAR, they offer free food and accomodation for a bunch of people, including many people working on AI safety. Alternatively, taking a quick look at https://www.aisafety.com/funders, the current best options are maybe LTFF, OpenPhil, CLR, or maybe AE Studios?

Did some more tinkering with this scenario. It is remarkably difficult to be 100% confident when determining the basic mechanics of this scenario, i.e.

And I'm beginning to suspect that

I admit I don't have a good answer for that.

(I suspect there may be something important in real life that is missing from this model.)

yeah I was mostly thinking neutral along the axis of "safey-ism" vs "accelerationism" (I think there's a fairly straight-forward right-wing bias on X, further exasperated by Bluesky)

Can you show how a repeated version of this game results in overall better deals for the company? I agree this can happen, but I disagree for this particular circumstance.

Fascinating post, like a version of prions on an orthogonal axis, but with way more possibility to expand. 

Wonder if other fields have a version of this, where by inverting or adjusting a basic principle you unlock a whole field as rich and diverse as the original. Perhaps imaginary numbers are analogous?

If you have another formal definition of "rational", I'm happy to help extrapolate what you're trying to predict.  Decision theories are a different level of abstraction than terminal rationality and goal coherence.

I don't think Twitter/X is really a very neutral space in general. I agree it maybe is more neutral from the perspective of the AI/ML community. (And certainly feels that way from the pespective of many people in that community.)

The problem with words like "better" and "left on the table" is that better globally is not necessarily better for me, and leaving value on the table is not a problem for me if it all comes from your part.

Option A: I get 5 units of utility, you get 5 units of utility.

Option B: I get 6 units of utility, you get 3 units of utility, 1 unit of utility gets burned.

Option A is clearly better for me out of these two.

It seems like there should be an option C: 6.5 units of utility for me, 3.5 units of utility for you. But maybe there isn't . Maybe it actually costs me 1 unit of utility to keep a gun pointing at your head. And without the gun, you might turn the table and demand 5 units of utility or bust.

In theory, if you were a robot, you could commit to act as if the gun is pointed at your head, even if there is none, and then we could split the money I saved by not having to buy the gun. In practice, humans don't seem to work that way sufficiently reliably.