LessWrong 2.0 Reader

I thought OpenAI’s deep research uses the full o3? 

See the series by RogerDearnaley for a deep analysis of the problems with a superintelligence that's purely loving. In particular, specifying exactly what it loves is key, or humans are very likely going to be replaced by insects (if it's love is based on individuals) or by superintelligence (if it's love is based on cognitive capacity and sentience levels). If you want humans to remain around, it's God to love humans specifically and you've got to somehow define what about humans it's supposed to love.

Yudkowky's writings also delve in to this. I wish I had a handy reference for AGI alignment target basics.

With Stargate, there is only Abilene site and relatively concrete prospect for maybe $40bn so far, enough to build a 1 GW Blackwell training system (4e27 FLOPs models) in 2025-2026, the same scale as was announced by Musk [LW(p) · GW(p)] this week. Anthropic compute for 2026 remains opaque ("a million of some kind of chip" [LW(p) · GW(p)]), Google probably has the most in principle, but with unclear willingness to spend. Meta didn't say anything to indicate that its Richland Parish site will see 1 GW of Blackwells in 2025-2026, it remains a vague 2 GW by 2030 thing.

Can you share data on the size of PauseAI protests over time?

I would say this clearly falls outside my bet as I said "solely for sale of Marijuana" and this news release says, "were each sentenced today to 30 months in prison" and "pleaded guilty in November 2023 to conspiracy to manufacture and distribute marijuana and conspiracy to commit money laundering"

So really a no-brainer. Unless I can look at their sentencing agreement and it says they got time-served on the conspiracy to commit money laundering and their sentence to 30 months is solely for the conspiracy to manufacture and distribute marijuana count.

It seems like you've done some research on this topic now. Do you want to take me up on my bet?

edit: Also your article is for a 30 months sentence which started back in November 2023. I'd also bet that those defendants are either released right now or are very close to it.

Linking to a previous comment [LW(p) · GW(p)]: 3,000+ words of longform quotes by various folks on the nature of personal identity in a posthuman future, and hiveminds / clans, using Hannu Rajaniemi's Quantum Thief trilogy as a jumping-off point.

Hal Finney's reflections on the comprehensibility of posthumans, from the Vinge singularity discussion which took place on the Extropians email list back in the day:

Date: Mon, 7 Sep 1998 18:02:39 -0700
From: Hal Finney 
Message-Id: <199809080102.SAA02658@hal.sb.rain.org>
To: extropians@extropy.com
Subject: Singularity: Are posthumans understandable?

[This is a repost of an article I sent to the list July 21.]

It's an attractive analogy that a posthuman will be to a human as a human is to an insect.  This suggests that any attempt to analyze or understand the behavior of post-singularity intelligence is as hopeless as it would be for an insect to understand human society. Since insects clearly have essentially no understanding of humans, it would follow by analogy that we can have no understanding of posthumans.

On reflection, though, it seems that it may be an oversimplification to say that insects have no understanding of humans.  The issue is complicated by the fact that insects probably have no "understanding" at all, as we use the term.  They may not even be conscious, and may be better thought of as nature's robots, of a similar level of complexity as our own industrial machines.  Since insects do not have understanding, the analogy to humans does not work very well.  If we want to say that our facility for understanding will not carry over into the posthuman era, we need to be able to say that insect's facility for would not work when applied to humans.

What we need to do is to translate the notion of "understanding" into something that insects can do.  That makes the analogy more precise and improves the quality of the conclusions it suggests.

It seems to me that while insects do not have "understanding" as we do, they do nevertheless have a relatively detailed model of the world which they interact with.  Even if they are robots, programmed by evolution and driven by unthinking instinct, still their programming embodies a model of the world.  A butterfly makes its way to flowers, avoides predators, knows when it is hungry or needs to rest.  These decisions may be made unconsciously like a robot, but they do
represent a true model of itself and of the world.

What we should ask, then, is whether insect's model of the world can be successfully used to predict the behavior of humans, in the terms captured by the model itself.  Humans are part of the world that insects must deal with.  Are they able to successfully model human behavior at the level they are able to model other aspects of the world, so that they can thrive alongside humanity?

Obviously insects do not predict many aspects of human behavior. Still, in terms of the level of detail that they attempt to capture, I'd say they are reasonably effective.  Butterflies avoid large animals, including humans.  Some percentage of human-butterfly interactions would involve attempts by the humans to capture the butterflies, and so the butterflies' avoidance instinct represents a success of their model.  Similarly for many other insects for whom the extent of their model of humans is as "possible threat, to be avoided".

Other insects have historically thrived in close association with humans, such as lice, fleas, ants, roaches, etc.  Again, without attempting to predict the full richness of human behavior, their models are successful in expressing those aspects which they care about, so that they have been able to survive, often to the detriment of the human race.

If we look at the analogy in this way, it suggests that we may expect to be able to understand some aspects of posthuman behavior, without coming anywhere close to truly understanding and appreciating the full power of their thoughts.  Their mental life may be far beyond anything we can imagine, but we could still expect to draw some simple conclusions about how they will behave, things which are at the level which we can understand.  Perhaps Robin's reasoning based on fundamental principles of selection and evolution would fall into this category.

We may be as ants to the post singularity intelligences, but even so, we may be able to successfully predict some aspects of their behavior, just as ants are able to do with humans.

So far o3 isn't released yet, so it might be able to do it.

I've read on Twitter that while Gemini could technically indeed produce images itself, they are not as high quality as pictures created by a dedicated diffusion model. So they just let the language model in the background write a prompt for the image model and call an API function, which is hidden from the user. That this also reduces the risk of jailbreaks may be more a welcome side effect.

This is a top-level comment collecting various quotes discussing the posthuman condition.