LessWrong 2.0 Reader

Yes, though this often involves some self deception about your true utility function. I suspect that some ace people did this to themselves to avoid zero sum competition they expect to painfully lose.

Of course for a "real" prisoners dilemma any form of coordination is ruled out from the start. But in real world instances, coordination can sometimes be introduced into systems that previously were prisoner's dilemmas. That's what I mean with "solving" a prisoner's dilemma. Making the dilemma go away.

The thing I'm pointing out here is that "coordination" is a very unspecific term, and one concrete form of coordination is being able to vote for cooperation. (Example: voting on a climate change bill instead of trying minimize your personal carbon footprint, which would make you personally significantly worse of with hardly any benefit on the whole, which is why you would defect but vote on cooperate.) I think voting is usually not appreciated as a method of coordination, only as a method of choosing the most popular policy/party, which doesn't need to involve solving a prisoner's dilemma.

Interesting, thanks! I do think RLHF makes the models worse at truesight^[1], see examples in other comments. But the gap isn't so big that it can't be caught up to eventually - what matters is how well you can predict future capabilities. Though, I would bet that the gap is still big enough that it would fail on the majority of examples from GPT-4-base (two years old now).

1. ^{^}

   I say "worse" for lack of a better concise word. I think what's really happening is that we're simply measuring two slightly different tasks in both cases. Are we measuring how well a completion model understands the generator of text it needs to produce, or how well a chat assistant can recognize the author of text?

Thanks for putting this up! Just to double check—there aren't any restrictions against doing multiple AISC projects at the same time, right?

Maybe a bit of a nitpick, but RLHF'd GPT-4o can still detect Eric Drexler's writing (chat link). I gave it the first paragraph of his latest blog post, which was written in February 2024, past 4o's knowledge cutoff date of October 2023. In general I'm not sure if RLHF actually makes the models worse at truesight. It would be interesting to see a benchmark comparing e.g. Llama base vs instruct on this capability.

With metaculus I do write my confidence in pretty much every forecast that I make. It has made this better. But even then, when coming back to long term forecasts, I sometimes find myself stunned by my specific %, likely forgetting some of the informations that brought me to that forecast in the first place.

Note that the game-theoretic "true" prisoner's dilemma is formulated to make coordination (both communication and outside-of-game considerations like reputation, side-payments, self-image, etc.) are ignored.  All of the non-Nash "solutions" are by introducing factors into the game that change the game very significantly.

Voting (especially when defectors aren't penalized, just forced to cooperate) is a pretty big variation, and needs to be explicitly modeled in order to determine what equilibra are rational.  

This is almost unrelated to real-world voting, which has SO MANY complicating and interfering factors that simple models just don't tell us much.

During the period when I was writing down dreams, I was also not using an alarm. I did train myself to wake up instantly, at the time that I wanted, to the minute. I agree that the half-awake state is anti-helpful for remembering dreams.

The alarm is helpful for starting out though.

It helps to know the contexts in which the term is often used, how it is motivated, and where the term probably came from. Thus, it is important to know the term in relation to other terms. 

I haven't heard the term "epistemic state" used often in the community, but depending on the context in which the person used the term, I imagine the person using it meant their overall "epistemic status [EA · GW]" for a collection of claims (or their confidence in any given claim). 

An epistemic status describes how confident a person is in a particular argument or idea. If the claim is important enough it is denoted with a percentage value that represents the probability that the cliam is true (from the person's point of view). This is a good practice of reasoning transparency, because "plausible" can often be interpreted to mean different degrees of confidence. 

As an example of how lacking transparency can lead to falure, when JFK questioned the Joint Chief of Staff about the chances of his plan's success for the invasion of Cuba, the response was a "fair chance" of success, so JFK invaded. It was only later that "fair chance" was explained to mean a 1/3 chance of success. 

As for the word epistemic more generally, it is indeed related to knowledge, but within the context of the Effective Altruism and lesswrong community, we care about the quality of the knowledge, not just quantity (since an effectiveness mindset [EA · GW] leads to more positive change). This is why Epistemic hygiene [? · GW] (I hear epistemics used synonymously within the community, and it's generally the word I use more) is emphasised so much (and why the sequences [? · GW] are so popular). It is very important for accurate beliefs to spread, so figuring out how to develop a set of community values that encourages accurate beliefs is what lesswrong is largely about. 

I would say that the community is focused on how to have good epistemics in practice, rather than exploring foundations, although it does have that [LW · GW]. The foundational roots of the term epistemics likely comes from the field of epistemology (this link leaves out some stuff, but it's an okay starting point). 

Hello Everyone!

I am a Brazilian AI/ML engineer and data scientist, I have been following the rationalist community for around 10 years now, originally as a fan of Scott Alexander's Slate Star Codex where I came to know of Eliezer and Lesswrong as a community, along with the rationalist enterprise. 

I only recently created my user and started posting here, currently, I’m experiencing a profound sense of urgency regarding the technical potential of AI and its impact on the world. With seven years of experience in machine learning, I’ve witnessed how the stable and scalable use of data can be crucial in building trustworthy governance systems. I’m passionate about contributing to initiatives that ensure these advancements yield positive social outcomes, particularly for disadvantaged communities. I believe that rationality can open paths to peace, as war often stems from irrationality.

I feel privileged to participate in the deep and consequential discussions on this platform, and I look forward to exchanging ideas and insights with all the brilliant writers and thinkers who regularly contribute here. 

Thank you all!