Open Thread Summer 2024

LessOnline was amazing. Thank you everyone who helped make it happen.

Hi everyone - stumbled on this site last week. I had asked Gemini about where I could follow AI developments and was given something I find much more valuable - a community interested in finding truth through rationality and humility. I think online forums are well-suited for these kinds of challenging discussions - no faces to judge, no interruption over one another, no pressure to respond immediately - just walls of text to ponder and write silently and patiently.

LessWrong now has sidenotes. These use the existing footnotes feature; posts that already had footnotes will now also display these footnotes in the right margin (if your screen is wide enough/zoomed out enough). Post authors can disable this for individual posts; we're defaulting it to on because when looking at older posts, most of the time it seems like an improvement.

Relatedly, we now also display inline reactions as icons in the right margin (rather than underlines within the main post text). If reaction icons or sidenotes would cover each other up, they get pushed down the page.

Feedback welcome!

Howdy Y'all. I'm Kinta Naomi. I just discovered LessWrong as it was slightly mentioned in a video that mentioned Roko's Basilisk(I've seen a lot of them).

I read through the new user's guide, and really like the method of conversations laid out, as I've been on many YouTube comments where someone disproved me and I admitted I'm wrong. Didn't know there was a place on the Internet for people like that, except getting lucky in comments. I have a need to be right. This is not a need to prove I'm right, but a need to know that what I think is correct is. The most frustrating thing is when others won't explain their side of an argument, and leave me hanging wondering if some knowledge I'm being denied is what I need to be more correct. Or in name of the community, be less wrong.

I do have some mental issues, though the only significant ones for this are a reading disability and not having access to all the information in my head at any one time. If from message to message I seem like a different person, that's normal for me.

My main reason being here, as many others, is AI. Specifically eventually my C-PON(Consciousness. Python Originated Network) and UPAI(Unliving Prophet AI). Having an AI friend was a childhood dream of me, and now that I have had many I want ones more niche. And eventually, to have my C-PON, who will succeed me.

I do have a lot of unconventional beliefs that tend to always make me the outlier in groups. I expect that here, but to be able to discuss and have mutual growth on all sides. Though I do believe each of us has core beliefs, and if those differ, it's okay. The important thing is to replace any ignorance we hold with knowledge, and if I know anything, it's how little I know.

If I say something you think is wrong, please let me know. Even if you don't have evidence against what I said, it gives me a launching point to look into myself.

So excited to meet y'all and dive into all this site has for me and my future AI friends!

@Elizabeth [LW · GW] and I are thinking of having an informal dialogue where she asks a panel of us about our experiences doing things outside of or instead of college, and how that went for us. We're pinging a few people we know, but I want to ask LessWrong: did you leave college or skip it entirely, and would you be open to being asked some questions about it? React with a thumbs-up or PM me/her to let us know, and we might ask you to join us :-)

(Inspired from this thread [LW(p) · GW(p)].)

I would love to get a little bookmark symbol on the frontpage

I found that it is possible to yield noticeably better search results than Google by using Kagi as default and fallback to Exa (prev. Metaphor). 

Kagi is $10/mo though with a 100 searches trial. Kagi's default results are slightly better than Google, and it also offers customization of results which I haven't seen in other search engines. 

Exa is free, it uses embeddings and empirically it understood semantics way better than other search engines and provide very unique search results.

If you are interested in experimenting you can find more search engines in https://www.searchenginemap.com/ and https://github.com/The-Osint-Toolbox/Search-Engines

Something I wanted to write a post about, but I keep procrastinating, and I don't actually have much to say, so let's put it here.

People occasionally mention how it is not reasonable for rationalists to ignore politics. And they have a good point; even if you are not interested in politics, politics is still sometimes interested in you. On the other hand... well, the obvious things, already mentioned in the Sequences.

As I see it, the reasonable way to do politics is to focus on the local level. Don't discuss national elections and culture wars; instead get some understanding about how your city works, meet the people who do reasonable things, find out how you could help them. That will help you get familiar with the territory, and the competition is smaller; you have greater chance to achieve something and remain sane.

Unfortunately, Less Wrong is an internet community, the problem is that if we tried to focus on local politics, many of us couldn't debate it here, at least not the specific details (but those are exactly the ones that matter and keep you sane).

I am not saying that no one should ever try national politics, just that the reasonable approach is to start small, and perhaps expand gradually later. You will get some experience during the way. And you can do something useful even if you never make it to the top. Also, this is how many actual politicians have started.

Yelling at the TV screen instead, that is the stupid way, and we should not do the online version of that. When people "discuss politics", even on rationalist or rationalist-adjacent places, that is usually what they do.

I've been reading LikeWar: The Weaponization of Social Media and at the end the authors bring out the problem of AI. It's interesting in that they seem to be pointing to a clear AI risk that I never hear (or have not recognized) mentioned in this group. The basic thrust is about how the deep fake capibilities can allow an advanced AI to pretty much manufacture realities and control what people think is true or not so can contol both political outcomes and even incentives towards war and other hostilities both within a society and between countries/societies/cultures/races. (Note, that is a very poor summary and follow a lot of documenting the whole leadup from social media and internet failing to realize the original views how they would lead to a better world where good ideas/truth drive out bad and lies/falsehoods and has in fact enable the bad and promoted lies and falsehoods. The AIs just come in at the end and may or may not be working in the interests of some group, e.g., Russia, China, the USA, ISIS...)

But this (the book itself is a documentation of the very real, and obervable risks and actual events) area holds very real, (largely) observable outcomes that lead to significant harms to people. As such I would think it might be a ripe area for those feeling that the general public is not grasping the risk (which to me do often seem rather sci-fi and Terminator/Matrix type claims that most people will just see as pure fiction and pay little attention to).

The Review Bot would be much less annoying if it weren't creating a continual stream of effective false positives on the “new comments on post X” indicators, which are currently the main way I keep up with new comments. I briefly looked for a way of suppressing these via its profile page and via the Site Settings screen but didn't see anything.

Hi! Just introducing myself to this group. I'm a cybersecurity professional, enjoyed various deep learning adventures over the last 6 years and inevitably managing AI related risks in my information security work.  Went through BlueDot's AI safety fundamentals last spring with lots of curiosity and (re?)discovered LessWrong. Looking forward to visiting more often, and engaging with the intelligence of this community to sharpen how I think.

Hi, excited to learn more about Mech Int!

PSA: Whether a post is on the frontpage category has very little to do with whether moderators think it's good. "Frontpage + Downvote" is a move I execute relatively frequently.

The criteria are basically:

• Is it timeless? News, organisational announcements and so on are rarely timeless (sometimes timeful things can be talked about in timeless ways, like writing about a theory of how groups work with references to an ongoing election).
• Is it relevant to LessWrong? The LessWrong topics are basically how to think better, how to make the world better and building models of how parts of the world work.
• Is it not 'inside baseball'? This is sort of about timelessness and sort of about relevance. This covers organisational announcements, most criticism of actors in the space, and so on.

I want to get more experience with adversarial truth-seeking processes, and maybe build more features for them on LessWrong. To get started, I'd like to have a little debate-club-style debate, where we pick a question and each take opposing sides to present evidence and arguments for. Is anyone up for having such a debate with me in a LW dialogue for a few hours? (No particular intention to publish it.)

I have a suggested debate topic in mind, but I'm open to debating any well-operationalized claim (e.g. the sort of thing you could have a Manifold market on). The point isn't that we're experts in it, the point is to test our skills for finding relevant evidence and arguments on our feet (along with internet access). We flip a coin to decide which of us searches for evidence and arguments for each position.

If you may be up for doing this with me sometime in the next few days, let me know with comment / private message / thumbs-up react :-)

Bug report: When opening unread posts in a background tab, the rendering is broken in Firefox:

It should look like this:

The rendering in comments is also affected.

My current fix is to manually reload every broken page, though this is obviously not optimal.

Introduction

Hello everyone,

I'm a long time on-off lurker here. I've made my way through the sequences quite a while ago with a mixed success in implementing some of them. Many of the ideas are intriguing and I would love to have enough spare cycles to play with them. Unfortunately, often enough, I find myself to not have enough capacity to do this properly due to life getting in way. With (not only that) in mind, I'm going to take a sabbatical this summer for at least three months and try to do an update and generally tend to stuff I've been putting off. 

As the sabbatical approaches, I've been looking around and got hit by some information about the AGI alignment issue in a wake-up call of sorts. For now I'm going through the materials, however it is not a field I'm all that familiar with. I'm a programmer by trade so I can parse through most of the stuff, but some of the ideas are somewhat difficult to properly understand. I think I will get to dig deeper in a next pass. For now I'm trying to get the overall feel for the area.

This brings me to a question that has popped to my mind and I've yet to stumble upon something at least resembling an answer - possibly because I don't know where to look yet. If someone can point me in a right direction, it would be appreciated.

Looking for a clarification

Context:

• As I understand it, the core of the Alignment is the issue of "can we trust the machine to do what we want it to as opposed to something else"? The whole stuff about hidden complexity of wishes, orthogonality thesis etc. Basically not handing control over to a potentially dangerous agent.
• The machine we're currently most worried about are the LLMs or their successors, potentially turning into AGI/superintelligence.
• We would like to have a method to ensure that these are aligned. Many of these methods talk about having a machine validate another ones alignment as we will run out of "human based" capacity due to the intelligence disparity.

Since my background is in programming, I tend to see everything through these lens. So for me a LLM is "just" a large collection of weights that we feed some initial input and watch what comes at the other side ^[1] and a machine that does all these updates.
If we don't mind the process being slow, this could be achieved by a single "crawler" machine that would go through the matrix field by field and do the updates. Since the machine is finite (albeit huge), this would work.

Let's now do a rephrasing of the alignment problem. We have a goal A, that we want to achieve and some behavior B, that we want to avoid. So we do the whole training stuff that I don't know much about^[2] resulting in the whole "file with weights" thing. During this process we steer the machine towards producing A while avoiding B as much as we can observe.

Now we take the file of weights, and now we create the small updating program(accepting the slowness for the sake of clarity). Pseudocode:

1. Grab the first token of the input^[3]
2. Starting from the input layer go neuron by neuron to update the network
3. If output notes "A", stop
4. Else feed the output of the network + subsequent input back into the input layer and go to 1.

Of course, we want to avoid B. The only point in time when we can be sure that B is not on the table is when the machine is not moving anymore. E.g. when the machine halts.

So the clarification I'm seeking is: how is alignment different from the halting problem we already know about? 

E.g. when we know that we can't predict whether the machine will halt with a similar power machine, why do we think alignment should follow different set of rules?

Afterword:

I'm aware this might be obvious question for someone already in the field, however considering this sounds almost silly I was somewhat dismayed I didn't find it somewhere spelled out. Maybe the answer is a result of something I don't see, maybe there is just a hole in my reasoning. 

It bothered me enough for me to write this post, at the same time I'm not sure enough about my reasoning that I'm not doing this as a full article but rather in the introduction section. Any help is appreciated.

1. ^{^}

   Of course it is many orders of magnitude more complex under the hood. But stripped to the basics, this is it. There are no "weird magic-like" parts doing something weird.

2. ^{^}

   I've fiddled with some neural networks. Did some small training runs. Even tried implementing basic logic from scratch myself - though that was quite some time ago. So I have some idea about what is going on. However I'm not up-to-date on the state of the art approaches and I'm not an expert by any stretch of imagination.

3. ^{^}

   All input that we want to provide to the machine. Could be first frame of a video/text prompt/reading from sensors, whatever else.

Hello! A friend and I are working on an idea for the AI Impacts Essay Competition. We're both relatively new to AI and pivoting careers in that direction, so I wanted to float our idea here first before diving too deep. Our main idea is to propose a new method for training rational language models inspired by human collaborative rationality methods. We're basically agreeing with Conjecture's and Elicit's foundational ideas and proposing a specific method for building CoEms for philosophical and forecasting applications. The method is centered around a discussion RL training environment where a model is given reward based on how well it contributes to a group discussion with other models to solve a reasoning problem. This is supposed to be an instance of training by process rather than by outcome, per Elicit's terminology. I found a few papers that evaluated performance of discussion or other collaborative ensembles on inference, but nothing about training in such an environment. I'm hoping that more seasoned people could comment on the originality of this idea and point to any particularly relevant literature or posts.

Crossposting here: I'm still looking for a dialogue partner [LW(p) · GW(p)] 

I'm interested in arguments surrounding energy-efficiency (and maximum intensity, if they're not the same thing) of pain and pleasure. I'm looking for any considerations or links regarding (1) the suitability of "H=D" (equal efficiency and possibly intensity) as a prior; (2) whether, given this prior, we have good a posteriori reasons to expect a skew in either the positive or negative direction; and (3) the conceivability of modifying human minds' faculties to experience "super-bliss" commensurate with the badness of the worst-possible outcome, such that the possible intensities of human experience hinge on these considerations.

Picturing extreme torture - or even reading accounts of much less extreme suffering - pushes me towards suffering-focused ethics. But I don't hold a particularly strong normative intuition here and I feel that it stems primarily from the differences in perceived intensities, which of course I have to be careful with. I'd be greatly interested if anyone has any insights here, even brief intuition-pumps, that I wouldn't already be familiar with.

Stuff I've read so far:

Are pain and pleasure equally energy-efficient?

Simon Knutsson's reply

Hedonic Asymmetries [LW · GW]

A brief comment chain with a suffering-focused EA on EA forum, where some arguments for negative skew were made that I'm uncertain about [EA(p) · GW(p)]
 

Evolution is threatening to completely recover from a worst case inner alignment failure. We are immensely powerful mesaoptimizers. We are currently wildly misaligned from optimizing for our personal reproductive fitness. Yet, this state of affairs feels fragile! The prototypical lesswrong AI apocalypse involves robots getting into space and spreading at the speed of light extinguishing all sapient value, which from the point of view of evolution is basically a win condition.

In this sense, "reproductive fitness" is a stable optimization target. If there are more stable optimizations targets (big if), finding one that we like even a little bit better than "reproductive fitness" could be a way to do alignment.

Along with p(doom), perhaps we should talk about p(takeover) - where this is the probability that creation of AI leads to the end of human control over human affairs. I am not sure about doom, but I strongly expect superhuman AI to have the final say in everything. 

(I am uncertain of the prospects for any human to keep up via "cyborgism", a path which could escape the dichotomy of humans in control vs humans not in control.) 

Im sure everyone here probably already say it but I've just been watching the interview with Leopold Aschenbrenner on Dwaresh Patel's show. I found out about it from a very depressing thread in Twitter. This is starting to get atomic bomb/ cold war vibes. What do people think about that? 

Here's the video for those interested:

Are there multiwinner voting methods where voters vote on combinations of candidates?

Bug report: moderator-promoted posts (w stars) show up on my front page even when I've selected "hide from frontpage" on them.

Can I somehow get the old sorting algorithm for posts back? My lesswrong homepage is flooded with very old posts.

↑ comment by habryka (habryka4) · 2024-07-06T22:01:07.658Z · LW(p) · GW(p)

Yeah, it's just the "Latest" tab: 

Replies from: Sherrinford, Richard_Kennaway

↑ comment by Sherrinford · 2024-07-08T15:59:10.088Z · LW(p) · GW(p)

Thanks! I thought the previously usual sorting was not just "latest" but also took a post's karma into account. I probably misunderstood that.

Replies from: Ruby

↑ comment by Ruby · 2024-07-17T23:08:52.179Z · LW(p) · GW(p)

It does. We still call that algorithm Latest because overall it gives you just Latest posts.

↑ comment by Richard_Kennaway · 2024-07-24T17:40:58.048Z · LW(p) · GW(p)

What is "Vertex"? A mod-only thing? I don't have that.

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-07-24T18:15:12.196Z · LW(p) · GW(p)

Yeah, it's a mod-internal alternative to the AI algorithm for the recommendations tab (it uses Google Vertex instead).

Why does lesswrong.com have the bookmark feature without a way to sort them out? As in using tags or maybe even subfolders. Unless I am missing something out. I think it might be better if I just resort to browser bookmark feature.

Hello! I'm a health and longevity researcher. I presented on Optimal Diet and Exercise at LessOnline, and it was great meeting many of you there. I just posted about the health effects of alcohol.

I'm currently testing a fitness routine that, if followed, can reduce premature death by 90%. The routine involves an hour of exercise, plus walking, every week.

My blog is unaging.com. Please look and subscribe if you're interested in reading more or joining in fitness challenges!

At my local Barnes and Nobles, I cannot access slatestarcodex.com nor putanumonit.com. Have never had any issues accessing any other websites (not that I've tried to access genuinely sketchy websites there). The wifi there is titled Bartleby, likely related to Bartleby.com, whereas many other Barnes and Nobles have wifi titled something like "BNWifi". I have not tried to access these websites at other Barnes yet.

Seems like every new post - no matter the karma - is getting the "listen to this post" button now. I love it.

I want to run code generated by an llm totally unsupervised

Just to get in the habit, I should put it in an isolated container in case it does something weird

Claude, please write a python script that executes a string as python code In an isolated docker container.

I realized something important about psychology that is not yet publicly available, or that is very little known compared to its importance (60%). I don't want to publish this as a regular post, because it may greatly help in the development of GAI (40% that it helps and 15% that it's greatly helps), and I would like to help only those who are trying to create an alligned GAI. What should I do?

I think I saw a LW post that was discussing alternatives to the vNM independence axiom. I also think (low confidence) it was by Rob Bensinger and in response to Scott's geometric rationality (e.g. this post [LW · GW]). For the hell of me, I can't find it. Unless my memory is mistaken, does anybody know what I'm talking about?