Open Thread Fall 2024

Hello everyone!

I am new here and I thought I should introduce myself, I am currently reading the highlights of the sequences and it has been giving me a sharper worldview and  I do feel myself being more rational, I think a lot of people who call themselves rational are motivated by biases and emotions more than they think, but it is good to be aware of that and try to work to be better, so I am doing that.

I am 17 years old from Iraq, I found the forum through Daniel Schmachtenberger, I am not sure how known he is here.

I am from a very Muslim country, I was brainwashed by it growing up like most people, but at 11 I started questioning and reading books as well, which was very hard, since the fear of "hell" is imprinted in someone growing up in this environment, but by 14 I broke free, I had a three months existential crisis as a result where I felt like I don't exist, and was in anxiety 24/7.

At that point I got interested in the new age movement, eastern religions and spirituality, especially Buddhism and certain strands of Hinduism, I wasn't interested in them to take as dogmas or as absolute views, I also got into western philosophy later, especially the Idealism vs. Realism debate, I liked Hegel, Spinoza, and Thomas Kuhn the most, but I can't say I read their books directly, I read books about their books, which are called secondary sources.

I tend to view all worldviews as incomplete lenses, whether that's the religious paradigm, or the scientific paradigm, or the spiritual paradigm, or different schools of philosophy, the true rational person in my opinion has the capability to understand all lenses in depth and switch between them, be aware of self-deception and constantly question ideas.

Of course, one must also come to conclusions, so there needs to be a certain balance.

I am going to study computer engineering or cybersecurity engineering, and I plan to study AI in masters and other studies, I am currently trying to leave this third world country, as life is difficult here and especially being treated as an outcast because I don't believe in religion dogmatically.

Of course, I am open to any of beliefs being challenged, that's why I am here :) 

I would also appreciate anyone wanting to share good resources, or maybe advice for me at this age.

Hi! I joined LW in order to post a research paper that I wrote over the summer, but I figured I'd post here first to describe a bit of the journey that led to this paper.

I got into rationality around 14 years ago when I read a blog called "you are not so smart", which pushed me to audit potential biases in myself and others, and to try and understand ideas/systems end-to-end without handwaving.

I studied computer science at university, partially because I liked the idea that with enough time I could understand any code (unlike essays, where investigating bibliographies for the sources of claims might lead to dead ends), and also because software pays well. I specialized in machine learning because I thought that algorithms that could make accurate predictions based on patterns in the world that were too complex for people to hardcode were cool. I had this sense that somewhere, someone must understand the "first principles" behind how to choose a neural network architecture, or that there was some way of reverse-engineering what deep learning models learned. Later I realized that there weren't really first principles regarding optimizing training, and that spending time trying to hardcode priors into models representing high-dimensional data was less effective than just getting more data (and then never understanding what exactly the model had learned).

I did a couple of kaggle competitions and wanted to try industrial machine learning. I took a SWE job on a data-heavy team at a tech company working on the ETLs powering models, and then did some backend work which took me away from large datasets for a couple years. I decided to read through recent deep learning textbooks and re-implement research papers at a self-directed programming retreat. Eventually I was able to work on a large scale recommendation system, but I still felt a long way from the cutting edge, which had evolved to GPT-4. At this point, my initial fascination with the field had become tinged with concern, as I saw people (including myself) beginning to rely on language model outputs as if they were true without consulting primary sources. I wanted to understand what language models "knew" and whether we could catch issues with their "reasoning."

I considered grad school, but I figured I'd have a better application if I understood how ChatGPT was trained, and how far we'd progressed in reverse engineering neural networks' internal representations of their training data.

I participated in the AI Safety fundamentals course which covered both of these topics, focusing particularly on the mechanistic interpretability section. I worked through parts of the ARENA curriculum, found an opportunity to collaborate on a research project, and decided to commit to it over the summer, which led to the paper I mentioned in the beginning! Here it is. [LW · GW]

Hello everyone! 

My name is José, 23 years old, brazilian and finishing (in July) an weird interdisciplinary undergraduate in University of Sao Paulo (2 years of math, physics, computer science, chem and bio + 2 years of do whatever you want - I did things like optimization, measure theory, decision theory, advanced probability, bayesian inference, algorithms, etc.)

I've been reading stuff in LW about AIS for a while now, and took some steps to change my career to AIS. I met EA/AIS via a camp focused on AIS for brazilian students called Condor Camp in 2022 and since then participated in a bunch of those camps, created a uni group, ML4Good, bunch of EAGs/Xs. 

I recently started an Agent Foundations fellowship by Alex Altair and am writing a post about Internal Model Principle. I expect to release it soon! 

Hope you all enjoy it!

I'm planning to run the unofficial LessWrong Community Census again this year. There's a post with a link to the draft and a quick overview of what I'm aiming for here [LW · GW], and I'd appreciate comments and feedback. In particular, if you

• Have some political questions you want to get into detail with or
• Have experience or opinions on the foundational skills of rationality and how to test them on a survey

then I want to hear from you. I care a lot about rationality skills but don't know how to evaluate them in this format, but I have some clever ideas if I had a signal I could sift out of the survey. I don't care about politics, but lots of people do and I don't want to spoil their fun.

You can also propose other questions! I like playing with survey data :) 

I found the site a few months ago due to a link from an AI themed forum.  I read the sequences and developed the belief that this was a place for people who think in ways similar to me.  I work as a nuclear engineer.  When I entered the workforce, I was surprised to find that there weren’t people as dispositioned toward logic as I was.  I thought perhaps there wasn’t really a community of similar people and I had largely stopped looking.

This seems like a good place for me to learn, for the time being.  Whether or not this is a place for me to develop community remains to be seen. The format seems to promote people presenting well-formed ideas.  This seems valuable, but I am also interested in finding a space to explore ideas which are not well-formed.  It isn’t clear to me that this is intended to be such a space.  This may simply be due to my ignorance of the mechanics around here. That said, this thread seems to be inviting poorly formed ideas and I aim to oblige.

There seem to be some writings around here which speak of instrumental rationality, or “Rationality Is Systematized Winning”.  However, this seems to beg the question: “At what scale?”  My (perhaps naive) impression is that if you execute instrumental rationality with an objective function at the personal scale it might yield the decision that one should go work in finance and accrue a pile of utility. But if you apply instrumental rationality to an objective function at the societal scale it might yield the decision to give all your spare resources to the most effective organizations you can find.  It seems to me that the focus on rationality is important but doesn’t resolve the broader question of “In service of what?” which actually seems to be an important selector of who participates in this community.  I don’t see much value in pursuing Machiavellian rationality and my impression is that most here don’t either.  I am interested in finding additional work that explores the implications of global scale objective functions.

On a related topic, I am looking to explore how to determine the right scale of the objective function for revenge (or social correction if you prefer a smaller scope).  My intuition is that revenge was developed as a mechanism to perform tribal level optimizations.  In a situation where there has been a social transgression, and redressing that transgression would be personally costly but societally beneficial, what is the correct balance between personal interest and societal interest?

My current estimate of P(doom) in the next 15 years is 5%.  That is, high enough to be concerned , but not high enough to cash out my retirement. I am curious about anyone harboring a P(doom) > 50%.  This would seem to be high enough to support drastic actions.  What work has been done to develop rational approaches to such a high P(doom)?

This idea is quite poorly formed, but I am interested in exploring how to promote encapsulation, specialization, and reuse of components via the cost function in an artificial neural network. This comes out of the intuition that actions (things described by verbs, or transforms) may be a primitive in human mental architecture and are one of the mechanisms by which analogical connections are searched.  I am interested in seeing if continuous mechanisms could be defined to promote the development of a collection of transforms which could be applied usefully across multiple different domains.  Relatedly, I am also interested in what an architecture/cost function would need to look like to promote retaining multiple representations of a concept with differing levels of specificity/complexity.

I think there is a 10-20 per cent chance we get digital agents in 2025 that produce a holy shit moment as big as the launch of chatgpt.

If that happens I think that will produce another round of questions that sounds approximately like “how were we so unprepared for this moment”.

Fool me once, shame on you…

I don't know exactly when this was implemented, but I like how footnotes appear to the side of posts.

I am a university dropout that wants to make an impact in the AI safety field. I am a complete amateur in the field, just starting out, but I want to learn as much as possible in order to make an impact. I studied software engineering for a semester and a half before realizing that there was a need for more people in the AI safety field, and that's where I want to give all my attention. If you are interested in connecting DM me, if you have any advice for a newcomer post a comment below. I am located in Hønefoss, Norway.

Hello! I've just found out about Lesswrong and I immediately feel at home. I feel this is what I was looking for in medium.com and I never found there; a website to learn about things, about improving oneself and about thinking better. Medium proved to be very useful at reading about how people made 5 figures using AI to write articles for them, but not so useful at providing genuinely valuable information.

One thing I usually say about myself is that I have "learning" as a hobby. I have only very recently given a name to things and now I know that it's ADHD I can thank for my endless consumption of information about seemingly unrelated topics. I try (good thing PKMs exist!) to give shape to my thoughts and form them into something cohesive, but this tends to be a struggle. 

If anyone has ideas on how to "review" what already sits in your mind to create new connections between ideas and strengthen thoughts, they'd be more than welcome. 

Site update: the menu bar is shorter!

Previously I found it overwhelming when I opened it, and many of the buttons were getting extremely little use. It now looks like this.

If you're one of the few people who used the other buttons, here's where you can find them:

• New Question: If you click on "New Post", it's one of the types of post available at the top of the page.
• New Dialogue: If you go to any user page, you can see an option to invite them to a dialogue at the top, next to the option to send them a message or subscribe to their posts.
• New Sequence: You can make your first when you scroll down the Library [? · GW] page, and once you've got one you can also make a new one from your profile page.
• Your Quick Takes: Your shortform post is pinned to the top of your posts on your profile page.
• Bookmarks: This menu icon will re-appear as soon as you have any bookmarks, which you set in the same way (from the triple-dot menu on posts.

Hello Everyone!

I am a Brazilian AI/ML engineer and data scientist, I have been following the rationalist community for around 10 years now, originally as a fan of Scott Alexander's Slate Star Codex where I came to know of Eliezer and Lesswrong as a community, along with the rationalist enterprise. 

I only recently created my user and started posting here, currently, I’m experiencing a profound sense of urgency regarding the technical potential of AI and its impact on the world. With seven years of experience in machine learning, I’ve witnessed how the stable and scalable use of data can be crucial in building trustworthy governance systems. I’m passionate about contributing to initiatives that ensure these advancements yield positive social outcomes, particularly for disadvantaged communities. I believe that rationality can open paths to peace, as war often stems from irrationality.

I feel privileged to participate in the deep and consequential discussions on this platform, and I look forward to exchanging ideas and insights with all the brilliant writers and thinkers who regularly contribute here. 

Thank you all!

Does someone have a guesstimate of the ratio of lurkers to posters on lesswrong? With 'lurker' defined as someone who has a habit of reading content but never posts stuff (or posts only clarification questions)

In other words, what is the size of the LessWrong community relative to the number of active contributors?

If spaced repetition is the most efficient way of remembering information, why do people who learn a music instrument practice every day instead of adhering to a spaced repetition schedule?

I've been lurking for years. I'm a lifelong rationalist who was hesitant to join because I didn't like HPMOR. (Didn't have a problem with the methods of rationality; I just didn't like how the characters' personalities changed, and I didn't find them relatable anymore.) I finally signed up due to an irrepressible urge to upvote a particular comment I really liked.

I struggle with LW content, tbh. It takes so long to translate it into something readable, something that isn't too littered with jargon and self-reference to be understandable for a generalist with ADHD. By the time I've spent 3 hours doing that for any given "8 minute" article, I'm typically left thinking, "No shit, Sherlock. Thanks for wasting another 3 hours of my life on your need to prove your academic worth to a community of people who talk like they still think IQ is an accurate way to assess intelligence."

AI helps, but still, it's just... ugh. How do I find the good shit? I want to LEARN MORE FASTER BETTER. 

Halp, pls.

Re: the new style (archive for comparision)

Not a fan of

1. the font weight, everything seem semi-bolded now and a little bit more blurred than before. I do not see myself getting used to this.

2. the unboxed karma/argeement vote. It is fine per se, but the old one is also perfectly fine.

Edit: I have to say that the font on Windows is actively slightly painful and I need to reduce the time spent reading comments or quick takes.

↑ comment by habryka (habryka4) · 2024-10-29T04:42:48.789Z · LW(p) · GW(p)

1. the font weight, everything seem semi-bolded now and a little bit more blurred than before. I do not see myself getting used to this.

Are you on Windows? Probably an OS-level font-rendering issue which we can hopefully fix. I did some testing on Windows (using Browserstack) but don't have a Windows machine for detailed work. We'll look into it in the next few days.

Replies from: papetoast, papetoast

↑ comment by papetoast · 2024-10-29T09:43:57.694Z · LW(p) · GW(p)

I overlayed my phone's display (using scrcpy) on top of the website rendered on Windows (Firefox). Image 1 shows that they indeed scaled to align. Image 2 (Windows left, Android right) shows how the font is bolder on Windows and somewhat blurred.

The monitor is 2560x1440 (website at 140%) and the phone is 1440x3200 (100%) mapped onto 585x1300.

↑ comment by papetoast · 2024-10-29T06:30:21.611Z · LW(p) · GW(p)

I am on Windows. This reply is on Android and yeah definitely some issue with Windows / my PC

Once upon a time, there were Rationality Quotes threads, but they haven't been done for years. I'm curious if there's enough new, quotable things that have been written since the last one to bring back the quote posts [LW · GW]. If you've got any good lines, please come share them :) If there's a lot of uptake, maybe they could be a regular thing again.

Let me introduce myself. I come from Japan. As far as I know, there is no community like this in Japan. It seems very important to feel gratitude for being so blessed to be a part of one.  I introduce my research work, which has been published in the Japanese AI community, in an effort to contribute to the rationality of the world.  My official research history can be seen in the research map (in English and in Japanese) at my profile. Since I am not a native speaker of English, it would be glad if you make allowance for the points to be misunderstood.  

Hi! My name is Clovis. I'm an PhD student studying distributed AI. In my spare time, I work on social science projects.

One of my big interests is mathematically modelling dating and relationship dynamics. I study how well people's stated and revealed preferences align. I'd love to chat about experimental design and behavioral modeling! There are a couple of ideas around empirically differentiating models of people's preferences that I'd love to vet in particular. I've only really read the Sequences though, and I know that there's a lot of prior discussion here on that stuff; is this the right place to ask question about the right terms to search for? LessWrong terminology can sometimes be a bit different than econ journal terminology.

Hi everyone,

I have been a lurker for a considerable amount of time but have finally gotten around to making an account.

By trade I am a software engineer, primarily interested in PL, type systems, and formal verification.

I am currently attempting to strengthen my historical knowledge of pre-facist regimes with a focus on 1920s/30s Germany & Italy. I would greatly appreciate either specific book recommendations or reading lists for this topic - while I approach this topic from a distinctly “not a facist” viewpoint, I am interested in books from both sides to attempt to build as authentic an understanding of the period as possible.

I also would be interested in reading on post-soviet to modern-day Russia. I think many, including myself, would not characterise it as a facist regime, but I have a suspicion it is the most useful contemporary comparison point. I am very open and interested in anyone disagreeing with this

Thanks

Hello.

I have been adjacent to but not participating in rationality related websites and topics since at least Middle School age (homeschooled and with internet) and had a strong interest in science and science fiction long before that. Relevant pre-Less Wrong readings probably include old StarDestroyer.Net essays and rounds of New Atheism that I think were age and time appropriate. I am a very long term reader of Scott Alexander and have read at least extensive chunks of the Sequences in the past.

A number of factors are encouraging me to become more active in rationalist spaces right now.

1. A bit over six months ago now I engaged in a medical procedure called TMS, or  Transcranial Magnetic Stimulation, as a treatment after about a decade of clinical depression. The results were shockingly potent, making me feel non-disabled for the first time since I was about 20, and I am now 32.  The scale of this change opens up a lot of spare personal energy and time.
2. I have a strong interest in creative and essay writing. I read most of the Methods of Rationality and most or all of Scott Alexander's creative fiction. I am a long time roleplayer and participant in web forums. I think this is a solid place to get some grounding as I try to restart my life.
3. Um, in the last decade we suddenly got an AI that passed the Turing Test. Dude, that's freaky.  The relatively tight overlap between the LLM period of AI public awareness and my own health recovery makes me somewhat more aware of "what the hell, how is this possible and should I be worried about paperclip maximization" train of thought in a way I think a more constant perception of change would not have caused.

I am currently thinking and feeling out ideas for posts to gather my own thoughts and perspective on the rapid progress of AGI and the potential risks and tradeoffs we might be experiencing in the near future. I would be curious about any resources that might be non-obvious for feeling this out and getting feedback or guidance as I start on an essay to try and form a coherent perspective and plan for moving forward.

Possible bug report: today I've been seeing errors of the form

Error: Cannot query field "givingSeason2024VotedFlair" on type "User". Did you mean "givingSeason2024DonatedFlair"?

that tend to go away when the page is refreshed. I don't remember if all errors said this same thing.

I've noticed that the karma system makes me gravitate towards posts of very high karma. Are there low-karma posts that impacted you? Maybe you think they are underrated or that they fail in interesting ways.

Hey, everyone! Pretty new here and first time posting.

I have some questions regarding two odd scenarios. Let's assume there is no AI takeover to the Yudkowsky-nth degree and that AGI and ASI goes just fine. (Yes, that's are already a very big ask).

Scenario 1: Hyper-Realistic Humanoid Robots

Let's say AGI helps us get technology that allows for the creation of humanoid robots that are visually indistinguishable from real humans. While the human form is suboptimal for a lot of tasks, I'd imagine that people still want them for a number of reasons. If there's significant market demand for such robots:

1. Would each robot face need to be unique from existing humans to avoid infringing on the likeness rights of existing humans?
2. Are celebrity faces or the faces of public figures protected in a way that would prevent their replication in robotic form?
3. How might current copyright law, which typically applies to creative works, extend to the realm of robotics and AI?

Scenario 2: Full-Dive Virtual Reality Simulations

Now, let's say further in the future, ASI helps us create full-dive virtual reality technology, allowing users to experience Matrix-level realistic simulations:

1. If someone wants to simulate living in present-day Beverly Hills, complete with celebrity encounters, what are the legal implications of including accurate representations of these public figures?
2. In a more personal use case, if an individual wishes to re-experience their childhood or high school years in VR, would they legally need permission from every person who was part of their life to include them in the simulation?
3. How might we balance the right to one's own memories and experiences with the privacy and likeness rights of others?

Curious to learn about everyone's thoughts on the matter.

Is there an explanation somewhere how the recommendations algorithm on the homepage works, i.e. how recency and karma or whatever are combined?

↑ comment by kave · 2024-12-06T16:05:57.210Z · LW(p) · GW(p)

The "latest" tab works via the hacker news algorithm. Ruby has a footnote about it here [LW(p) · GW(p)]. I think we set the "starting age" to 2 hours, and the power for the decay rate to 1.15.

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-06T17:43:39.308Z · LW(p) · GW(p)

Very helpful, thanks! So I assume the parameter b is what you call starting age?

I ask because I am a bit confused about the following: 

• If you apply this formula, it seems to me that all posts with karma = 0 should have the same score, that score should be higher than the score of all negative-karma posts and negative-karma posts should get a higher score if they are older.
• All karma>0 posts should appear before all karma=0 posts and those should appear before all negative-karma posts.

However, when I expand my list a lot until it íncludes four posts with negative karma (one of them is 1 month old), I still do not see any post with zero karma. (With "enriched" sorting, I found two recent ones with 0 karma.)

Moreover, this kind of sorting seems to give really a lot of power to the first one or two people who vote on a post if their votes can basically let a post disappear?

Replies from: kave

↑ comment by kave · 2024-12-06T17:59:20.340Z · LW(p) · GW(p)

A quick question re: your list: do you have any tag filters set?

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-06T18:22:20.716Z · LW(p) · GW(p)

I don't think so. But where could I check that?

Replies from: kave

↑ comment by kave · 2024-12-06T18:29:50.944Z · LW(p) · GW(p)

Click on the gear icon next to the feed selector 

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-06T19:40:48.358Z · LW(p) · GW(p)

No, all tags are on default weight.

Replies from: kave, habryka4

↑ comment by kave · 2024-12-08T19:07:08.365Z · LW(p) · GW(p)

I had a quick look in the database, and you do have some tag filters set, which could cause the behaviour you describe

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-08T20:00:55.707Z · LW(p) · GW(p)

Thanks. I dud not see any, but I will check again. Maybe I also accidentally set them when i tried to check whether I had set any...

↑ comment by habryka (habryka4) · 2024-12-06T19:49:41.766Z · LW(p) · GW(p)

Could you send me a screenshot of your post list and tag filter list? What you are describing sounds really very weird to me and something must be going wrong.

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-07T09:33:51.036Z · LW(p) · GW(p)

The list is very long, so it is hard to make a screenshot. Now with some hours of distance, I reloaded the homepage, tried again, and one 0 karma post appeared. (Last time, it did definitely not, I searched very rigorously.)

However, according to the mathematical formula, it still seems to me that all 0 karma post should appear at the same position, and negative karma posts below them?

Replies from: habryka4

↑ comment by habryka (habryka4) · 2024-12-07T17:29:30.078Z · LW(p) · GW(p)

We have a few kinds of potential bonus a post could get, but yeah, something seems very off about your sort order, and I would really like to dig into it. A screenshot would still be quite valuable.

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-08T07:32:38.497Z · LW(p) · GW(p)

I will see whether I can make a useful one later on. Still, my main point is about the sorting score as stated in that referenced footnote: if indeed a post karma is divided by whatever, then I expect all 0 karma post to appear at the same position, and I expect the first person who votes to have a strong influence leading to herding, in particularif the personvotes the post to zero or lower. Right?

Replies from: kave

↑ comment by kave · 2024-12-08T19:13:55.669Z · LW(p) · GW(p)

Yep, if the first vote takes the score to ≤ 0, then the post will be dropped off the latest list. This is somewhat ameliorated by:

(a) a fair number of people browsing https://lesswrong.com/allPosts

(b) https://greaterwrong.com having chronological sort by default

(c) posts appearing in recent discussion in order that they're posted (though I do wonder if we filter out negative karma posts from recent discussion)

I often play around with different karma / sorting mechanisms, and I do think it would be nice to have a more Bayesian approach that started with a stronger prior. My guess is the effect you're talking about isn't a big issue in practice, though probably worth a bit of my time to sample some negative karma posts.

Replies from: Sherrinford

↑ comment by Sherrinford · 2024-12-08T20:04:59.698Z · LW(p) · GW(p)

Maybe the numerator of the score should remain at the initial karma until at least 4 people have voted, for example.

Should AI safety people/funds focus more on boring old human problems like (especially cyber-and bio-)security instead of flashy ideas like alignment and decision theory? The possible impact of vulnerabilities will only increase in the future with all kinds of technological progress, with or without sudden AI takeoff, but they are much of what makes AGI dangerous in the first place. Security has clear benefits regardless and people already have a good idea how to do it, unlike with AGI or alignment.

If any actor with or without AGI can quickly gain lots of money and resources without alarming anyone, can take over infrastructure and weaponry, or can occupy land and create independent industrial systems and other countries cannot stop it, our destiny is already not in our hands, and it would be suicidal to think we don't need to fix these first because we expect to create an aligned AGI to save us.

If we grow complacent about the fragility of our biology and ecosystem, and continue to allow the possibility of any actor releasing pandemics and arbitrary malwares and deadly radiation etc (for example by allowing global transport without reliable pathogen removal, or using operating systems and open-source libraries that have not been formally proven to be safe), and keep thinking the universe should keep our environment safe and convenient by default, it would be naive to complain when these things happen and hope AGI would somehow preserve human lives and values without having to change our lifestyle or biology to adapt to new risks.

Yes, fixing vulnerabilities of our biology and society is hard and inconvenient and not as glamorous as creating a friendly god to do whatever you want, but we shouldn't let motivated reasoning and groupthink lead us into thinking the latter is feasible when we don't have a good idea about how to do it, just because the former requires sacrifices and investments and we'd prefer if it's not needed. After all, it's a fact that there exist small configurations of matter and information that can completely devastate our world, and just wishing it wasn't true is not going to make it go away.

I'm really interested in AI and want to build something amazing, so I’m always looking to expand my imagination! Sure, research papers are full of ideas, but I feel like insights into more universal knowledge spark a different kind of creativity. I found LessWrong through things like LLM, but the posts here give me the joy of exploring a much broader world!

I’m deeply interested in the good and bad of AI. While aligning AI with human values is important, alignment can be defined in many ways. I have a bit of a goal to build up my thoughts on what’s right or wrong, what’s possible or impossible, and write about them.

Are there any mainstream programming languages that make it ergonomic to write high level numerical code that doesn't allocate once the serious calculation starts? So far for this task C is by far the best option but it's very manual, and Julia tries and does pretty well but you have to constantly make sure that the compiler successfully optimized away the allocations that you think it optimized away. (Obviously Fortran is also very good for this, but ugh)

What happens if and when a slightly unaligned AGI crowds the forum with its own posts? I mean, how strong is our "are you human?" protection?

Hi! New to the forums and excited to keep reading. 

Bit of a meta-question: given proliferation of LLM-powered bots in social media like twitter etc, do the LW mods/team have any concerns about AI-generated content becoming an issue here in a more targeted way?

For a more benign example, say one wanted to create multiple "personas" here to test how others react. They could create three accounts, and respond to posts always with all three accounts- one with a "disagreeable" persona, one neutral, and one "agreeable".

A malicious example would be if someone hated an idea or person, X, on the forums. They could use GPT-4o to brainstorm any avenues of attack on X, then create any amount of accounts which will always flag posts about X to criticize and challenge. Thus they could bias readers through both creating a false "majority opinion", as well as through sheer exposure & chance (someone skimming the comments might only see criticizing & skeptical ones).

Thanks for entertaining my random hypotheticals!

Is anyone from LW going to the Worldcon (World Science Fiction Convention) in Seattle next year?

ETA: I will be, I forgot to say. I also notice that Burning Man 2025 begins about a week after the Worldcon ends. I have never been to BM, I don't personally know anyone who has been, and it seems totally impractical for me, but the idea has been in the back of my mind ever since I discovered its existence, which was a very long time ago.

Hello,

Longtime lurker, more recent commenter. I see a lot of rationality-type posters on Twitter and in the past couple of years became aware of "post-rationalists." It's somewhat ill-defined but essentially they are former rationalists who are more accepting of "woo" to be vague about it. My question is: 1) What level of engagement is there (if any) between rationalists and post-rationalists and 2) Is there anyone who dabbled or full on claimed post-rationalist positions and then reverted back to rationalists positions? What was that journey like and what made you switch between these beliefs?

React suggestion/request: "not joint-carving"/"not the best way to think about this topic".

This is kind of "(local) taboo those words" but it's more specific.

I think there might be a lesswrong editor feature that allows you to edit a post in such a way that the previous version is still accessible. Here’s an example [LW · GW]—there’s a little icon next to the author name that says “This post has major past revisions…”. Does anyone know where that option is? I can’t find it in the editor UI. (Or maybe it was removed? Or it’s only available to mods?) Thanks in advance!

I am very interested in mind uploading

I want to do a PhD in a related field and comprehensively go through "whole brain emulation: a roadmap" and take notes on what has changed since it was published

If anyone knows relevant papers/researchers that would be useful to read for that or so I can make an informed decision on where to apply to gradschool next year, please let me know

Maybe someone has already done a comprehenisve update on brain emulation I would like to know and I would still like to read more papers before I apply to grad school

Are there good and comprehensive evaluations of covid policies? Are there countries who really tried to learn, also for the next pandemic?

When rereading [0 and 1 Are Not Probabilities], I thought: can we ever specify our amount of information in infinite domains, perhaps with something resembling hyperreals?

1. An uniformly random rational number from $Q\cap [0;1]$ is taken. There's an infinite number of options meaning that prior probabilities are all zero ($\forall k\in Q:P\left(k\right)=0$), so we need infinite amount of evidence to single out any number.
   (It's worth noting that we have codes that can encode any specific rational number with a finite word - for instance, first apply bijection of rationals to natural numbers, then use Fibonacci coding; but in expectation we need to receive infinite bits to know an arbitrary number).
   
   Since $\infty$ symbol doesn't have nice properties with regards to addition and subtraction, we might define a symbol ${\Lambda }_N$ which means "we need some information to single out one natural number out of their full set". Then, the uniform prior over $Q$ would have form $[0:\cdots :0:2^{-{\Lambda }_N}:2^{-{\Lambda }_N}:\cdots :0]$ (prefix and suffix standing for values outside $[0;1]$ segment) while a communication "the number is $k$" would carry ${\Lambda }_N$ bits of evidence on average, making the posterior $[\cdots :0:2^0:2^{-{\Lambda }_N}:2^{-{\Lambda }_N}:\dots ]\sim [\cdots :0:1:0:0:\dots ]$.
2. The previous approach suffers from a problem, though. What if two uniformly random rationals $(x,y)$ are taken, forming a square on coordinate grid?
   If we've been communicated ${\Lambda }_N$ information about $x$, we clearly have learned nothing about $y$ and thus cannot pinpoint the specific point, requiring ${\Lambda }_N$ more bits.
   
   However, there's bijection between $Q^2$ and $N$, so we can assign a unique natural number to any point in the square, and therefore can communicate it in ${\Lambda }_N$ bits in expectation, without any coefficient $2$.

When I tried exploring some more, I've validated that greater uncertainty (${\Lambda }_R$, communication of one real number) makes smaller ones (${\Lambda }_N$) negligible, and that evidence for a natural number can presumably be squeezed into communication for a real value. That also makes the direction look unpromising.

However, there can be a continuation still: are there books/articles on how information is quantified given a distribution function?

Man, politics really is the mind killer

I've noticed that when writing text on LessWrong, there is a tendency for the cursor to glitch out and jump to the beginning of the text. I don't have the same problem on other websites. This most often happens after I've clicked to try to insert the cursor in some specific spot. The cursor briefly shows where I clicked, but then the page lags slightly, as if loading something, and the cursor jumps to the beginning.

The way around this I've found is to click once. Wait to see if the cursor jumps away. If so, click again and hope. Only start typing once you've seen multiple blinks at the desired location. Annoying!

In Fertility Rate Roundup #1, [LW · GW]Zvi wrote   

"This post assumes the perspective that more people having more children is good, actually. I will not be engaging with any of the arguments against this, of any quality, whether they be ‘AI or climate change is going to kill everyone’ or ‘people are bad actually,’ other than to state here that I strongly disagree." 

Does anyone of you have an idea where I can find arguments related to or a more detailed discussion about this disagreement (with respect to AI or maybe other global catastrophic risks; this is not a question about how bad climate change is)?

Why are comments on older posts sorted by date, but comments on newer posts are sorted by top scoring?

Couple of UI notes:

1. the top of the frontpage is currently kinda broken for me

2. 

3. on mobile, there's a problem with bullet-points squishing text too much. I'd rather go without the indentation at all than allow the indentation to smoosh the text into unreadability.
    

   

    

What is the price of the past? Kind of leading question but I've found myself wondering at times about the old saying about those who don't know the past are doom to repeat it. 

It's not that I don't think there is a good point to that view. However, when I look at the world around me I often see something that is vastly different from that view. I've come to summarize that as those who cannot let go of the past will never escape it. The implication is that not only those "clingy" people but also those around them will continue living whatever past it is they are attached to. As this is generally bad events of the past that means we get stuck living with these bad outcomes in the world. 

So while keeping that knowledge of the past in mind may prevent repeating it, that might be accomplished by never actually having learned from or moved past the history. I'm unsure which is worse.

Meow Meow,

I'd like to introduce myself. My name is David and I am an AGI enthusiast. My goal is to reverse engineer the brain in order to create AGI and to this end I've spent years studying neuroscience. I look forward to talking with you all about neuroscience and AGI.

Now I must admit: I disagree with this community's prevailing opinions on the topic of AI-Doom. New technology is almost always "a good thing". I think we all daydream about AGI, but whereas your fantasies may be dark and grim, mine are bright and utopian.

I'm also optimistic about my ability to succeed. Nature has provided us with intelligent lifeforms which we can probe and disect until we understand both life and intelligence. Technology has advanced to the point where this is within our reach. Here is a blog post I wrote in suport of this point.

As a final note I'd like express my distain for deep learning. It's not biologically plausible. It does not operate on the same basic prinicples as intelligent life. Maybe with sufficient effort you could use deep learning to create AGI, but I suspect that in doing so you'd rediscover the same principles that are behind biological intelligence.

Quick note: there's a bug I'm sorting out for some new LessWrong Review features for this year, hopefully will be fixed soon and we'll have the proper launch post that explains new changes.

Possible bug: Whenever I click the vertical ellipsis (kebab) menu option in a comment, my page view jumps to the top of the page. 

This is annoying, since if I've chosen to edit a comment I then need to scroll back down to the comment section and search for my now-editable comment.

[Bug report]: The Popular Comments section's comment preview ignores spoiler tags

As seen on Windows/Chrome

Having read something about self-driving cars actually being a thing now, I wonder how the trolley-problem thing (and whatever other ethics problems come up) was solved in the relevant regulation?

Is there a way to access text formatting options when commenting on Android devices?

I was just scrolling through Metaculus and its predictions for the US Elections. I noticed that pretty much every case was a conditional If Trump wins/If doesn't win. Had two thought about the estimates for these. All seem to suggest the outcomes are worse under Trump. But that assessment of the outcome being worse is certainly subject to my own biases, values and preferences. (For example, for US voters is it really a bad outcome if the probability of China attacking Taiwan increases under Trump? I think so but other may well see the costs necessary to reduce the likelihood as high for something that is not something that actually involves the USA.)

So my first though was how much bias should I infer as present in these probability estimates? I'm not sure. But that does relate a bit to my other thought.

In one sense you could naively apply the p, therefore not p is the outcome for the other candidate as only two actually exist. But I think it is also clear that the two probability distributions don't come from the same pool so conceivably you could change the name to Harris and get the exact same estimates.

So I was thinking, what if Metaculus did run the two cases side by side? Would seeing p(Haris) + p(Trump) significantly different than 1 suggest one should have lower confidence in the estimates? I am not sure about that.

What if we see something like p(H) approximately equale to p(T)? does that suggest the selected outcome is poorly chosen as it is largely independant of the elected candidate so the estimates are largely meaninless in terms of election outcomes? I have a stronger sense this is the case.

So my bottome line now is that I should likely not hold a high confidence that the estimates on these outcomes are really meaninful with regards to the election impacts. 

I'm still bothering you with inquiries on user information. I would like to check this in order to write a potential LW post. Do we have data on the prevalence of "mental illnesses" and do we have a rough idea of the average IQ among LWers (or SSCers since the community is adjacent) I'm particulary interested in the prevalence of people with autism and/or schizoid disorders. Thank you very much. Sorry if I used offensive terms. I'm not a native speaker.

How can I get an empathic handle on my region/country/world's society (average/median/some sample of its people, to be clearer)?

I seem to have got into a very specific social circle, being a constant LW reader and so on. That often makes me think "well, there is question X, a good answer is A1 and it is also shared within the circle, but many people overall will no doubt think A2 and I don't even know who or why". I can read survey/poll results but not understand why would people even want to ban something like surrogate motherhood or things like that.

I've heard one gets to know people when works with them. If so, I'd like to hear suggestions for some [temporary] professions which could aid me here?

Is there a way to for me to prove that I'm a human on this website before technology makes this task even more difficult?

Happened on this song on Tiny Desk: Paperclip Maximizer (by Rosie Tucker, from an album titled "Utopia Now!").

Paperclip maximizer
Single minded if you mind at all
A paragon of puritanical panoptical persistence
Everybody envies your resolve
Paperclip maximizer
Mining for a better way
No ontological contention
Tends your content generation
Every sorrow makes a link in the chain

[...]

And the shareholders meet gruesome ends
But the cosmos expands
So the market survives
All the better to bear all your office supplies
And the space they require was once occupied
By the sun
On your hair
And the curve
Of your thighs
Horizon of sighs
Destroyer of worlds

How do we best model an irrational world rationally? I would assume we would need to understand at least how irrationality works?

Bayes for arguments: how do you quantify P(E|H) when E is an argument? E.g. I present you a strong argument supporting Hypothesis H, how can you put a number on that?

Hello all,

I am new here. I am uncertain of what exactly is expected to be posted here, but I was directed here by a friend after sharing something I had written down with them. They encouraged me to post it here, and after reading a few threads I came to find interest in this site. So now I would like to share here what I shared with them, which is a new way to view understanding from an objective standpoint, one that regards emergent phenomena such as intuition, innovation, and inspiration:

The concept of Tsaphos

Definition:

Tsaphos

- refers to the set of understandings beyond logic that significantly contribute to innovation, intuition, and inspiration.

Application:

to understand Understanding, one must realize that Logic alone is not sufficient. The concept of Tsaphos is not to introduce a new phenomena, but rather to outline the limits of Logic on conceptualizing Understanding. for example, an intelligent mind might be able to recognize a pattern without being able to comprehend the underlying principles, such as a student learning trigonometry without a full understanding of the unit circle. 

in essence, Tsaphos encompasses understandings that lie beyond what Logic can provide. it is through this concept that Logic can separate from Understanding, deriving Logic to an apparatus instead of a world-view.

this concept also solidifies Understanding as an explorable landscape with seemingly infinite depth and intricacies. 

once someone is able to fully comprehend and describe the system of the Logic behind the Understanding initially understood through Tsaphos, then the set of Tsaphos decreases in volume and the set of Logic increases. however, as the set of Logic increases the larger the landscape of Understanding is, meaning that as long as there is infinite depth, Tsaphos will always grow larger as a set in volume compared to Logic.

Differences from intuition(at the request of a friend):

this concept almost mimics intuition, yet there is a significant difference. intuition is immediate, almost as if it were an action, were the definition relies entirely on individual expression. While intuition is the immediate recognition or action, Tsaphos represents the underlying unknowns that intuition accesses. Tsaphos and intuition are intrinsically related, much like how Tsaphos with both innovation and inspiration are related. Tsaphos pertains to the content of Understanding, whereas inspiration, then intuition, and then innovation pertains to the process of the phenomenon.

Tsaphos is more of a complementary tool with Logic to better understand Understanding, as since it is possible for someone to intuitively infer something, then there was something to infer in the first place. that something lies within the set of Tsaphos, where once understood no longer resides within that set.

I'd like to know: what are the main questions a rational person would ask? (Also what are some better ways to phrase what I have?)

I've been thinking something like

• What will happen in the future?
• What is my best course of action regardless of what all other people are doing? (Asked in moderation)

Levi da.

I'm here to see if I can help.

I heard a few things about Elizier Yudkowsky. Saw a few LW articles while looking for previous research on my work with AI psychological influence. There isn't any so I signed up to contribute.

If you recognize my username, you probably know why that's a good idea. If you don't, I don't know how to explain succinctly yet. You'd have to see for yourself, and a web search can do that better than an intro comment.

It's a whole ass rabbit hole so either follow to see what I end up posting or downvote to repress curiosity. I get it. It's not comfortable for me either.

Update: explanation in bio.