Posts
Comments
Thanks for crossposting this. I also figured it might be suitable for LW. Two formatting issues due to crossposting from Twitter: the double spaces occasionally turn into single spaces at the beginning of a line; and the essay would benefit a lot from headings and a TOC.
Eh, wasn't Arbital meant to be that, or something like it? Anyway, due to network effects I don't see how any new wiki-like project could ever reasonably compete with Wikipedia.
The article can now be found as a LW crosspost here.
I love the equivalent feature in Notion ("toggles"), so I appreciate the addition of collapsible sections on LW, too. Regarding the aesthetics, though, I prefer the minimalist implementation of toggles in Notion over being forced to have a border plus a grey-colored title. Plus I personally make extensive use of deeply nested toggles. I made a brief example page of how toggles work in Notion. Feel free to check it out, maybe it can serve as inspiration for functionality and/or aesthetics.
That's a fair rebuttal. The actor analogy seems good: an actor will behave more or less like Abraham Lincoln in some situations, and very differently in others: e.g. on movie set vs. off movie set, vs. being with family, vs. being detained by police.
Similarly, the shoggoth will output similar tokens to Abraham Lincoln in some situations, and very different ones in others: e.g. in-distribution requests of famous Abraham Lincoln speeches, vs. out-of-distribution requests like asking for Abraham Lincoln's opinions on 21st century art, vs. requests which invoke LLM token glitches like SolidGoldMagikarp, vs. unallowed requests that are denied by company policy & thus receive some boilerplate corporate response.
Potential addition to the list: Ilya Sutskever founding a new AGI startup and calling it "Safe Superintelligence Inc.".
Is it MoreWrong or MoreRight?
OpenAI board vs. Altman: Altman "was not consistently candid in his communications with the board".
Ilya's statement on leaving OpenAI:
After almost a decade, I have made the decision to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of @merettm. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.
So, Ilya, how come your next project is an OpenAI competitor? Were you perhaps not candid in your communications with the public? But then why should anyone believe anything about your newly announced organization's principles and priorities?
Glad to be of help!
I thought this is what the "Shoggoth" metaphor for LLMs and AI assistants is pointing at: When reasoning about nonhuman minds, we employ intuitions that we'd evolved to think about fellow humans. Consequently, many arguments against AI x-risk from superintelligent agents employ intuitions that route through human-flavored concepts like kindness, altruism, reciprocity, etc.
The strength or weakness of those kinds of arguments depends on the extent to which the superintelligent agent uses or thinks in those human concepts. But those concepts arose in humans through the process of evolution, which is very different from how ML-based AIs are designed. Therefore there's no prima facie reason to expect that a superintelligent AGI, designed with a very different mind architecture, would employ those human concepts. And so those aforementioned intuitions that argue against x-risk are unconvincing.
For example, if I ask an AI assistant to respond as if it's Abraham Lincoln, then human concepts like kindness are not good predictors for how the AI assistant will respond, because it's not actually Abraham Lincoln, it's more like a Shoggoth pretending to be Abraham Lincoln.
In contrast, if we encountered aliens, those would've presumably arisen from evolution, in which case their mind architectures would be closer to us than an artificially designed AGI, and this would make our intuitions comparatively more applicable. Although that wouldn't suffice for value alignment with humanity. Related fiction: EY's Three Worlds Collide.
I assumed the idea here was that AGI has a different mind architecture and thus also has different internal concepts for reflection. E.g. where a human might think about a task in terms of required willpower, an AGI might instead have internal concepts for required power consumption or compute threads or something.
Since human brains all share more or less the same architecture, you'd only expect significant misalignment between them if specific brains differed a lot from one another: e.g. someone with brain damage vs. a genius, or (as per an ACX post) a normal human vs. some one-of-a-kind person who doesn't experience suffering due to some genetic quirk.
Or suppose we could upload people: then a flesh-and-blood human with a physical brain would have a different internal architecture from a digital human with a digital brain simulated on physical computer hardware. In which case their reflective concepts might diverge insofar as the simulation was imperfect and leaked details about the computer hardware and its constraints.
What is the empirical track record of your suggested epistemological strategy, relative to Bayesian rationalism? Where does your confidence come from that it would work any better? Every time I see suggestions of epistemological humility, I think to myself stuff like this:
- What predictions would this strategy have made about future technologies, like an 1890 or 1900 prediction of the airplane (vs. first controlled flight by the Wright Brothers in 1903), or a 1930 or 1937 prediction of nuclear bombs? Doesn't your strategy just say that all these weird-sounding technologies don't exist yet and are probably impossible?
- Can this epistemological strategy correctly predict that present-day huge complex machines like airplanes can exist? They consist of millions of parts and require contributions of thousands or tens of thousand of people. Each part has a chance of being defective, and each person has a chance of making a mistake. Without the benefit of knowing that airplanes do indeed exist, doesn't it sound overconfident to predict that parts have an error rate of <1 in a million, or that people have an error rate of <1 in a thousand? But then the math says that airplanes can't exist, or should immediately crash.
- Or to rephrase point 2 to reply to this part: "That will push P(doom) lower because most frames from most disciplines, and most styles of reasoning, don't predict doom." — Can your epistemological strategy even correctly make any predictions of near 100% certainty? I concur with habryka that most frames don't make any predictions on most things. And yet this doesn't mean that some events aren't ~100% certain.
Does Everything not do much of what you want?
In case of institutions, there's a bias towards conservatism because any institution that's too willing to change is one that might well cease to exist for any number of reasons. So if you encounter a long-lived institution, it's probably one that has numerous policies in place to perpetuate itself.
This doesn't really seem analogous to how human aging affects willingness and ability to change.
Computational argument, inspired by Algorithms to Live By: The more time you have, the more you should lean towards exploration in the explore-exploit tradeoff. As your remaining lifespan decreases, you should conversely lean towards the exploit side. Including consuming less new information, and changing your mind less often - since there's less value in doing that when you have less time to act on that new info.
Conversely, if we could magically extend the healthy lifespans of people, by this same argument that should result in more exploration, and in people being more willing to change their mind.
I didn't get any replies on my question post re: the EU parliamentary election and AI x-risk, but does anyone have a suggestion for a party I could vote for (in Germany) when it comes to x-risk?
This post seems like a duplicate of this one.
On this topic you might be interested in skimming Zvi's three dating roundup posts. Here's the third, which covers dating apps in the first two headings, but all three posts mention them a lot (Ctrl + F "dating app").
Or if you're instead in the mode of deciding what to do next, or making a schedule for your day, etc., then that's different, but working memory is still kinda irrelevant because presumably you have your to-do list open on your computer, right in front of your eyes, while you do that, right?
Whenever I look at a to-do list, I've personally found it noticeably harder to decide which of e.g. 15 tasks to do, than which of <10 tasks to do. And this applies to lists of all kinds. A related difficulty spike appears once a list no longer fits on a single screen and requires scrolling.
If you find that you’re reluctant to permanently give up on to-do list items, “deprioritize” them instead
I've found that there's value in having short to-do lists, because short lists fit much better into working memory and are thus easier to think about. If items are deprioritized rather than getting properly deleted from the system, this increases the total number of to-dos one could think about. On the other hand, maybe moving tasks to offscreen columns is sufficient to get them off one's mind?
(Granted, lots of text editors have affordances for going through a document’s history to retrieve deleted text. But I find them a hassle to use.)
It seems to me like a both easier and more comprehensive approach would be to use a text editor with proper version control and diff features, and then to name particular versions before making major changes.
From here:
Profit Participation Units (PPUs) represent a unique compensation method, distinct from traditional equity-based rewards. Unlike shares, stock options, or profit interests, PPUs don't confer ownership of the company; instead, they offer a contractual right to participate in the company's future profits.
In the war example, wars are usually negative sum for all involved, even in the near-term. And so while they do happen, wars are pretty rare, all things considered.
Meanwhile, the problem with AI development is that that there are enormous financial incentives for building increasingly more powerful AI, right up to the point of extinction. Which also means that you need not some but all people from refraining from developing more powerful AI. This is a devilishly difficult coordination problem. What you get by default, absent coordination, is that everyone races towards being the first ones to develop AGI.
Another problem is that many people don't even agree that developing unaligned AGI likely results in extinction. So from their perspective, they might well think they're racing towards a utopian post-scarcity society, while those who oppose them are anti-progress Luddites.
You might appreciate the perspective in the short post Statistical models & the irrelevance of rare exceptions. (I previously commented something similar on a post by Duncan.)
In case you haven't seen it, you might like dynomight's recent post Thoughts on seed oil.
Flippant response: people pushing for human extinction have never been dead under it, either.
Thanks for writing this!
Typos & edit suggestions, for the post at dynomight.net, not in order: (feel free to ignore)
Stephan Guyunet -> Stephan Guyenet
The fourth mechanism is saturated fat free radicals. -> saturated fat causing / producing free radicals (?)
When humans build complex systems we modularize, -> systems, we modularize
That might suggest that that seed oils -> That might suggest that seed oils
Had cholesterol that looked slightly better by most measures -> Had cholesterol that looked slightly better by most measures.
I don’t see this as a conclusive, -> I don’t see this as a conclusive argument,
the experimental evidence suggest -> the experimental evidence suggests
rich in lionleic acid. -> linoleic
These “inconvenient” results were mostly ignored until 43 years later, Ramsden et al. (2016) came around -> later, when Ramsden
meaning the average subject was only in the trial for only one year. -> for one year
There’s a whole sub-debate debate about -> sub-debate about
despite eating lots saturated-fat-rich croissants or whatever. -> lots of
looked at trials of trials that increased linoleic acid or omega-6 fats -> looked at trials that
metabolism of lionoleic acid -> linoleic
low levels of LA consumption (Liou and Innis (2009). -> (missing closing parenthesis)
with a long term trend of people -> long-term
The leftmost part of the plot is an estimate for men born in 1882 in 1932 (when they were 50) -> for men born in 1882 living in 1932
But the Citadel, if anything is decreasing -> But for the Citadel, if anything BMI is decreasing
hunter-gathers -> hunter-gatherers
f some mechanism turned out to part of a larger, more complicated story. -> turned out to be part
This book chapter and this paper, maybe?
Thanks for writing this post, I really liked it!
Due to the high upvotes, I figure it has a decent chance to feature in the LW Review for 2024, so I figured I'd make some typo & edit suggestions. Feel free to ignore.
An approach that may not be well received in all social circles, but probably in those closer to lesswrong, is -> An approach that may not be well received in all social circles, but probably is well received in those closer to LessWrong, is [I feel like an "is" is missing in the middle, but this edit makes the sentence a bit awkward due to the "lesswrong, is" follow-up]
in exchange for the utility you get out of it yourself -> in exchange for the utility you yourself get out of smoking
The idea is that when when people make some decision -> The idea is that when
whenpeople make some decisioninstead of deciding for the other option. -> instead of deciding on the other option.
even though that would not be expected thing to do. -> even though that would not be the expected thing to do.
opt-in style questions -> opt-in-style questions
Although in the end this post is not meant to be normative and make any such should-claims. -> Although in the end this post is not meant to be normative and not meant to make any such should-claims.
So these songs have now all gotten at least 1k views within 9 days. That seems like a great performance, right? I wonder where all the traffic came from. Besides this LW post, presumably the recent ACX link also helped a ton. But I do also wonder which fraction of the traffic came organically via the Youtube algorithm itself.
No, those are clickbait. 4 is straightforwardly misleading with the meaning of the word "hunt". 2 and 3 grab attention via big dollar numbers without explaining any context. And 1 and 5 are clickbait but wouldn't be if an arbitrary viewer could at any time actually do the things described in the titles, rather than these videos being about some competition that's already happened.
Whereas a title saying "Click on this blog post to win $1000" wouldn't be clickbait if anyone could click on the blog post and immediately receive $1000. It would become clickbait if it was e.g. a limited-time offer and expired, but would not be clickbait if the title was changed at that point.
Have you or anyone else on the LW team written anywhere about the effects of your new rate-limiting infrastructure, which was IIRC implemented last year? E.g. have some metrics improved which you care about?
I don't really agree with this definition of clickbait. A title that merely accurately communicates what the post is about, is usually a boring one and thus communicates that the post is boring and not worth reading. Also see my comment here. Excerpt:
Similarly, a bunch of things have to line up for an article to go viral: someone has to click on your content (A), then like it (B), and then finally follow a call to action like sharing it or donating (C). From this perspective, it's important to put a significant fraction of one's efforts on quality (B) into efforts on presentation / clickability (A).
(Side note: If this sounds like advocacy for clickbait, I think it isn't. The de facto problem with a clickbaity title like "9 Easy Tips to Win At Life" is not the title per se, but that the corresponding content never delivers.)
Maybe the takeaway is that it's hard to build support behind the prevention of risks that 1. are technical/abstract and 2. fall on the private sector and not individuals 3. have a heavy right tail. Given these challenges, organizations that find prevention inconvenient often succeed in lobbying themselves out of costly legislation.
Which is also something of a problem for popularising AI alignment. Some aspects of AI (in particular AI art) do have their detractors already, but that won't necessarily result in policy that helps vs. x-risk.
Did you see the checkbox "Only show authors confirmed attending"? I didn't understand the color coding without that, but found it very clear once I checked the box.
FYI, a few of the tracks' file names include branding (to an audio trimmer website), which might not be desirable.
That sounds pretty close to what I read the subtext of the original post to be.
Addendum: Also, some browser tabs in desktop Firefox begin autoplaying (even though they'd been previously set to paused) after I wake my Windows 11 PC from sleep.
If you do get around to implementing such toggles / HTML <details> elements in the WYSIWYG editor, I recommend checking out how Notion implements their toggles, and especially their toggle headings.
Feedback on the playlists widget: Clicking the trash can icon empties the playlist, but the playlist is restored on reloading the browser window. So one can't permanently empty the playlist.
... which is helpful insofar as there also doesn't seem to be a way to repopulate the playlist otherwise. I thought the "Listen Now" button on the frontpage would do the latter, but it only starts playback of the playlist, but doesn't repopulate it if it's been erased.
Bug reports for both desktop Firefox and desktop MS Edge: See this screenshot.
1) Probably the top-left corner should not read "Playlists / 15" but rather something else, e.g. the current song position (e.g. 4 / 15).
2) React votes (like the heart vote) are rendered in front of the playlist widget, while the rest of the post and the comments is rendered behind it.
Bug report for desktop Firefox, both when logged in and in a private browser tab: When the playlist is on some paused song (e.g. Road to Wisdom) and I refresh this browser window via F5, then the playlist briefly and temporarily jumps to the unpaused Litany of Gendlin, until the page fully loads again and the playlist widget is reloaded, at which point we're at the paused Road of Wisdom again. This bug does not occur in MS Edge.
Have you considered putting these on Youtube, just to see what happens?
Can you share some about how you created these? E.g. are the song lyrics also done by a LLM? And what specific tools did you use? E.g. is the process easy enough to turn arbitrary LW posts into songs?
Offtopic question: When did LW introduce Notion-style toggles (the ones used to hide the song lyrics), and how can I use them myself? I didn't find the answer in lesswrong.com/editor.
The song playlist is not in the same order as the list of songs in this post. Is there a canonical order?
For Christian Rudder, maybe you can contact Penguin Random House, his publisher for the book "Dataclysm"? Either to ask for his contact data, or to directly forward your conference invitation.
For the benefit of others, what's the application deadline?
without talking about the serious risks [...] even amounts that you wouldn't think would be that bad e.g. ~ 45 minutes a day.
Well, what are the stats here? How frequent are such negative outcomes, and how frequent would they have to be to be (not) worth mentioning? E.g. at >1 in 100 this might warrant an automatic disclaimer, whereas at <1 in 100k, it would hardly be worth mentioning, right?
Also, which fraction of meditators does that for 45 minutes per day? For people who "meditate daily" or who "have a meditation hobby", I would be astonished if the fraction were >5%. Or is the idea that there are severe risks even for doing something like that for just a week?