Posts
Comments
I see. Friction management / affordance landscaping is indeed very important for interface UX design.
Seems like just pasting into the chat context / adding as attachments the relevant info on the default Claude web interface would work fine for those use cases.
Main concern right now is very much lab proliferation, ensuing coordination problems, and disagreements / adversarial communication / overall insane and polarized discourse.
- Google Deepmind: They are older than OpenAI. They also have a safety team. They are very much aware of the arguments. I don't know about Musk's impact on them.
- Anthropic: They split from OpenAI. To my best guess, they care about safety at least roughly as much as them. Many safety researchers have been quitting OpenAI to go work for Anthropic over the past few years.
- xAI: Founded by Musk several years after he walked out from OpenAI. People working there have previously worked at other big labs. General consensus seems to be that their alignment plan (as least as explained by Elon) is quite confused.
- SSI: Founded by Ilyia Sutskever after he walked out from OpenAI, which he did after participating in a failed effort to fire Sam Altman from OpenAI. Very much aware of the arguments.
- Meta AI: To the best of my knowledge, aware of the arguments but very dismissive of them (at least at the upper management levels).
- Mistral AI: I don't know much but probably more or less the same or worse than Meta AI.
- Chinese labs: No idea. I'll have to look into this.
I am confident that there are relatively influential people within Deepmind and Anthropic who post here and/or on the Aligment Forum. I am unsure about people from other labs, as I am nothing more than a relatively well-read outsider.
The Pantheon interface features comments by different LLM personas.
@dkl9 wrote a very eloquent and concise piece arguing in favor of ditching "second brain" systems in favor of SRSs (Spaced Repetition Systems, such as Anki).
Try as you might to shrink the margin with better technology, recalling knowledge from within is necessarily faster and more intuitive than accessing a tool. When spaced repetition fails (as it should, up to 10% of the time), you can gracefully degrade by searching your SRS' deck of facts.
If you lose your second brain (your files get corrupted, a cloud service shuts down, etc), you forget its content, except for the bits you accidentally remember by seeing many times. If you lose your SRS, you still remember over 90% of your material, as guaranteed by the algorithm, and the obsolete parts gradually decay. A second brain is more robust to physical or chemical damage to your first brain. But if your first brain is damaged as such, you probably have higher priorities than any particular topic of global knowledge you explicitly studied.
I write for only these reasons:
- to help me think
- to communicate and teach (as here)
- to distill knowledge to put in my SRS
- to record local facts for possible future reference
Linear, isolated documents suffice for all those purposes. Once you can memorise well, a second brain becomes redundant tedium.
Now some object-level engagement with your piece:
Very interesting. There are indeed well-read people who see Thiel as the ideological core of this Trump administration, and who view this as a good thing. I was under the (I now see, wrong) impression that Thiel-centrality was an hallucination by paranoid leftists. Thank you very much for providing a strong and important update to my world model.
Your personal website states that you are of Syrian extraction. Thiel is gay. Both of these facts point to a worldview that has trascended identity politics. I believe that identity politics as currently practised is mostly dumb and harmful, so I guess this is good news. Maybe even extremely good news. However, I am unsure how far it applies.
This ideological development is extremely interesting.
May I ask a series of questions?
- Is Echelon a thing that exists right now, or is it a thing that Thiel wants to build?
- Do you think Trump understands Thiel's ideas?
- Same question as above, but for Musk.
- Same as above, but for Sam Altman.
To restate my criticism in a more thorough way:
Your post reads like you are trying to vibe with a reader who already agrees with you. You cannot assume that in an open forum. There are many reasonable people who disagree with you. Such is the game you have decided to play by posting here. In this corner of the internet, you may find libertarians, socialists, conservatives, antinatalists, natalists, vegans, transhumanists, luddites, and more engaging in vigorous yet civilized debate. We love it.
Try to make the reader understand what you are trying to convey and why you believe it is true before vibing. It is useless to broadcast music that will be heard as noise by most of your audience. Help them tune their receivers to the correct frequency first.
Show your work. How did you come to believe what you believe? Why do you think it is true? What evidence would convince you that it is false?
We come here to search for truth, and hate vibing over false things. You have not given us good evidence that the thing you are vibing about is true.
Welcome. Do better and post again.
This post is pretty much devoid of world-modeling. It is instead filled to the brim with worldview-assertions.
Dear author, if I were to judge only by this post I would be forced to conclude that your thought process is composed solely of vibing over quotations. I hazard the guess that you can maybe do better.
The nearest thing I can think of off the top of my head is the Pantheon interface. Probably more unconventional than what you had in mind, though.
Upon reflection, I think I want to go further in this direction, and I have not done so due to akratic / trivial inconveniences reasons. Here is a list of examples:
- I used to take only cold showers, unless I needed to wash my hair. May be a good idea to restart that.
- I've wanted to center my workflow around CLI / TUI programs (as opposed to GUI) programs for a while now. It is currently in a somewhat awkward hybrid state.
- I used to use Anki and enjoy it. I dropped it during a crisis period in my life. The crisis has abated. It is imperative that I return.
I strongly agree with this post, and feel like most people would benefit from directionally applying its advice. Additional examples from my own life:
- One time, a close friend complained about the expense and effort required to acquire and prepare good coffee, and about the suffering incurred whenever he drank bad coffee. I have since purposefully avoided developing a taste in coffee. I conceive of it as a social facilitator, or as a medium to simultaneously ingest caffeine, water and heat.
- Back during my teenage years, one day I decided I would drink just water from then on. I have since dropped the fanaticism, and occasionally partake in other beverages. Soda now tastes obscenely strong, if not outright gross. I am healthier and wealthier than the counterfactual, since water is healthier and cheaper than all alternatives.
However, the assumption that high-quality high-skill human feedback is important and neglected by EAs has not been falsified.
To your best guess, is this still true?
Maybe one can start with prestige conservative media? Is that a thing? I'm not from the US and thus not very well versed.
Interesting. Do you have the code published somewhere?
I applaud the scholarship, but this post does not update me much on Gary Marcus. Still, checking is good, bumping against reality often is good, epistemic legibility is good. Also, this is a nice link to promptly direct people who trust Gary Marcus to. Thanks!
Hi sorry for soft-doxxing you but this information is trivially accesible from the link you provided and helps people evaluate your work more quickly:
danilovicioso.com
Cheers!
Oh. That's nice of her.
In the gibbs energy principle quote you provide, are you implying the devil is roughly something like "the one who wishes to consume all available energy"? Or something like "the one who wishes to optimize the world such that no energy source remains untapped"?
This post is explicitly partisan and a bit hard to parse for some people, which is why I think they bounced off and downvoted, but I think this writer is an interesting voice to follow. I mean, a conservative who knows deleuze and cybernetics? Sign me up! (even though I'm definitively not a conservative)
Hi! Welcome! Is your thesis roughly this?:
"The left latched into the concept of "diversity" to make the right hate it, thus becoming more homogeneous and dumber"
I think the thesis of the poster is roughly: The left latched into the concept of "diversity" to make the right hate it, thus becoming more homogeneous and dumber. Seems plausible, yet a bit too clever to be likely.
All of it. Thinking critically about AI outputs (and also human outputs), and taking mitigating measures to reduce the bullshit in both.
Yeah people in here (and in the EA Froum) are participating in a dicussion that has been going on for a long time, and thus we tend to assume that our interlocutors have a certain set of background knowledge that is admittedly quite unusual and hard to get the hang of. Have you considered applying to the intro to EA program?
Thank you for doing, that and please keep doing it. Maybe also run a post draft trough another human before posting, though.
Huh. Maybe. I think the labs are already doing something like this, though. Some companies pay you to write stuff more interesting than internet mediocrity. They even pay extra for specialist knowledge. Those companies then sell that writing to the labs, who use it to train their LLMs.
Side point: Consider writing shorter posts, and using LLMs to critique and shorten rather than to (co)write the post itself. Your post is kind of interesting, but a lot longer than it needs to be.
Huh. OK that looks like a thing worth doing. Still, I think you are probably underestimating how much smarter future AIs will get, and how useful intelligence is. But yes, money is also powerful. Therefore, it is good to earn money and then give it away. Have you heard of effective altruism?
Well good luck creating AI capitalists I guess. I hope you are able to earn money with it. But consider that your alpha is shrinking with every passing second, and that what you will be doing has nothing to do with solving alignment.
Because building powerful AI is also hard. Also, it is very expensive. Unless you happen to have a couple billion dollars lying around, you are not going to get there before OpenAI or Anthropic or Google Deepmind.
Also, part of the problem is that people keep building new labs. Safe Super Intelligence Inc and Anthropic are both splinters from OpenAI. Elon left OpenAI over a disagreement and then founded xAI years later. Labs keep popping up, and the more there are the harder it is to coordinate to not get us all killed.
Hi. The point of AI alignment is not whether the first people to build extremely powerful AI will be "the good guys" or "the bad guys".
Some people here see the big AI labs as evil, some see the big AI labs as well-intentioned but misguided or confused, some even see the big labs as being "the good guys". Some people in here are working to get the labs shut down, some want to get a job working for the labs, some even already work for them.
Yet, we all work together. Why? Because we believe that we may all die even if the first people building super-AIs are the most ethical organization on Earth. Because aligning AI is hard.
EDIT: See this post for understanding why even smart and well-intentioned people may get us all killed from AI.
I developed a simple first-order mechanism that measures the divergence between initial, user-introduced insights and their subsequent reproduction by AI. For instance, using a vector-space model of semantic representations, I compared the detailed descriptions provided by the user with the AI’s output.
Can we see the code for this? It would further discussion a lot.
I began by asking ChatGPT-4 to analyze our ongoing conversation and assess the novelty of the insights. ChatGPT-4 estimated that the ideas in our dialogue might be present in fewer than 1 in 100,000 users—an indication of exceptional rarity when compared against mainstream AI reasoning patterns.
Did you try asking multiple times in different context windows?
Did you try asking via the API (ie without influences from the "memory" feature)?
Do you have the "memory" feature turned on by default? If so, have you considered turning it off at least when doing experiments?
In summary: have you considered the fact that LLMs are very good at bullshitting? At confabulating the answers they think you would be happy to hear instead of making their best efforts to answer truthfully?
Oh I know! That is why I added "somehow". But I am also very unsure over exactly how hard it is. Seems like a thing worth whiteboarding over for an hour and then maybe doing a weekend-project-sized test about.
I think I agree more with your counterargument than with your main argument. Having broad knowledge is good for generating ideas, and LLMs are good for implementing them quickly and thus having them bump against reality.
The title "infohazardous glossary" sounds pretty insane. The contents of that webpage also strike me as pretty insane. The page is also structured as a glossary, and the concepts explained within it have very likely contributed to the insanity of the people who have heavily interacted with them. Therefore, the title "infohazardous glossary" seems pretty accurate after all.
My policy with this kind of stuff is to consider it harmful but also to consider it harmful to be scared of it's harmfulness. Generally disregard, but also maybe play with it for a little bit if I'm feeling curious and sane. It is interesting yes, but also mostly wrong and can be harmful to those who are on an epistemically/emotionally shaky place right now.
Epistemic status:
Confidence: Strong idea, weakly held.
Provenance: My own lived experience, put down in words by myself before even hearing about Ziz. All I know about Zizianism I have learned very recently (mostly from this thread), and I have a very negative opinion of it.
Masculinity and feminity have a biological basis, but most people's experience of them are strongly influenced by cultural factors. These cultural factors have been selected for being economically beneficial to agrarian societies. They are quite misaligned with what is beneficial for the happiness of post-industrial individuals. Poor societies made up of dumb people could not afford to not pigeonhole everyone into "straight men" and "straight women". We can now afford to have those categories and also the whole LGBTQ set of categories, although sometimes with a bit of friction when it bumps against the poorest and dumbest parts of our society. These frictions (and also in some cases a descriptive inadequacy of the LGBTQ labels) hurt people. Still, most individuals who are confident that their environment affords them to do so would probably benefit from a bit of experimentation / de-pigeon-holing.
When/if we get to a good post-TAI future, we will be able to afford to drop the concepts of discrete genders and discrete sexual orientations altogether. This will be a good thing, because it will make individuals freer.
This ""unihemispheric sleep" thing seems like it came from crazy and is an excellent way to produce even more crazy. A tale as old as time: small group of people produce some interesting ideas and all is mostly fine until they either take too many drugs or get the bright idea of intentionally messing up their sleep. This starts a self reinforcing loop of drugs / messed up sleep causing crazy causing drugs / messed up sleep causing even more crazy.
Did Ziz intend this to be seen as a metaphor (or) to be taken literally?
What is included within "latent knowledge" here?
Does it include both knowledge encoded in M1's weights and knowledge introduced in-context while running it?
I happen to care about animal suffering, and I am as baffled as you about the move of caring about animal suffering for explicitly anti-speciecist reasons yet dismissing wild animal suffering. Seems pretty inconsistent.
Maybe it originates from a sort of wishful thinking? As in "looks intractable, therefore I wish it were unimportant, therefore it is".
Attorneys seem well-equipped to design LLM debate games, or to at least consult for that design process.
Not exactly what you were asking for, but maybe food for thought: what if we (somehow) mapped an LLM's latent semantic space into phonemes?
What if we then composed tokenization with phonemization such that we had a function that could translate English to Latentese?
Historically, lesswrong has been better at truth-finding than at problem-solving.
I hope that this thread is useful as a high signal-to-noise ratio source.
This site is very much a public forum, so I would advise any actor whishing to implement a problem-solving stance to coordinate in a secure manner.
Holden Karnosfky published a list of "Utopia links" in his blog Cold Takes back in 2021:
Readers sent in a number of suggestions for fictional utopias. Here are a couple:
- Chaser 6 by Alicorn (~12 pages). This story makes heavy use of digital people (or something very much like them) [...]
- The 8-chapter epilogue to Worth the Candle [...]
- More suggestions in the comments, including the Terra Ignota series.
[...]
I forcefully endorse Chaser 6. I find myself thinking about it about once a month at a rough guess. The rest I haven't checked.
The Utopia links were motivated by Karnosfky's previous writings about Utopia in Cold Takes: Why describing Utopia Goes Badly and Visualizing Utopia.
I endorse Three worlds collide as a fun and insightful read. It states upfront that it does not feature AGI:
This is a story of an impossible outcome, where AI never worked, molecular nanotechnology never worked, biotechnology only sort-of worked; and yet somehow humanity not only survived, but discovered a way to travel Faster-Than-Light: The past's Future.
Yet, it's themes are quite relevant for civilization-scale outer alignment.
I know people who have gotten access to similarly important governmental systems at younger ages.
Is there a way you could make this claim credible without compromising people you do not want to compromise? Alternatively, are there public sources documenting similar situations?
This is why I consider it bad informational hygiene to interact with current models in any modality besides text. Why pull the plug now instead of later? To prevent frog-boiling.
It's a bit late in my current timezone and I read this stuff long ago and don't quite know where so please do not believe me much, but here goes:
I have the impression that social-trust-stuff is pretty load-bearing in hypnosis (ie if a psychiatrist does it you have to trust them for it to work, if a stage magician does it on a member of the public they bring onto the stage then they have to be skilled and the rest of the audience has to be receptive). So going from that I'd say that it is probably kind of a real thing but different and less powerful than you are probably imagining, and that doing it on yourself seems hard and maybe impossible.
Anyways don't quote me on this, I will retract this answer upon the slightest correction, I did zero research for it, I wrote it on a whim.
Productivity tools built on LLMs (or regexes) manipulating structured text might well provide a better UX (and of course, have the potential to be more private) than current proprietary web-based tool services (ie google calendar, todoist, etc). Local tools may interface with such services when necessary via APIs.
I've heard that LLMs have an easier time processing XML than idiosyncratically-formatted text. Maybe it is a good time to learn to write in XML? Or to acquire the habit of using XML by default for personal structured data management / productivity stuff (eg calendars, todo lists, personal CRMs)? Or to at least install an XML extension in your IDE / editor of choice?
For the record: I expect the impact to be positive. Separate points made in separate comments for the sake of discourse.
Consider sticking them inside STEM and political science university campuses, particularly those with good graduate programs. I expect this to increase the magnitude of impact. Here I'm deliberately refraining from commenting on the sign of impact (ie whether doing this would be net-positive or net-negative).