The code that reminds the AI of Hamas mentions checkpoints...

No idea what might trigger the Polish language. (Does any of the words in the text by coincidence mean something in Polish?)

That's an interesting idea. However, people who read this comments probably already have power much greater than the baseline -- a developed country, high intelligence, education, enough money and free time to read websites...

Not sure how many of those 20 doublings still remain.

the agent goes double-or-nothing until losing everything. That means that the effects of the AI are mitigated.

The side effects of the agent failing might still kill us.

For example, the failure could be something like "build a huge device which with probability 20% enables faster-than-light travel (which would allow colonizing more galaxies), and with probability 80% causes false vacuum collapse or otherwise destroys the entire universe".

Or something on smaller scale, where the failure means blowing up the Earth, destroying all life, etc.

the default way that people make their voices heard in politics these days is by stopping things or banning things or blocking things or slowing things down.

Maybe because it is easier to agree on a binary question ("should this be allowed or banned?") than on an open-ended one ("something should be done -- but what specifically?"). Give people a binary choice, and there is a chance that enough of them will agree. Give them an open-ended question, and most people will come with their own proposals, unwilling to support anyone else's proposal (unless they are allowed to do large modifications, which other people will oppose).

(Here an individualistic culture probably makes it worse, because coming with your own proposal is high-status.)

I guess most people have this experience, so they don't even try to make proposals to the public. Instead, if possible, they act alone, or with a small group of friends.

We can go around the neighborhood, show everybody the mockup and say, "Are you excited about us doing this to the park?" Then if we have a reasonable number of signatures on a petition, we get to build it.

I am often too pessimistic, but I would expect many people to say "no", for reasons including "no specific reason, it just sounds suspicious to me: why you? why now? is this perhaps some kind of scam?" or "I will only agree if you update your proposal to include <my pet peeve, completely unrelated to the project>", plus a few people saying "I don't give a fuck, so I will vote 'no' in principle (maybe try to bribe me if you want my 'yes')".

However, there are two situations near me where people somehow succeeded to build something for the community, so I should probably try to learn the details. In one case, it is a community garden: area between two garages was surrounded by a fence, and how there are tables and chairs, and about once in a month someone organizes some activities for kids there. In another case, in place of a former shop, a community center was set up. I think the latter is just one person's activity who someone got grant money to rent the place (maybe also made a non-profit for that purpose) so I would still kinda classify that as a pro-social grant-supported unilateral action. No idea how the former may have succeeded.

BTW, you seem impressed by George Church very much, because you linked his page 3 times. :D

The usual naive pro-trans argument goes like this: "some people are intersex, therefore sex is arbitrary, therefore it is okay for people to identify as whatever they want".

But if we take sex and gender as two dimensions, then it's like: "on the sex dimension, most people are at one end, but a few people are in between... and on the gender dimension, traditionally it matches the sex, but some people (importantly: not necessarily those who are in between on the sex dimension) identify as the opposite".

I guess this would be considered an anti-trans argument these days, because it allows people to express positions such as "I am attracted to the female sex, regardless of gender", or "I think bathrooms should be separated by sex (without having a strong opinion on intersex people)".

With coin, the options are "head" and "tails", so "head" moves you in one direction.

With LLMs, the options are "worse than expected", "just as expected", "better than expected", so "just as expected" does not have to move you in a specific direction.

Yesterday, I saw a video on social media of a woman discussing how, since moving to the US, she has started mysteriously gaining weight even though she's eating the same amount of food as before she moved here.

Just wanted to add a data point, that 20 years ago a classmate told me a similar thing. When she returned from USA, the mysterious weight gain stopped (I don't remember whether she returned to the original weight, or just stopped gaining more).

Larry Page allegedly dismissed concern about AI risk as speciesism.

That's what we get for living in a culture where calling something "...ism" wins the debate.

You just need to get good at creative thinking, management and framing ideas.

Yeah, the skills necessary for the (near) future.

Though I wonder about implications for education. For the sake of argument, let's imagine that the AIs remain approximately as powerful as they are today for a few more decades, i.e. no Singularity, no paperclips. How should we change education, to make the new generation adapt to this situation.

In case of adults, we have already learned "creative thinking, management and framing ideas" by also doing lots of the things that the LLMs can now do for us. For example, I let LLMs write JavaScript code for me, but the reason I can evaluate that code, suggest improvement, etc. is that in the past I wrote a lot of JavaScript code by hand. Is it possible to get these skills some other way? Or will the future humans only practice the loop of: "AI, do what I want. AI, figure out the problem and fix it. AI, try harder. AI, try superhard. Nevermind, AI, delete the project, clear your cache, and try again." :D

Ah, I can totally relate to this. Whenever I think about asking for money, the Impostor Syndrome gets extra strong. Meanwhile, there are actual impostors out there collecting tons of money without any shame. (Though they may have better social skills, which is probably the category of skill that ultimately gets paid best.)

Another important lesson I got once, which might be useful for you at some moment: "If you double your prices, and lose half of your customers as a result, you will still get the same amount of money, but only work half as much."

Also, speaking from my personal experience, the relation between how much / how difficult work someone wants you to do, and how much they are willing to pay you, seems completely random. One might naively expect that a job that pays more will be more difficult, but often it is the other way round.

It happens right in front of my house. Addicts steal things at shops, sell them at a pawn shop, buy drugs from a dealer waiting right in front of the pawn shop (difficult not to notice: the only guy who wears a black hoodie in the middle of a hot day), then inject the drugs behind the pawn shop.

When the shops are closed, or the addicts draw too much attention from the security, they try breaking into our houses and cellars instead. Every door in the neighborhood has signs of an attempt to pry it open.

What can we do about this?

Make shop theft impossible? Unlikely to happen, they would need to have the security literally everywhere.

Prevent the pawn shop from buying stolen stuff? I don't understand the details, but I was told that if they make the seller sign a paper saying "I totally swear I didn't steal this", they are legally ok. The pawn shop owner definitely knows that he deals with stolen stuff; that's why he moves everything between the shops, so that when they rob your house, you won't find your stuff in the shop window on your street the next day.

Prevent the dealer from selling drugs? That's quite tricky, legally, because owning a small amount of drug "for your own use" is not illegal here. Of course the dealer only brings one small bag of powder each time. Also, you pay to one guy, and get the powder from another guy, so it is legally tricky to determine at which moment exactly the drug was sold. (The first guy didn't give you any drug, and the second guy didn't take any money from you. If you only catch one of them, he will probably claim it was just a misunderstanding.)

So our only remaining option is to form a vigilante squad, and... well, I am not going to write down anything that may or may not happen afterwards. Didn't expect this to happen to me, and yet, here I am.

Reducing penalties for drug use is a well-sounding idiocy. In theory, you reduce the penalties for drug use, but in practice, you reduce the penalties for drug distribution, because most of the time when you catch a dealer, he can argue that this was all for his own use. And he only takes with him one bag of powder at a time. Yes, in theory it is possible to find the store full of bags, but you just made it needlessly complicated. When drug possession is a crime, you can catch the dealer, he says it was for his own use, you arrest him anyway. (Ideally, you would give him exponentially increasing sentences, starting with community service.)

This feels like debating a holocaust denier. We are moving from "it did not happen at all" to "maybe it wasn't six million Jews but only five million". ("You did not name a single historian, Greek city state, solitary event, or personality from history" -> "ancients simply did not keep accurate records ... what evidence we do have shows the numbers to be always exaggerated")

The argument by inaccurate records goes both ways. If there is a genocide today, we probably know about it, and someone at least makes a note in Wikipedia. In the past, ethnic groups could be erased with no one (other than the people involved in the war) noticing. The fact that the list of known genocides in 20th century is longer than the list of known genocides in e.g. 12th century is mostly because of better bookkeeping.

And yet, despite choosing a century randomly (if I tried on purpose, I could have chosen e.g. the 13th century with Albigenian Crusade as a good example), Wikipedia mentions "Massacre of the Latins" with about 60 000 dead in the 12th century. In a world where the population was not even 1/10 of what it is today, so relatively comparable with the numbers that you have mentioned. And we have no idea about what massacres might have happened in 12th century Africa.

So yes, today we have more victims in absolute numbers, but that's because we have larger populations and stronger weapons. When you have to kill your enemies using a hand axe, I guess you get quite tired after chopping off dozen heads. With a nuke, you just press a button and thousands die. And yet, despite the other side having nukes, most Japanese survived WW2. (Which is something they totally did not expect, given their usual behavior towards defeated enemies.) The people in the past were as efficient at killing their enemies with swords, as we are with the weapons of mass destruction today.

"Now go, attack the Amalekites and totally destroy all that belongs to them. Do not spare them; put to death men and women, children and infants, cattle and sheep, camels and donkeys." (1 Samuel 15:3) Tell me again how civilians were not considered valid targets in the past.

You mention compelling prisoners of war to labor, as an analogy to slavery. Yeah, but that was an exception during the war. (Except for the Soviets, who conveniently kept many of the prisoners of war long after the war was over.) Now compare to a situation thousand years ago, when the slave trade was a crucial part of European economy, comparable to oil trade today. The reason entire countries converted to Christianity was to stop the unending slave raids from their neighbors. (Christians had a taboo against enslaving each other. So did Muslims. Both of them considered it okay to enslave each other, and the pagans.) Or consider Africa: the first black slaves brought to America were legally bought in Africa from the local African slave traders. Americans did not invent slavery; they just provided a huge new market for it.

Sorry, I think it is you who needs to learn history. Yes, humans suck today; the "Noble Savages" were not any better, probably much worse.

Attempts to jailbreak LLMs seem obvious to humans. What does it mean?

Maybe it is a selection bias -- a non-obvious jailbreak would simply seem to me like "someone made a valid argument", so I wouldn't classify it as a jailbreak attempt.

Is there enough similarity so that we could create an input-checker AI which would only read the input for the purpose of determining whether it is a jailbreak attempt or not... and only if the input is considered okay, it would be passed to the second AI that actually tries to respond to it?

(That is, does the fact that the AI tries to respond to the input make it more vulnerable? As an analogy, imagine a human who is supposed to (a) write a reply to a comment, or (b) verify that the comment is written in English. The comment happens to contain a triggering description, or a mindkilling argument, or some other bad meme. I would expect the person who only verifies the English language to be less impacted, because they interact with the content of the meme less.)

Assuming that it is possible to jailbreak every AI, including the input checker, are there universal jailbreaks that apply to all AIs, or do you need to make a tailored jailbreak for each? Could we increase the resistance by having several different input-checker AIs, and have each input checked by three randomly selected ones?

(It is important that the algorithm that implements "if two out of three AIs say that the content is safe, but one says that it is a jailbreaking attempt, reject the input" is classical, not an LLM -- otherwise it would be more efficient to jailbreak this one.)

Interesting perspective. The difficult part will be that the proposed metrics are of the "more or less" type, rather than "yes or no". So one must be familiar with multiple beliefs, in order to put the specific one on the scale.

Psychological comfort -- each belief implicitly divides people into two groups: those who understand it and those who don't; the former is better. Knowing the Pythagorean theorem can make you proud of your math skills.

It gets suspicious when a seemingly simple belief explains too much. Knowing the Pythagorean theorem allows you to calculate the longest side of a right-angled triangle, a distance between two points in N-dimensional Euclidean space even for N>3, or allows you to prove that sin²(φ) + cos²(φ) = 1, but that's it. If it also told you how to dress, what to eat, and which political party to vote for, that would be suspicious.

On the opposite end of the scale, mathematics as a whole claims to explains a lot, it is practically involved in everything, but it is a ton of knowledge that takes years or decades to study properly. It would be suspicious if something similarly powerful could be understood by merely reading a book and hanging out with some group.

(Elephant in the room: what about "rationality", especially the claim that "P(A|B) = [P(A)*P(B|A)]/P(B)" explains the entire multiverse and beyond? I think it is kinda okay, as long as you remember that you also need specific data to apply the formula to; mere general knowledge won't help you figure out the details. Also, no one claims that the Bayes Theorem can only be used for good purposes, or that it makes you morally superior.)

Self-Sealing Mechanisms -- be careful when the belief is supported by things other than arguments and data; for example by violence (verbal or otherwise). This is also tricky: is it okay to "cancel" a crackpot? I think it is okay to fire crackpots from academic/scientific institutions; those obviously wouldn't be able to do their job otherwise. But if you start persecuting heretical thoughts expressed by people in their free time, on their blogs, etc., that goes too far.

I sometimes says that "political orientation" is basically "which part of complex reality you decided to ignore". (That doesn't mean that if you ignore nothing, you are unable to have opinions or make decisions. But you opinions will be usually be like "this is complicated, in general it is usually better to do X, but there are exceptions, such as Y". The kind of reasoning that would get you kicked out of any ideological group.)

"Thousands" is probably not enough.

Imagine trying to generate a poem by one algorithm creating thousands of random combinations of words, and another algorithm choosing the most poetic among the generated combinations. No matter how good the second algorithm is, it seems quite likely that the first one simply didn't generate anything valuable.

As the hypothesis gets more complex, the number of options grows exponentially. Imagine a pattern such as "what if X increases/decreases Y by mechanism Z". If you propose 10 different values for each of X, Y, Z, you already have 1000 hypotheses.

I can imagine finding some low-hanging fruit if we increase the number of hypotheses to millions. But even there, we will probably be limited by lack of experimental data. (Could a diet consisting only of broccoli and peanut butter cure cancer? Maybe, but how is the LLM supposed to find out?) So we would need to find a hypothesis where we accidentally already made all the necessary experiments and even described the intermediate findings (because LLMs are good at words, but probably suck at analyzing the primary data), but we somehow failed to connect the dots. Not impossible, but requires a lot of luck.

To get further, we need some new insight. Maybe collecting tons of data in a relatively uniform format, and teaching the LLM to translate its hypotheses into SQL queries it could then verify automatically.

(Even with hypothetical ubiquitous surveillance, you would probably need an extra step where the raw video records are transcribed to textual/numeric data, so that you could run queries on them later.)

Siege of Melos: Athens demanded a tribute, Melos refused to pay, the Athenians executed the men of fighting age and sold the women and children into slavery. They then settled 500 of their own colonists on the island.

Battle of Plataea: After the battle of Plataea, the city [Caryae] was captured by the allied Greeks, the city's men were executed and the women were enslaved.

Miletus: Persians under Darius the Great punished Miletus for rebellion by selling all of the women and children into slavery, killing the men, and expelling all of the young men as eunuchs, thereby assuring that no Miletus citizen would ever be born again.

Battle of Thebes: Thirty thousand were sold into slavery and six thousand slain in the final fighting. The city was burnt to the ground, sparing only the temples, the Cadmae citadel and the house of Pindar, out of gratitude for Pindar's verses praising Alexander's ancestor, Alexander I of Macedon.

If we tried to make universities examine non-students, the easiest way for the universities to cheat would be to teach literal passwords to their students, and then fail everyone who doesn't know the password. Then they could demonstrate that even in a fair competition only their students are able to pass the exams.

With some cleverness you could insert arbitrary passwords even into hard sciences, for example by asking "what was Einstein's most important contribution to physics?" Even if you knew everything about physics, and everything about Einstein, it wouldn't help you figure out what was his most important contribution in the teacher's opinion. (Could be any of them. Could even be something meta.)

Or you could take a set of arbitrary details (something so specific that it is virtually impossible for a human to memorize all things in the same category of relevance) and keep repeating the selected details to your students every year. Ask those details in exams every year; then ask them again on the final exam.

And with soft sciences... basically everything already is like this. (Or am I wrong? There is a way to test this empirically: take students from a few universities and make them take exams at a different university that teaches nominally the same subject; preferably in a different country.)

Not sure if your point is that you disagree with my description of war in ancient Greece and the Old Testament, or that you think I cherry-picked convenient examples in trend that goes overwhelmingly in the opposite direction, or you are ignoring the data points completely and your argument is merely "what you said pattern-matches a political tribe X, now here are some books written by a tribe Y".

Choosing Elites (1985)

With other variables held constant, a measure of “academic ability” that combined high-school records and SAT test scores was about ten times as important as a measure of “student effort” (study time and so forth), in terms of how much was learned.

Other things equal, a hundred point increase in both the SAT verbal and math scores made a slightly greater difference as to how much more students learned than did the difference between the best and worst instructors in the course.

Students’ ratings of the competence and personability of their teachers had absolutely no relationship with how much the students learned. But the teachers’ grades in graduate school did.

Yep. When we deny the IQ differences, we have to attribute the difference in the outcomes to something else... and schools and teachers seem like a natural choice. Therefore we overestimate the impact of education.

Don’t Make No Waves…Don’t Back No Losers: An Insiders’ Analysis of the Daley Machine (1975)

I couldn't choose a specific quote, it is all very interesting.

Making the Corps (1997)

The review is interesting, but the part I liked most was this thought at the end:

In fact, ["From each according to his ability, to each according to his need"] sounds pretty nice when taken at face value. For example, this is how families operate: from the parents according to their ability, to the children according to their need. Many insular, highly religious communities also operate this way. In Amish, Mennonite, and Hasidic communities, charitable giving (“alms”) often provides a social safety net for members in need. Some egalitarian hunter-gatherer tribes also functioned like this. And to be honest, it does seem like it would be pretty nice to live in this kind of a community, where everyone looks out for each other.

So what’s going on? Is this kind of “communism” good actually? Libertarians have an easy answer for this – communes are fine if they’re voluntary. As long as nobody’s being violently coerced and people are allowed to leave if they want to, then it’s fine. Alright, so that’s easy but also unsatisfying. Why do people voluntarily participate in these communes, even when it means a net-loss for the most capable and productive members? Why doesn’t the Laffer curve apply to the most productive Amish farmer who has his wealth redistributed to the neediest in the community? Why didn’t the Margaret Thatcher quote about socialists “running out of other people’s money” apply to the most capable hunters in egalitarian hunter-gatherer tribes?

I think the reason why some redistributive systems work and some don’t is related to the psychology of tribalism. A redistributive economic system where everyone views themselves as part of the same tribe and a system in which they view themselves as two tribes [...] are completely different in terms of the psychology of the participants and the ultimate sustainability of the system. The usual critique of wealth redistribution is that it disincentivizes economic production (Laffer curve, Margareth Thatcher quote, etc). But I think this criticism only applies to redistributive systems where the people taking a net-loss and the people taking a net-gain view themselves as two different groups. When they view themselves as part of the same group, as with the Amish or egalitarian hunter-gatherers, then the criticism seems to fall flat, as the most productive members aren’t deterred from working by the net-loss they’re taking.

Explicitly accounting for the one-tribe / two-tribe distinction also helps clear up a lot of debates in political philosophy. [...] The one-tribe democracy is the ideal: a society where people view themselves as all part of the same group, evaluate policies based on what’s best for society as a whole, and then vote based on their individual judgment of what’s best for the group, with this hopefully leading to policies converging on the correct answer. Of course people might disagree about which policies are the best and have heated debates about them, but ultimately the goal is the betterment of the society, and the debate is just over what policies best achieve that goal. On the other hand, in the two-tribe democracy, the goal is simply to vote to take away power, resources, and status from the Out-Group and give them to the In-Group.

This sounds correct to me. I think that an important part is how much I expect other people to reciprocate: not necessarily in this reality, but at least counterfactually (in a hypothetical reality where our roles are reversed). I am okay with doing things for my kids, because I expect them to grow up and do things... not necessarily for me, but further along the line. On the other hand, it would create a tension between me and my partner if we didn't split our domestic duties in a way that seems fair to both. In other words, I need to see that "each according to one's abilities" also applies to the others, even if their abilities are different from mine.

The part about needs is trickier, because the needs are potentially unlimited. Also, everyone needs to take a break and relax sometimes, but that means they are not working according to their abilities at the moment. So this all is not necessarily philosophically coherent, but it is still psychologically easier for me to work harder when I believe that everyone else does (even if they are at a different level of ability).

When I see people not doing things, or doing things in a way that makes sure that I will get none of it (for example a clique of people, perhaps hard-working, but making sure they only do things for the other people in the clique), that pisses me off, and then I am also not in the mood to share with them.

I guess, in small groups people punish the ones who don't do their share (e.g. by refusing to share with them), but in large groups it is difficult to track everyone's contribution and also to control how the tax money is used. Also, in small groups, if the group pisses you off too much, you can leave and join another group; this is more difficult when the "group" is a country, and leaving means selling your house, learning another language, adapting to a different culture, and there are only a few dozen choices available on the entire planet, and most of them either refuse to accept you or are quite dysfunctional.

substack, not blogspot

There's indeed been no one with that level of reach that has spread this much misinformation and started this many negative rumors in the space as David Gerard and RW. Whoever the second closest contender is, is likely not even close.

First place goes to David Gerard / RationalWiki -- not exactly the same thing, but a huge overlap. There are also other editors on RationalWiki, but most of them are much less active and not as hostile towards the rationalist community as David. David is also the driving force behind the SneerClub, plus he edits Wikipedia, which vastly expands his possibilities (he can write a "snarky" comment on RW, have a journalist quote it, then he can quote the journalist in Wikipedia, protect the article against changes, and remove all quotes that oppose his narrative).

The distant second place goes to Alexander Kruel, mostly for his historical achievements, because he is not active recently. (But a decade ago, about 2/3 of negative statements about the rationalist community could be traced back to David/RW and about 1/3 to Alexander's blog.)

There is no third place, IMHO. Everything else is just a blip on the radar, including the negative press recently sparked by the Zizians.

Nice link, the rationalist community should consider moving to Kazakhstan! ;)

I don't think this kind of comparison is important, because RW and LW are different types of websites. RationalWiki advertises itself as a resource for debunking pseudoscience, but I think that the numbers on that graph are mostly driven by their active involvement in hot topics of various culture wars: they have lots of articles on Trump, GamerGate, feminism, et cetera. On the other hand, Less Wrong actively tries to avoid all this, and tries to keep focus on mostly nerdy topics. (Also, the most popular LW writers have started their own blogs.)

So we basically have political clickbait competing for popularity with a nerdy walled garden, and if at the end of the day their numbers are similar... I think this is a pretty damning result for RationalWiki.

I think libel suits are really bad tools for limiting speech

Especially when the punishment is not only that you may lose the suit, but also the money you have to spend on your legal defense even when you win.

That said, RationalWiki is primarily a bullying (excuse me, snarky) website that most people only consider okay because they believe that the targets deserve it. (And they often do -- but the problem is, once you have a shiny powerful gun, it is too tempting to expand the scope.) When your mission is to attack people publicly, you better stick to the facts, and don't dismiss criticism with "but it is funnier this way", or the lawyers may have the last laugh.

The point is exactly that the shittiness seems decreasing on the historical scale, albeit very slowly.

Compare e.g. Moses and Hitler. Both of them became famous for being leaders who demonized their enemies and tried to exterminate them to the last one. Yet the latter is considered the archetype of evil, because he did the thing during the 20th century, when we expected people to do better. The former is considered a holy man by multiple religions, and his violent actions are not considered a stain on his character, because "back then, everyone was like that".

Looking at historical documents, it seems to me that "kill everyone who resists, and install a puppet government" (the Western countries recently) is an improvement over "kill everyone who resists, and make the survivors learn your language and your version of history" (Russia today, and the Western countries a century or more ago) which is an improvement over ""kill everyone who resists, sell everyone else as a slave" (e.g. the ancient Greek city states) which was an improvement over "kill everyone, except for a few young women whom you decide to keep as sex slaves after you raped them" (the Old Testament, and probably everyone before them).

obviously the very concept of survival of the fittest is one that does not even come into play until the 18th and 19th centuries

The theory is new. But the practice... was already practiced by chimpanzees.

behave, both as individuals and as nations, in a way that people of the past would find despicably cruel

Please educate me more, don't merely throw hints like this! Should we burn some cats for fun as an antidote to our general decadence, or did you have other specific things in mind?

I agree with the part that the scale should not matter. (Except, as Richard said, the age of the universe is the same for everything in it, so the largest structures had relatively not enough time to do anything interesting.)

But I think the atoms and galaxies simply do not have enough internal complexity to do something as interesting as cells or animals. Atoms, because there is little inside. Galaxies, because the things inside them do not interact enough.

I think another general mistake here is nerds thinking that they can ride a tiger, instead of getting eaten by it.

The wannabe 4D-chess thinking goes like this: "Here is a group of people who hate reason and experts. Therefore they have no experts among themselves. Therefore, if I join them and gain their trust, as a smart person I will quickly rise to the top, and then I will have an unparalleled opportunity to change the world."

What actually happens: The guy joins the movement, and as long as he follows the crowd, he can get quite popular. But the first moment he tries to do something different, his popularity/influence drops instantly.

I respect Hanania for admitting his mistake. Everyone else is delusional. This is basically the same mistake Dominic Cummings made a few years ago, and apparently we didn't learn a bit.

On the other hand, people from the United States are often the first to tell you that "freedom of speech" is not a general aspiration for making the world a better place, but merely a specific amendment to their constitution, which importantly only applies to censorship done directly by the government... therefore it does not apply to censorship by companies or mobs or universities or whatever.

(As an extreme example, from this perspective, a country without an official government would count as 100% free speech, even if criticizing the local warlord gets you predictably tortured to death; as long as the local warlord is not considered a "government" because of some technicality.)

depicts society as a kind of conspiracy to suppress ambition, on the principle that misery loves company

ugly picture of human nature as spiteful and vengeful, and because it implies there is no will to serious liberation (I wanted to end work and death), in fact it predicts that such attempts will be resisted

anyone setting out to become a rich and powerful capitalist, as Thiel did, has to worry about becoming a focus of negative attention, especially when there are political movements that attack wealth and/or privilege

I agree that there definitely is a general human instinct to suppress the successful ones. I am much less certain about the exact details of what triggers it, and how could it possibly be overcome.

Notice that there is also an opposite instinct to worship the successful ones: many people practically worship their leaders, sportsmen, actors... What decides which reaction activates when they see success?

(Also, how do you distinguish between "people do X because they see others do X" and "there is a reason why multiple people independently decide to do X"? For example, my first reaction to the previous question was "people are likely to hate those who are hated by others, and worship those who are worshiped by others", which seems like something Girard might say, but that also seems like a lazy answer; maybe people arrive at the same conclusions because they see the same triggers.)

A possible explanation is that people try to suppress others when the distance between them is small, and worship them when the distance is large. If you are a peasant, a more successful peasant is probably a witch and needs to be burned... but a king is practically a different species. A person similar to you evokes the feeling of "it should have been me instead", which leads to resentment.

Familiarity breeds contempt / “A prophet is not without honor except in his own town, among his relatives and in his own home.” / “An expert is an ordinary fellow from another town.” = people are much less likely to admire someone whom they already knew before that person became famous.

So, to trigger the worship instinct, people should know as little as possible about your past; you should appear in front of them as a fully developed success. To a more credulous audience, you could perhaps give a story about how you were already awesome as a baby (it helps if someone else says that).

...this is not an optimistic perspective either: it offers you a recipe to individually overcome the talent suppression instinct (every time you level up, move to a different city, and get rid of everyone who knows you; use pseudonyms online, and burn them when you level up), but it does not suggest a way out for the society as a whole.

Yes, I think examples could make this much clearer.

That's too abstract, I have no idea what it is supposed to mean and how it is supposed to be used.

When I think about a good business idea, but end up doing nothing, I often later find out that someone else did it.

How would a language like this survive a change in ontology? You take a category and split it into 5 subcategories. What if two years later you find out that a sixth subcategory exists?

If you update the language, you would have to rewrite all existing texts. The problem would not be that they contain archaic words -- it would be that all the words are still used, but now they mean something different.

Seemingly similar words (prepending one syllable to a long word or a sentence) will result in a wildly different meaning.

I think this article would be much better with many specific examples. (If that would make it too long, just split it into a series of articles.)

I agree. Any punishment in a system has the side effect of punishing you for using the system.

The second suggestion is an interesting one. It would probably work better if you had an AI watching you constantly and summarizing your daily activities. If doing some seemingly unimportant X predictably makes you more likely to do some desirable Y later, you want to know about it. But if you write your diary manually, there is a chance that you won't notice X, or won't consider it important enough to mention.

A summary, please?

Young men who make 9 figures by default get driven crazy, all checks and balances on them now gone.

I believe that I would have behaved quite responsibly; probably put all the money in index funds and live on the interest, and probably even keep a fake job (which would allow me as much work from home or vacation as I would need) and generally try to look inconspicuous. But I guess people this conservative usually don't make 9 figures. (Too late for the experiment, though; I am not young anymore.)

I would like to be able to follow people without worrying about what it looks like.

Perhaps there should be two options: follow publicly (maybe called "share") and follow privately.

Kelsey Piper discusses the administrative nightmare that is trying to use your home to do essentially anything in America.

I agree, but in the meanwhile, is there a way to outsource the bureaucratic part on someone? Like, if you want to make a shop in your garage, you could just call one, they would tell you the changes you will most likely be required to do, and you can pay them to do the paperwork. So you would still need to spend money and wait for an uncertain outcome, but you wouldn't need to deal with the paperwork, so you could do something else while waiting.

Tantum has a mostly excellent thread about the difference between a rival and an enemy, or between positive-sum rivalry and competition versus zero-sum hostility

Seems related to the paradox of tolerance. If the reason to allow multiple competing opinions is that empirically it makes the society better on average, this does not need to extended to the opinions that empirically make the society worse quite predictably. Tolerance is a means, not an end (the end is something like human flourishing), so there is no need to be absolutist about it.

And yet, even if some things are clearly harmful, it is difficult to draw the exact line, and often profitable to sacrifice to Moloch by getting closer to the line than your opponent.

Megan McArdle reminds us that Levels of Friction are required elements of many of civilization’s core systems, and without sufficient frictions, those systems break.

Yes, some things can be good if only a few people do them, but a disaster if too many start to do. This is difficult to communicate, because many people only think in the categories of "good" and "bad", and require some consistent principle that if it is okay for 1 person to do something, it is also okay for 1 000 000 people to do the same thing.

It would probably be bad to say that 1 specific person is allowed to do X, but 999 999 other people are not. But it makes perfect sense to say that it is okay when 1 person does X, but the system will collapse when 1 000 000 people decide to so it.

I’m surprised we don’t have a word for the shift when the bids for your time goes above your supply for time vs before, it feels like a pretty fundamental life shift where it changes your default mode of operation.

"Annual income twenty pounds, annual expenditure nineteen pounds nineteen and six, result happiness. Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery." -- This, but about your free time.

cutting corners, lying, and cheating will get you ahead in the short run, and sometimes even in the long run, but tying your own fortunes to someone who behaves this way will go very badly for you.

The difference between being the one who cheats, and associating with someone who cheats: If you cheat, there are (to simplify it a lot) two possible outcomes: you win, or you lose. If you associate with the cheater, the outcome "he wins" still has a very large subset of "he wins, but betrays you"; so the "you win" part is very small.

I guess people overestimate their ability to make a win/win deal with an experienced cheater. They either assume some "honor among thieves" (if I help him scam those idiots, surely he will feel some gratitude), or rely on some kind of mutually assured destruction (if he tried to stab me in the back, I would turn against him and expose him, and he knows that, therefore he wouldn't try).

But that doesn't work. The former, because from his perspective, you are just another one in the long line of idiots to be scammed. The latter, because he is already planning this a few moves ahead of you, and probably already had some experience in the past, so when he finally turns against you, you will probably find yourself in some kind of trap, or you will find him immune against your attempts at revenge.

Acid rain is the classic example of a problem that was solved by coordination, thus proving that such coordination only solves imaginary problems. Many such cases.

Is there some (ethically horrible, but justifiable by long-term consequentialism) solution to this? For example, whenever you vaccinate children, always deny the vaccine to randomly selected 1%, so that some children keep dying, so that everyone knows that the disease is real and the vaccine necessary?

we have systematized VC-backed YC-style founders

Commoditize your complement.

Chinese TikTok claims to spill the tea on a bunch of ‘luxury’ brands producing their products in China, then slapping ‘Made in Italy’ style tags on them. I mean, everyone who is surprised raise your hand, that’s what I thought, but also why would the Chinese want to be talking about it if it was true?

Maybe they think it will make people more okay to buy Chinese stuff that doesn't even pretend to be Italian, because they will realize they were buying that already?

My experience with manipulators is that they understand what you want to hear, and they shamelessly tell you exactly that (even if it's completely unrelated to truth). They create some false sense of urgency, etc. When they succeed to make you arrive at the decision they wanted you to, they will keep reminding you that it was your decision, if you try to change your mind later. Etc.

The part about telling you exactly what you want to hear gets more tricky when communicating with large groups, because you need to say the same words to everyone. One solution is to find out which words appeal to most people (some politicians secretly conduct polls, and then say what most people want to hear). Another solution is to speak in a sufficiently vague way that will make everyone think that you agree with them.

I could imagine an AI being superhuman at persuasion simply by having the capacity to analyze everyone's opinions (by reading all their previous communication) and giving them tailored arguments, as opposed to delivering the same speech to everyone.

Imagine a politician spending 15 minutes talking to you in private, and basically agreeing with you on everything. Not agreeing in the sense "you said it, the politician said yes", but in the sense of "the politician spontaneously keeps saying things that you believe are true and important". You probably would be tempted to vote for him.

Then the politician would also publish some vague public message for everyone, but after having the private discussion you would be more likely to believe that the intended meaning of the message is what you want.

I use written English much more than spoken English, so I am probably wrong about the pronunciation of many words. I wonder if it would help to have a software that would read each sentence I wrote immediately after I finished it (because that's when I still remember how I imagined it to sound).

EDIT: I put the previous paragraph in Google Translate, and luckily it was just as I imagined. But that probably only means that I am already familiar with frequent words, and may make lots of mistakes with rare ones.

This may be obvious, but you can't learn a language by memorizing words only. You need to speak entire sentences in that language, to train your inner LLM. Maybe try an audiobook.

I guess I have a different model of DOGE. In my opinion, their actual purpose is to have a pretext to fire any government employee that might pose an obstacle to Trump. For example, suppose that you start investigating some illegal activity and... surprise!, the next day you and dozen other randomly selected people are fired in the name of fighting bureaucracy and increasing government efficiency... and no one will pay attention to this, because too many things like that happen every day.

Just checking if I understood your argument: is the general point that an algorithm that can think about literally everything is simpler and therefore easier to make or evolve than an algorithm that can think about literally everything except for itself and how other agents perceive it?

I approximately see the context of your question, but I am not sure what exactly are you talking about. Maybe please try less abstract, more ELI5, with specific examples what you mean (and the adjacent concepts that you don't mean)?

Is it about which forces direct agent's attention in short term? Like, a human would do X, because we have an instinct to do X, or because of a previous experience that doing X leads to pleasure, either immediately or in longer term. And avoid Y, because of innate aversion, or a previous experience that Y causes pain.

Seems to me that "genetics" is a different level of abstraction than "pleasure and pain". If I try to disentangle this, it seems to me that humans

• immediately act on a stimulus (including internal, such as "I just remembered that...")
• that is either a hardwired instinct, or learned i.e. a reaction stored in memory
• the memory is updated by things causing pleasant or painful experience (again, including internal experience, e.g. hearing something makes me feel bad, even if the stimulus itself is not painful)
• both the instincts and the organization of memory are determined by the genes
• which are formed by evolution.

Do you want a similar analysis for LLMs? Do you want to attempt to make a general analysis even for hypothetical AIs based on different principles?

Is the goal to know all the levels of "where we can intervene"? Something like: "we can train the AI, we can upvote or downvote its answers, we can directly edit its memory..."?

(I am not an expert on LLMs, so I can't tell you more than the previous paragraph contains. I am just trying to figure out what is the thing you are interested in. It seems to me that people already study the individual parts of that, but... are you looking for some kind of more general approach?

The words don't ring a bell. You don't provide any explanation or reference, so I am unable to tell whether I am unfamiliar with the concept, or just know it under a different name (or no name at all).

In a more peaceful world the science advanced faster and the AI already killed us?

Some of the problems you mentioned could be solved by creating a wrapper around the AI.

Technologically that feels like taking a step back -- instead of throwing everything at the AI and magically getting an answer, it means designing a (very high-level) algorithm. But yeah, taking a step technologically is the usual response to dealing with limited resources.

For example, after each session, the algorithm could ask the AI to write a short summary. You could send the summary to the AI at the beginning of the new session, so it would kinda remember what happened recently, but also have enough short-term left for today.

Or in a separate chat, you could send the summaries of all previous sessions, and ask the AI to make some observations. Those would be then delivered to the main AI.

Timing could be solved by making the wrapper send a message automatically each 20 seconds, something like "M minutes and S seconds have passed since the last user input". The AI would be instructed to respond with "WAIT" if it chooses to wait a little longer, and a text if it wants to say something.

"AI fiction seems to be in the habit of being interesting only to the person who prompted it"

Most human fiction is only interesting to the human who wrote it. The popular stuff is but a tiny minority out of all that was ever written.

So it's like a lottery where you can e.g. increase your possible winnings ×2, by reducing your chance to win ÷3 ?

On average a bad move, but if you only look at the people who won most, it seems like the right choice.

Thank you for doing the epistemic hard work in difficult, potentially high-stakes situations!

Our usual heuristics stop working when the numbers get high. For example, there is a small but nonzero probability that a random mentally ill person will conclude that you are a leader of the worldwide conspiracy against them, and therefore it is necessary to kill you. In normal situations, the probability is small enough that you simply ignore it; you are not going to include this possibility in your decisions about how to live your everyday life.

Then, by your hard work and lots of luck, your project succeeds, and you become a famous person, known by millions of people. Congratulations! Also, a few weeks later, a mentally ill person breaks into your home and murders you. They explain to the police that you were a leader of the worldwide conspiracy against them. The probability of any specific mentally ill person doing this is too small, but once you are known to millions of people, which includes thousands of seriously mentally ill people, the small probabilities can quickly add up.

Every time your organization scales 10x, it exposes itself to yet another magnitude of weirdness. If you have 10 members, one of them will be really annoying. If you have 100 members, one of them will steal something from another member. If you have 1000 members, one of them will rape another member. If you have 10000 members, one of them will murder another member. If you have 100000 members... I don't know what happens at that scale, but it is certainly a horrible thing. The shields of "yes, this is technically possible, but come on, the probability of that happening in real life is extremely small" are falling apart one by one.

Then you have the noise which makes it impossible to figure out things already on the scale of 100 people.