Posts
Comments
Even if we assume that's true (it seems reasonable, though less capable AIs might blunder on this point, whether by failing to understand the need to act nice, failing to understand how to act nice or believing themselves to be in a winning position before they actually are), what does an AI need to do to get in a winning position? And how easy is it to make those moves without them being seen as hostile?
An unfriendly AI can sit on its server saying "I love mankind and want to serve it" all day long, and unless we have solid neural net interpretability or some future equivalent, we might never know it's lying. But not even superintelligence can take over the world just by saying "I love mankind". It needs some kind of lever. Maybe it can flash its message of love at just the right frequency to hack human minds, or to invoke some sort of physical effect that let's it move matter. But whether it can or not depends on facts about physics and psychology, and if that's not an option, it doesn't become an option just because it's a superintelligence trying it.
What odds are you willing to give taking friction into account?
How much are you willing to bet? I’ll take you up on that up to fairly high stakes.
I agree that alien visits are fairly unlikely, but not 99% unlikely.
-
This seems untrue. For one thing, high-powered AI is in a lot more hands than nuclear weapons. For another, nukes are well-understood, and in a sense boring. They won’t provoke as strong of a “burn it down for the lolz” response as AI will.
-
Even experts like Yann LeCun often do not merely not understand the danger, they actively rationalize against understanding it. The risks are simply not understood or accepted outside of a very small number of people.
-
Remember the backlash around Sydney/Bing? Didn’t stop her creation. Also, the idea that governments are working in their nations’ interests does not survive looking at history, current policy or evolutionary psychology (think about what motivations will help a high-status tribesman pass on his genes. Ruling benevolently ain’t it.)
-
You think RLHF solves alignment? That’s an extremely interesting idea, but so far it looks like it Goodharts it instead. If you have ideas about how to fix that, by all means share them, but there is as yet no theoretical reason to think it isn’t Goodharting, while the frequent occurrence of jailbreaks on ChatGPT would seem to bear this out.
-
Maybe. The point of intelligence is that we don’t know what a smarter agent can do! There are certainly limits to the power of intelligence; even an infinitely powerful chess AI can’t beat you in one move, nor in two unless you set yourself up for Fool’s Mate. But we don’t want to make too many assumptions about what a smarter mind can come up with.
-
AI-powered robots without super intelligence are a separate question. An interesting one, but not a threat in the same way as superhuman AI is.
-
Ever seen an inner city? People are absolutely shooting each other for the lolz! It’s not everyone, but it’s not that rare either. And if the contention is that many people getting strong AI results in one of them destroying the world just for the hell of it, inner cities suggest very strongly that someone will.
Exercise and stimulants tend to heighten positive emotions. They don’t generally heighten negative ones, but that’s probably all to the good, right? Increased social interaction, both in terms of time and in terms of emotional closeness, tends to heighten both positive and negative emotions.
Strongly upvoted. This is a very good point.
150,000 people die every day. That's not a small price for any delays to AGI development. Now, we need to do this right: AGI without alignment just kills everyone; it doesn't solve anything. But the faster we get aligned AI, the better. And trying to slow down capabilities research without much thought into the endgame seems remarkably callous.
Eliezer has mentioned the idea of trying to invent a new paradigm for AI, outside of the conventional neural net/backpropagation model. The context was more "what would you do with unlimited time and money" than "what do you intend irl", but this seems to be his ideal play. Now, I wish him the best of luck with the endeavor if he tries it, but do we have any evidence that another paradigm is possible?
Evolved minds use something remarkably close to the backprop model, and the only other model we've seen work is highly mechanistic AI like Deep Blue. The Deep Blue model doesn't generalize well, nor is it capable of much creativity. A priori, it seems somewhat unlikely that any other AI paradigm exists: why would math just happen to permit it? And if we oppose capabilities research until we find something like a new AI model, there's a good chance that we oppose it all the way to the singularity, rather than ever contributing to a Friendly system. That's not an outcome anyone wants, and it seems to be the default outcome of incautious pessimism.
This is excellently written, and the sort of thing a lot of people will benefit from hearing. Well done Zvi.
Well said! Though it raises a question: how can we tell when such defenses are serving truth vs defending an error?
As for an easier word for “memetic immune system”, Lewis might well have called it Convention, as convention is when we disregard memes outside our normal mileu. Can’t say for Chesterton or Aquinas; I’m fairly familiar with Lewis, but much less so with the others apart from some of their memes like Chesterton’s Fence.
Good analogy, but I think it breaks down. The politician’s syllogism, and the resulting policies, are bad because they tend to make the world worse. I would say that Richard’s comment is an improvement, even if you think it might be a suboptimal one, and that pushing back against improvements tends to result in fewer improvements. Don’t let the perfect be the enemy of the good is a saying for very good reason.
The syllogism here is more like:
-
Something beneficial ought to be done
-
This is beneficial.
-
Therefore I probably ought not to oppose this, though if I see a better option I’ll do that instead of doubling down on this.
How functional can our community be without pushing back against people like Ziz? Richard’s comment seems to be a way of doing so, and thus potentially useful. It’s fine if you disagree with him, but while I agree the comment was flag-planting, some degree of flag-planting is likely necessary for a healthy discussion. Consider the way well kept gardens die by pacifism (can’t link on my phone, but if you’re not familiar with it there’s an excellent Yudkowsky post of that name that seems relevant). Zizianism is something worth planting a few flags to stop.
This is the sort of work we need to be doing to understand neural nets. Excellent job!
The wanting vs liking distinction seems relevant here. Politics can be truly fun, especially when you're discussing it with someone who's clearly presenting their views in good faith, and when you can both learn something from the interaction. However, it's easy for the wanting to stay strong long after the liking has completely disappeared.
I wonder if that's a common trait of most or all addictive things, or at least of "non-physical" addictions (things where you don't suffer withdrawals, yet still may find yourself spending more time on them than you wish while not enjoying them much or at all). These days, Twitter is the classic example of an unfulfilling time sink. But Twitter really is great, at least when you're starting out by learning news and seeing new ideas from your favorite thinkers. But the urge for "just another tweet" can persist for hours, while the fun of it, in my experience at least, lasts more like fifteen or twenty minutes.
It is-for a certain type of unstable person. Ziz would likely have come up with different crazy ideas without Less Wrong. Compare Deepak Chopra on quantum mechanics: he pushes all manner of “quantum” bullshit, yet you can hardly blame physics for this, and if physics weren’t known, Chopra would almost certainly just be pushing a different flavor of insanity.
More like "enjoy the dive!"
Combating bad regulation isn’t a solution, but a description of a property you’d want a solution to have.
Or more specifically, while you could perhaps lobby against particular destructive policies, this article is pushing for “helping [government actors] take good actions”, but given the track record of government actions, it would make far more sense to help them take no action. Pushing for political action without a plan to steer that action in a positive direction is much like pushing for AI capabilities without a plan for alignment… which we both agree is insanely dangerous.
The state is not aligned. That should be crystal clear from the medical and economic regulations that already exist. And bringing in a powerful Unfriendly agent into mankind’s efforts to create a Friendly one is more likely to backfire than to help.
How do you propose nudging regulation to be better without nudging for more regulation?
Regulation in most other areas has been counterproductive. In AI, it will likely be even more so: there's at least some understanding of e.g. medicine by both the public and our rulers, but most people have no idea about the details of alignment.
This could easily backfire in countless ways. It could drive researchers out of the field, it could mandate "alignment" procedures that don't actually help and get in the way of finding procedures that do, it could create requirements for AIs to say what is socially desirable instead of what is true (chatGPT is already notorious for this), making it harder to tell how the AI is functioning...
It is socially desirable to call for regulation as a solution for almost any problem you care to name, but it is practically useful far more rarely. This is AI alignment. This is potentially the future of humanity at stake, and all human values. If we cannot speak the truth here, when will we ever speak it?
There are, of course, potentially reasonable counterarguments. Someone might believe that AI capabilities are more fragile than AI alignment, for instance, such that regulation would tend to slow capabilities without greatly hampering alignment, and the time bought gave us a better chance of a good outcome. Perhaps. But please consider, are you calling for regulation because it actually makes sense, or because it's the Approved Answer to problems?
Please don't make this worse.
There's potentially an aspect of this dynamic that you're missing. To think an opponent is making a mistake is not the same thing as them not being your opponent (as you yourself point out quite rightly, people with the same terminal goals can still come into conflict around differences in beliefs about the best instrumental ways to attain them), and to think someone is the enemy in a conflict is not the same thing as thinking that they aren't making mistakes.
To the extent that Mistake/Conflict Theory is pointing at a real and useful dichotomy, it's a difference in how deep the disagreement is believed to lie, rather than a binary between a world of purely good-faith allies who happen to be slightly confused and a world of pure evil monsters who do harm solely for harm's sake. And that means that in an interaction between dissidents and quislings, you probably will get the dynamic that Zack is pointing out.
Dissidents are likely to view the quislings as being primarily motivated by trying to get personal benefits/avoid personal costs by siding with the regime, making the situation a matter of deliberate defection, aka Conflict Theory. Quislings are likely to view dissidents (or at least to claim to) as misguided (the Regime is great! How could anyone oppose it unless they were terminally confused?), aka Mistake Theory. However, this Mistake Theory perspective is perfectly compatible with hating dissidents and levying all manner of violence against them. You might be interested in watching some interviews with pro-war Russians about the "Special Military Operation": a great many of them evince precisely this perspective, accusing Ukrainians of making insane mistakes and having no real interests opposed to Russia (i.e. they don't view the war through Conflict Theory!), but if anything that makes them more willing to cheer on the killing of Ukrainians, not less. It's not a universal perspective among Putin's faithful, but it seems to be quite common.
The dynamic seems to be not so much that one side views the other with more charity ("oh, they're just honestly mistaken; they're still good people") so much as that one side views the other with more condescension ("oh our enemies are stupid and ignorant as well as bad people").
That’s a documentary about factory farming, yes? What people do to lower animals doesn’t necessarily reflect what they’ll do to their own species. Most people here want to exterminate mosquitoes to fight diseases like malaria. Most people here do not want to exterminate human beings.
"Men care for what they, themselves, expect to suffer or gain; and so long as they do not expect it to redound upon themselves, their cruelty and carelessness is without limit."-Quirinus Quirrell
This seems likely, but what is your evidence for it?
This is blatantly wrong. Restricting the lives of other people just to gain a little more cachet for you and your fellow urbanites. Clear defection, clear evil.
How is that a response?
What is the specific difference between “regurgitated” information and the information a smart human can produce?
The human mind appears to use predictive processing to navigate the world, i.e. it has a neural net that predicts what it will see next, compares this to what it actually sees, and makes adjustments. This is enough for human intelligence because it is human intelligence.
What, specifically, is the difference between that and how a modern neural net functions?
If we saw a human artist paint like modern AI, we’d say they were tremendously talented. If we saw a human customer support agent talk like chatGPT, we’d say they were decent at their job. If we saw a human mathematician make a breakthrough like the recent AI-developed matrix multiplication algorithm, we’d say they were brilliant. What, then, is the human “secret sauce” that modern AI lacks?
You say to learn how machine learning works to dispel undue hype. I know how machine learning works, and it’s a sufficiently mechanical process that it can seem hard to believe that it can lead to results this good! But you would do well to learn about predictive processing: the human mind appears similarly “mechanical”. When we compare AI/ML to actual human capabilities, rather than to treating ourselves as black boxes and assuming processes like intuition and creativity are magical, AI/ML comes out looking very promising.
https://davidrozado.substack.com/p/what-is-the-iq-of-chatgpt
I would like to leave this here as evidence that the model stated above is not merely right on track, but arguably too conservative. I was expecting this level of performance in mid 2023, not to see it in January with a system from last year!
Have you tried this before? It sounds potentially helpful, but there’s nothing about what you’ve achieved with this method, only why it might work.
This is true, but it doesn’t answer the question of why not to simply use nuclear blackmail on such states. And the answer to that is that the US wants to limit the destruction of war. Nuclear blackmail is great, right up until someone calls your bluff. But then it helps to have conventional forces if you do not wish to have massive losses to local civilians, local infrastructure, and one’s own prestige.
"There are many animals which have what are called dominance contests. They rush at each other with horns - trying to knock each other down, not gore each other. They fight with their paws - with claws sheathed. But why with their claws sheathed? Surely, if they used their claws, they would stand a better chance of winning? But then their enemy might unsheathe their claws as well, and instead of resolving the dominance contest with a winner and a loser, both of them might be severely hurt." -Professor Quirell
Or to be more explicit, anything less than total war is a dominance contest between factions, not a no-holds-barred attempt to win. Nuclear weapons are useful for deterrence, but if there is a situation in which neither side is willing to simply back down, but both also want to limit the destruction, then a conventional military becomes very helpful.
Citation very much needed. What, specifically, do you disagree with?
Do you believe that the human mind is magical, such that no computer could ever replicate intelligence? (And never mind the ability it has shown already from chemistry to StarCraft…)
Do you believe that intelligence cannot create better tools than already exist, such that an AI couldn’t use engineering to meaningful effect? How about persuasion?
Do you believe that automation taking over the economy wouldn’t be a big deal? How about taking over genetics research, which is often bottlenecked by an inability to consider how genes interact, precisely something a computer could help with? Or is learning how to alter our very cores no big deal?
Do you have a specific argument against the plausibility or significance of a singularity? Or is this simply pattern matching to a cult without any further thought? Because “this sounds weird; it must be wrong” simply doesn’t work. Flight, nuclear power, genetics-all sounded more like science fiction than any real world possibility.
Fair enough, but it is equally incomplete to pretend that that’s an argument against the possibility of singularity-grade technology emerging in the foreseeable future.
By analogy, there have been many people who had crazy beliefs about radioactivity: doctors who prescribed radium as medicine, seemingly on the grounds that it was cool, and anything cool has to be good for you right? (A similar mentality led some of the ancient Chinese to drink mercury.) Atomic maximalists, who thought that anything and everything would get better with a reactor strapped to it, and never mind the price of uranium, the need for radiation shielding or the fact that reactors are heavy due both to the need for cooling and power generation systems and the simple fact that they benefit greatly from economies of scale. Not the sort of thing that you necessarily want to bolt onto every car and aircraft! Atom-phobes who were convinced that any attempt to utilize nuclear power would automatically become the next Chernobyl.
All of these were crazy, cult-like beliefs. Yet the insanity of people who turned poorly-understood scraps of nuclear theory into unreasoning optimism or pessimism does not have a single thing to say on the reality of radioactivity. Atomic bombs and nuclear reactors still work, no matter how foolish the radium suppository crowd of the early twentieth century was. And they still have sharp limits, no matter how crazily enthusiastic the “atomic cars in twenty years” crowd was.
By all means point out how Ziz’ cult was influenced by singularitarian ideas here. Even point out how the great opportunities and risks that a singularity might bring are a risk factor for cult-style mistakes. But don’t pretend that that prevents advanced technology from existing. Nature simply doesn’t care how we think about it, and isn’t going to make AI impossible just because Ziz had foolish ideas about AI.
While true, that’s not actually relevant here. While LW does not have perfect agreement on exactly how morality works, we can generally agree that preventing vaccine waste is a good idea (at least insofar as we expect the vaccine to be net-beneficial, and any debates there are largely empirical disagreements, not moral ones). Nearly all consequentialists will agree (more people protected), as well as deontologists (it’s generally desirable to save lives, and there’s no rule against doing so by utilizing vaccines that would otherwise end up in the trash) and virtue ethicists (saving lives is virtuous, and it’s hardly a vice to help prevent waste; if anything it’s the virtue of frugality).
Insofar as we expect the vaccine to be beneficial (yes there’s a potential debate there, but that’s a different topic), saying that there’s a lot of moral weight here is perfectly reasonable. Consider a phlogiston theorist and a modern chemist looking at a bonfire: they might not agree on the precise nature of fire, but they can certainly agree that there’s a lot of fire present. So too here.
Strongly upvoted for clarification and much greater plausibility given that clarification.
"Back then it was called Czechoslovakia. I am puzzled about the disagreement votes, given that I have hedged my statement as "try to teach you, even if not very efficiently". Not sure how people do things on the other side of the planet, but I imagine that there are these things called textbooks, which are full of information, and they at least make you read them. I am not saying that the information is especially useful, or especially well explained; just that it is there, and the school exposes you to it."
Typically the material covered in a given day was a repeat, nearly word for word, of the previous day's lecture, or the previous month's. Seventh grade mathematics spent over a month on adding negative numbers, despite the fact that pretty much everyone already knew how to, and that if someone had somehow managed to avoid understanding how despite the same lecture being given every day, presumably they would need some other method of instruction than repeating it again. While this could, perhaps, be considered trying to teach a few days' worth of material over months and years, that is so far from what any sane person would do if they actually wanted students to learn that I think it's fair to disagree with saying that American schools aim to teach. If the Czechoslovakian system was better, well and good. I can actually believe that; for all the faults of the Eastern Bloc, it was reportedly quite good at education, and I learned a decent amount studying on my own from an excellent Russian textbook called алгебра и начало анализа.
There were textbooks, but they never made us read them. We typically did not have time to do so, with study time instead going to the aforementioned mostly-useless lectures and equally repetitive and useless homework.
"There were subjects that I hated, mostly history. That one was taught literally as a list of facts that I considered utterly irrelevant -- I couldn't care less about what year exactly which king was born, or what year exactly was a battle that happened centuries ago. There were subjects that I would have learned on my own anyway, especially math and computer science. A subject where the school provided the most added value for me, was probably chemistry. Explained sufficiently well that some information stuck in my brain, and yet not something I would have studied on my own."
Most of this seems like a good case against schooling. Your experience with chemistry is a fair point, but how much value have you gotten out of your chemistry knowledge, and if the answer is a lot, due to a job or the like, wouldn't you have studied it when you needed it? If the answer is that you never needed it for work, yet are much happier for the sheer beauty of the knowledge (admittedly plausible; there are many things in life that may not be immediately necessary to know, but are still incredibly fun to understand), and would never have studied it unless forced, that is a valid point in favor of compulsory schooling. I submit that it is outweighed by all the downsides, but that is a valid point. At a minimum we can agree that the American schooling system ought to be more like the Czechoslovakian one.
"Perhaps this is another case of what I call "America succeeds to go to all extremes simultaneously". Excellent universities, dystopian elementary and high schools. In Eastern Europe things are more... mediocre, at both extremes. The universities are meh, but the elementary schools are kinda okay, mostly. Or maybe one of us generalizes too much from personal experience. It would be interesting to make a survey (not limited to the rationalist community)."
That is a good point, and quite plausible. To be perfectly honest, I have a hard time imagining a non-dystopian elementary or high school; the concept sounds like a non-dystopian prison sentence. It is very interesting to talk to someone who apparently went through a plausibly non-evil school system.
"I was seriously considering the possibility of homeschooling my children, but it turned out that my daughter enjoys school (she is currently in 2nd grade), so I am like: "well, if there is no problem, I do not need to solve it". Of course, enjoying and learning new things are not the same; she probably likes the fact that she is one of the best in the classroom. Then she does Khan Academy and Duolingo at home. That doesn't really seem like an argument in favor of the system... unless the point is that she has enough time and energy left to pursue her own goals. (I am quite tired when I get home from work.)"
If she enjoys it, fair enough, though I would strongly recommend keeping an eye out for the possibility that it's a poor use of her time or may become so, even if she's happy so far. But if she's learning (and Khan Academy and Duolingo will see to it that she is) and happy, it sounds like things are going well.
"Dunno, maybe it's like the mass transit. Some places give up on the idea completely, and then it becomes really bad. Other places make it a priority, and then it is not necessarily great, but it is okay. There might literally be two attractors; either the idea of school is high-status or low-status, and that determines whether the teachers love or hate their jobs, etc."
That is an interesting idea. It seems implausible that public school could have reasonable incentives, given the inherent isolation of government programs from market forces and meaningful feedback. However the existence of a high status equilibrium for teachers resulting in at least a strikingly less horrific system is possible, and might explain why the Czechoslovakian educational system was apparently much better.
I would also like to thank you for the very polite tone of your reply. I am also trying to be polite and respectful, but this is enough of a touchy subject for me due to how horrific my schooling was that my previous comment held more than a little anger. Despite this you responded extremely well.
"Schools at least try to teach you."
I am curious where you went to school. That was not my experience, and I was in an unusually good school district by American standards. Some of my friends had noticeably worse experiences than I did. Are you conflating the nominal purpose of a school with its real-world actions? Alternatively, did you go through a good enough school system that it might be worth replacing a great many existing "educational" systems with yours as a stopgap along the way to school abolition?
"Jobs typically do not try at all... and when they try, it is usually very narrow, the thing you will immediately need for your work."
Exactly! Surely that is precisely how it ought to be? Forced learning is a difficult thing to justify, and when a job teaches you something, it is incentivized to help you learn it in short order. Note that I am not calling for removing the option to learn and/or take classes, just removing the requirement. Likewise I am not calling for the forced imposition of child labor, just wondering if children who want to work should have the right, and questioning the idea that it's so much worse than forced schooling when by many metrics it's better.
"In school there is more slack; you can spend a lot of time daydreaming, you finish earlier, you have summer holidays. At work you get responsibilities, unrealistic deadlines, for many people it is difficult to stop thinking about their job when the shift ends." This is true, and a fair point. But a situation that creates little to no to negative value is bad, even if there's enough slack to make it less bad than it could conceivably be. And a situation that creates meaningful economic value is at least potentially worthwhile (again, I suspect a minority of children would choose to work, and none of them should be forced to. But the option should potentially exist, and morally it's plausibly better than an "education" that tends not to be actually educational.)
"The "childhood magic" from my perspective means, importantly, not being responsible for paying the mortgage or making enough money to buy food. As long as that is your parents' responsibility, and you are allowed to completely ignore this, you are mentally a child. When the metaphorical gun is put to your temple, adulthood begins."
Alright. But A: That can coexist with working (maybe you keep all your money and let your parents worry about the mortgage and grocery budget) and B: there are people placed under incredible stress to perform in school, sometimes much greater than the stress that most adults face. You can argue that it shouldn't be as stressful, given that even total failure is unlikely to starve you, but in practice it often is.
How is this different from adults having jobs?
To be clear, there are plenty of good reasons why one might not want children to work. You might want them to be able to enjoy childhood without the burden of a job, you might want them to focus on learning to be more productive later. But "the people paying them are motivated by profit" is equally true of adult jobs.
"Oh right, the whole world doesn't have education as a right."
Are you trying to argue from existing law to moral or practical value? That would be easier if the whole world hadn't had slavery and monarchy until fairly recently.
"That both destroy magic doesn't mean the destruction is it to the same degree."
That's a good point. But jobs ideally produce value. School often doesn't, and "learning" in a toxic setting specifically makes it harder to learn later. That's a harm specific to school; most jobs do not have it.
"And school has its own magic. Jobs tend to have way less magic of their own."
I'm glad it was magical for you. That's far from universal. The largest problem with schooling is the compulsion. If you enjoyed it or benefited from it, well and good. But those who didn't should not have been forced into it. The alternative to compulsory schooling doesn't have to be no option to go, it can be letting people choose.
Upvoted for preregistration.
That ignores systematic problems with schooling, which even good schools will tend to suffer from:
Teaching by class risks both losing the kids at the bottom and boring the kids at the top, whereas individual study doesn't have this problem.
Teaching by lecture is much slower than learning by reading. Yes, some students benefit from audio learning or need to do a thing themselves to grasp it, but those capable of learning from reading have massive amounts of time wasted, as potentially do the kinesthetic types who should really be taking a hands-on approach.
Teaching a broad curriculum forces vast amounts of time and effort to go towards subjects a student will never use. Specialization avoids this. Broad curricula are sometimes justified on the grounds that they'll give a student more options later if they don't know what they want to do, or on the grounds that they make the student "well-rounded". However, the first justification seems extremely hollow in the face of opportunity costs and the tendency of aversive learning to make the victim averse to all learning in the future. The second, meanwhile, seems hard to take seriously upon actually experiencing "well-rounded" education or seeing its effects on others: it turns out people just don't tend to use ideas they're not interested in that were painfully forced into their minds.
Also relevant, though you could fairly note that the best schools will not suffer from these as much:
Public schools do not tend to benefit much from good performance nor suffer from bad. They are not incentivized to do a good job and thus tend not to.
Political and educational fads can result in large amounts of schooling going towards pushing pet ideas of the administrators, rather than anything that is plausibly worthwhile. This can even be worse than a simple waste of time: I've seen multiple classmates develop unhealthy guilt due to forced exposure to political propaganda.
You are correct that some schools are much better than others. But there are serious systematic problems here, and some schools being somewhat less bad doesn't change that fact.
How is this any different from school, except that you could get paid rather than your parents losing money to pay the teachers? There are many valid arguments against child labor (though also many valid arguments that the child should be allowed to decide for themselves), but nearly all of them apply to schooling as well. School eliminates the time of childhood magic, actively makes it harder to be curious (many jobs would not have this effect) and you don't even get paid.
I don’t know how common loss of attention span is, but certainly reduced interest in learning occurs extremely often.
Also, potential evidence that more damage occurs than is commonly recognized: in the modern world, we generally accept that one needs to be in one’s late teens or even early twenties to handle adult life. Yet for most of human history, people took on adult responsibilities around puberty. Part of the difference may be the world becoming more complex. But how much of it is the result of locking people up in environments with very little social or intellectual stimulation until they’re 18?
The world looks exactly like one would expect it to if school stunted intellectual and emotional maturity.
I would think that it's valid, but a smaller effect than getting taught a bundle of random things in a gratuitously unpleasant way resulting in those who have been taught in school having a deep-seated fear of learning, not to mention other forms of damage. Prior to going to school, I had an excellent attention span, even by adult standards. After graduating high school, it took two years before I could concentrate on anything, and I still suffer from brain fog.
Should society eliminate schools?
That depends on what would replace them. One could imagine a scenario in which schools were eliminated, no other form of learning filled the gap, and mankind ended up worse off as a result. However, schooling in its present form seems net-negative relative to most realistic alternatives. Much of this will focus on the US, as that is the school system I'm most familiar with, but many of the lessons should transfer.
Much of the material covered has no conceivable use except as a wasteful signal. "The mitochondrion is the powerhouse of the cell": everyone in the US gets taught that, but almost no one knows what it means in any real sense, nor does anyone benefit from knowing it unless they're either going into biology or interested in biology. And the people who are becoming biologists still need to know what that actually means! And that's even before we get to material like the fates of King Henry's wives: divorced beheaded died, divorced beheaded survived. In what world is that the most pressing thing to learn?
Even the plausibly-useful material tends to be covered slowly and with heavy emphasis on following steps by rote instead of understanding what's actually going on. Not only does that make that curriculum much less helpful for actual learning than one might expect from the topics, but it can actively drive students away from curiosity and critical thinking.
How many people have been traumatized into a fear of math?
On top of this, we must consider the price of schooling, both financial and opportunity costs. In fiscal 2022, the Department of Education consumed over 600 billion dollars. That's not trivial, and one wonders what other uses that amount of money could be put to. And children losing a large portion of their childhoods is a staggering human cost. And what do we get in return for such sacrifices? One in five high school graduates can't read. Over a decade of their lives taken from them in the name of learning, and they never even learned how to read.
If we hadn't grown up with school as a normal, accepted thing, if we weren't used to going along with it because it would be awkward not to, what would we see? What would you think about a society that locks children up to perform forced labor that isn't even economically productive, tries to justify it in the name of learning, then barely even teaches anything?
This is a crime against humanity.
Is that true? Isn't at least one clear difference that it's difficult to stop engaging in a bias, but heuristics are easier to set aside? For example, if I think jobs in a particular field are difficult to come by, that's a heuristic, and if I have reason to believe otherwise (perhaps I know a particular hiring agent and know that they'll give me a fair interview), I'll discard it temporarily. On the other hand, if I have a bias that a field is hard to break into, maybe I'll rationalize that even with my contact giving me a fair hearing it can't work. It's not impossible to decide to act against a bias, but it's harder not to overcorrect.
He cites the observation that socialized firms have not taken over the economy. That's clearly true and clearly relevant. Your response was that you'd already explained why socialized firms might not take over even if they were productive. What were those reasons again? Reviewing your post, it looks like it might be the difficulty of gaining investment and brain drain from the most productive workers leaving, but both of those reasons would be strong arguments against socialization. Rose Wrist's ideas for gaining investment anyway are interesting, but until socialized firms actually do raise enough funding to compete, saying that they theoretically maybe can sounds remarkably hollow.
The point of evidence is to see things that are more likely under one hypothesis than another. In the world where socialized firms are better, I do not expect to see them failing to take over. In the world where they are not, I do expect that it's possible to generate arbitrarily long lists of pro-socialism citations.
The strength of a case depends on the strength of the evidence, not on the number of citations!
The specific handwave I'm referring to is Amartya Sen's.
"In the case of the free rider hypothesis, these 'rational fools' act based on such a narrow conception of self-interest that they don't take into account the obviously damaging long-term consequences of their behavior, both to the firm and ultimately to themselves. Normal, reasonable people - who are different to rational economic man - are usually happy to put efforts into a collective endeavor that will deliver benefits for them in the long run, even if that means foregoing some short-term gains."
This sounds like it would predict that people reliably cooperate on prisoner's dilemmas, and pick stag in stag hunts. In reality, of course, that's not a thing! Cooperation exists, but tends to require coordination mechanisms. Worse, it sounds like it's advocating an incoherent decision theory. While there are certainly times where it's wise to make a choice that isn't the best in the most narrow, myopic possible sense (Newcomb's problem is the obvious example, or superrationality dynamics), that's very different than putting efforts into collective endeavor in the hopes of collective success.
The evidence you cite is interesting, though Lao Mein's evidence suggests it isn't a slam dunk. But Sen is committing a fallacy here, and the same fallacy as was often used in support of socialist regimes. As such, it's a valid answer to tailcalled's question.
Surely the good or bad effects of socialism are a function of policy? Whether or not a policy arises democratically and/or revolutionarily does not change the policy itself. This is a striking non-sequitur.
The Scandinavian countries are indeed pretty good places to live. This likely has nothing to do whatsoever with democracy per se, but with the fact that the Scandinavian model does not regulate to anything resembling more strongly socialist nations, despite the fact that they famously have a large welfare system. There is no casual mechanism whereby voting for a leader would make the policies of that leader better-though obviously a leader that harmed the people in legible-to-them ways might get voted out! But that would be democracy changing policy, not democracy making a given policy better. As a real-world test case, consider the Maduro regime in Venezuela. While his democratic bona fides are somewhat questionable (there are people who think he stole his election from Juan Guaido), he certainly had enough popular support to be a serious candidate. And that did not prevent his policies from having predictably impoverishing results on Venezuela.
At least one critical similarity is that this plan relies on people ignoring economic incentives, and tries to handwave this away by pretending that people will cooperate in the face of free rider dynamics in the hopes of future payoffs. If that was true, game theory would be a lot simpler.
Are you on Data Secrets Lox? That is much more the place for this sort of discussion, and it would let us talk about whatever you like without transgressing the no politics board.
Hence the charitable reading that the OP might be calling for a different version of socialism that might conceivably be beneficial. My point isn’t that there’s zero chance that he’s right; my point is that there’s no way to say “hey, let’s do this thing that’s superficially similar to catastrophic policies” without it either not conveying useful information, or that useful information requiring a long political debate to hash out. And that’s not appropriate for the “Politics is the mind-killer, let’s improve our rationality on easier cases” forum. I’d welcome the post and subsequent debate on e.g. a Scott Alexander forum or comment section. But this isn’t the place for it.
This is not a place for politics. It is especially not a place for politics that have consistently led to catastrophe. The uncharitable reading of this post is that it is simply ignoring the harms of socialism, which is a trivial error of rationality. The charitable reading is that it is proposing a new take on socialism which could actually be beneficial. However, explaining such a point and answering the inevitable objections requires a long political discussion on a board that was explicitly created to avoid such things, due to their tendency to make rationality much harder.
If the charitable reading is correct, this might be an interesting debate, and even perhaps one where we could all learn something. But this simply isn’t the forum for it.
On the one hand, that's literally true. On the other, I feel like the connotations are dangerous. Existential risk is one of the worst possible things, and nearly anything is better than slightly increasing it. However, we should be careful that that mindset doesn't lead us into Pascal's Muggings and/or burnout. We certainly aren't likely to be able to fight existential risk if it drives us insane!
I strongly suspect that it's not self-sacrificing researchers who will solve alignment and bring us safely through the current crisis, but ones who are able to address the situation calmly and without freaking out, even though freaking out seems potentially justified.
Wouldn't it be relevant in that someone could recognize unproductive, toxic dynamics in their concerns about AI risk as per your point (if I understand you correctly), decide to process trauma first and then get stuck in the same sorts of traps? While "I'm traumatized and need to fix it before I can do anything" may not sound as flashy as "My light cone is in danger from unaligned, high-powered AI and I need to fix that before I can do anything", it's just as capable of paralyzing a person, and I speak both from my own past mistakes and from those of multiple friends.