Posts
Comments
Yeah that sounds very plausible. It also seems plausible that we get regulation about transparency, and in all the cases where the benefit from interpretability has something to do with people interacting you get the results being released at least semi-publicly. Industrial espionge also seems a worry. The USSR was hugely successful in infultrating the Manhatten project and contined to successfully steal US tech throughout the cold war.
Also worth noting the more information about how good one's own model is also increases AI risk in the papers model, although they model it as a discrete shift from no information to full information so unclear well that model applies.
I think there are quite a lot of worlds where understanding the black box better is bad.
If alignment is really really hard we should expect to fail in which case the more obvious it is that we've failed the better because the benefits of safety aren't fully externalised. This probably doesn't hold in worlds where we get from not very good AI to AGI very rapidly.
Potentially counterintuitive things happen when information gets more public. In this paper https://www.fhi.ox.ac.uk/wp-content/uploads/Racing-to-the-precipice-a-model-of-artificial-intelligence-development.pdf
increasing information has weird non-linear effects on the amount spent on safety. One of the pieces of intuition behind that is that having more information about your competitors can cause you to either speed up or slow down depending on where they in fact are in relation to you.
Also seems like risk preferences are important here. If people are risk averse then having less information about the expected outcomes of their models makes them less likely to deploy them all else equal.
I think I'm most excited about 15, 16 and 6b because of a general worldview of 1) alignment is likely to be really hard and it seems like we'll need assistance from the best aligned systems to solve the problem and 2) that ~all the risk comes from RL agents. Getting really really good microscope AI looks really good from this perspective, and potentially we need a co-ordinated movement towards microscope AI and away from RL models in which case building a really compelling case for why AGI is dangerous looks really important.
https://plato.stanford.edu/entries/ramsey/#Life
Paragraphs 2 and 3 of the 'Life and works' section
He convinced Keynes that Bayesianism was correct!
I prefer hypocrisy to cruelty.
More gennerally I think this just misses the point of drowning child. The argument is not that you have this set of preferences and therefore you save the child, the argument is that luxury items are not of equal moral worth to the life of a child. This can be made consistent with taking off your suit first if think the delay has a sufficiently small probability of leading to the death of child and you think the death of a child and the expensive suit are comparable.
The claim that most EAs want to look like they're contributing to AI safety rather than having a deep commitment is just deeply at odds to my personal experience. The EAs I meet are in general the most committed people to solving problems that I've ever met. I might try to come up with a more systematic argument for this, but my gut is that that's crazy.
This may be random or something, but in my experience there is a higher probability that rationality people aren't committed to solving problems and want to use rationality to improve their lives personally. But outside view is that this shouldn't be surprising given that the core of the rationality movement is trying have true beliefs.
Great article, small (but maybe significant) nitpick. I don't think there's great evidence that innovation-by-bureaucracy is bad, and I actually think it's pretty good. The sheer quantity of innovation produced by the two world wars and space program is spectacular, as is the record of DARPA post WW2. Even the Soviet Union could do could innovation by bureaucracy. At the company level, it's true that lots of big innovations come from small new companies, but at the same time Bell Labs, Intel, Sony were monsters. I actually think that the retreat of government is a big part of problem.
Ok, so we have different epistemic systems, I obviously put much higher value academic social science ,where I think the methodology is good, than you do.
It's a plausible claim that people with lower impulse control less enjoyment of experience, it's not one that I've seen made before, my intuition says the effect size is probably small. Fundamentally, show me the evidence, although presumably it can't be from academic social science.
Finally, it's important to distinguish between empirical and moral claims. So, I think prisons are bad but in principle can be morally justified. I think mafia bosses should go to prison because I'm not a patient centered deontologist who rejects the the legitimacy of the state. Similarly, I think there are lots of plausible moral systems in which restricting children's sovereignty can be fine or even morally required.
On the empirical matter, I don't know we just disagree. But I'm 19, I was in school until I was 18, it was fine. My brother's 17, I asked him, he said schools fine. I went to a normal school, he went to a normal school. My girlfriends also 19, she liked school. Maybe we're all massive outliers, but when you ask people who've been to prison what they think of prison they say it's terrible and when you ask people who've been to school what they think of school they mostly say it's fine.
I think it's a good insight and one I basically agree with that children, before they're 16 or so in the UK at least, are treated in a way which robs them of their sovereignty and all else equal this is bad for both deontic and consequentialist reasons. I basically agree that the vast majority of content taught to 11-16 year neither increases their productivity in the workforce, nor do they enjoy studying it, nor does it make them better people. It is also true that when people are homeschooled or unschooled they do fine (I've read like 4 papers on outcomes of homeschooling and unschooling so I won't make a stronger empirical claim than 'fine'), and that biggest difficulty that certainly unschoolers have is others reactions.
However, I think you're dramatically overstating your case. I think schooling for 3-10 year olds is incredibly valuable. There's excellent empirical evidence that early years education is good for a whole range outcomes and is fantastic for the children of low income parents. In general both the actual content of the curriculum, i.e literacy and numeracy are very useful - functionally illiterate people have difficult lives. I there's also the effect of general improvement in cognitive capacity and teaching the ability to make abstract, logical deductions. We see the effect of this when we ask individuals in communities in which there is no primary schooling to think abstractly. The most of these studies is on Russian peasants in 1907 but this has been replicated for instance with modern hunter-gatherer tribes. I also think that the socialisation aspect is important as well as the teaching of non-cognitive skills - in Pinker's history of violence he provides evidence of the impact of schooling in reducing violence by improving impulse control, based on impulse control being a transferable, trainable characteristic.
I also just reject your claim that schooling is that unpleasant. Often it's the place where kids spend the most time interacting with their friends, it gives people achievable goals which they get clear, regular feedback on and that's pretty motiving in general and people who are bored often just fuck about at the back of the class which isn't ideal but also doesn't seem that bad. Just in general lots of the time in school is mostly made up of chatting to people you either don't mind, quite like, or forge some of the deepest and most valuable friendships you ever have with.
Potentially a way in which this heuristic makes sense in real world is if the utility of 0$ was negative. If I were an banana seller then if I sold nothing, when I got my next shipment of bananas I wouldn't have enough space in my warehouse and I'd have to throw out some bananas. In this case, I have to take out an insurance policy against 0$ in all cases except certainty. This will hold even if insurance costs are proportional to probability of 0$ if there were fixed transaction costs to buying insurance.
The sufficiency stagnation point is a good one, especially given that is suggests that the people becoming religious on the margin are likely to be the best individuals of the population not currently committed to strong social institutions, to start better ones than religions.
Potentially a crux is that the ideas that really broad social institutions can be based around may mostly be based around certain types of really strong emotions like tribalism and faith, the crux being if 'mostly' means 90%, 99% or 99.999%.
I think it's unlikely that there'd be a crowding out effect currently on the margin (although I expect you would as some point if you're attracting progressively less sociable people), as you say because it builds know how, but also because it builds social capital and maybe breaks the negative feedback loop of loneliness.
My second claim is that religion is much much better as a community organising force than any other institution other than unions. I think this is because it can attract a very high percentage of a population, it persists through gennerations, and there aren't the same types of barriers you get with groups organised around a specific interest, and they don't skew middle class (often at least.)
I think more people being religious is good on the margin and basically don't think that religion is a signifiant barrier to spread and advance of good ideas and practices.
I think your mistake is your crux not including the broader social benefits that religion brings. Religion is maybe the only, or at least one of very few, forces that can bring large numbers of different people from within a community together on a regular basis and, in Europe and probably the US this has enormous benefits. What research on happiness and well-being tell us is that, assuming you have your base material needs met, your social realations become the most important factor in your happiness. This I am very confident about.
What I am less confident about but I think is still important is that having good institutions for working and middle class people to organise around has historically been very important for them achieving good politcal outcomes, and socially desirable outcomes in the form of inclusive economic instituions to use Acemoglu and Robinson's framework. I think the success of Christian democratic parties in building the social democractic state in Northern and Western Europe and their taking the place of the aristocratic conservative parties is an example of this being relevent in the context of high income democratic political economy. The other very prominant example is the importance of black churches in the African-American political movement.This is very important if you think that technology is endogenous to political and economic institutions.
The other piece I am less confident about is that religion is in general a positive socialising force and a good way of making young men who would otherwise have poor life outcomes and cause social harm, to be turned into better citizens. Examples of this are the pretty robust evidence that students at Catholic Schools have better exam results, the success in intellectual fields of the European and American Jewry, and the importance of the Catholic Church in the rehabilitation of latin Americans involved gang crime.
The other key part I disagree about is the degree to which religion stops people from adopting Good ideas and Good practices. My argument for this is that religion is a major force in very few peoples lives in rich democracies. Very few people are observant of the religion that they say that they belive in. Church attendances are very low in all Christian countries except the US, and even there it's quite low. I have much less evidence for this applying today, but historically individuals who've wanted to adopt ideas that their religion would seem to prohibit have been able to find ways of making it work. For the great Enlightenment thinkers of Laplace, Paine, Lagrange, Locke this was a belief in a God that existed but had no impact on Earth, for the Slave owners of the US South it was that God was fine with slavery, for the emerging middle classes of Europe in the long 19th century it was Calvinism, for Democratic polticians representing 2nd and 3rd genneration of immigrants in the North East it was the switch from pro-life to pro-choice.
The bit of your argument that I buy, is that it seems very plausible that a very smart Hindu devotes his life to the study of Sanskrit rather than something you and I would consider more useful (as Aymrata Sen nearly did), and there's a lower probability that Ben Shapiro is a conservative and potentially that more Americans belive in climate change.
I've purposefully restcrited this to thinking about rich democracies because I think that's what your post was about. Hinduism in Indian is a different ball game and Islam in low income unstable countries again seems outside the scope of your post.