Posts

rhollerith_dot_com's Shortform 2022-01-21T02:13:20.810Z
One Medical? Expansion of MIRI? 2014-03-18T14:38:23.618Z
Computer-mediated communication and the sense of social connectedness 2011-03-18T17:13:32.203Z
LW was started to help altruists 2011-02-19T21:13:00.020Z

Comments

Comment by RHollerith (rhollerith_dot_com) on why won't this alignment plan work? · 2024-10-11T04:33:35.189Z · LW · GW

plug that utility function (the one the first AI wrote) into it

Could some team make an good AGI or ASI that someone could plug a utility function into? It would be very different from all the models being developed by the leading labs.

Comment by RHollerith (rhollerith_dot_com) on Thomas Kwa's Shortform · 2024-10-11T04:05:01.741Z · LW · GW

A brief warning for those making their own purifier: five years ago, Hacker News ran a story, "Build a do-it-yourself air purifier for about $25," to which someone replied,

One data point: my father made a similar filter and was running it constantly. One night the fan inexplicably caught on fire, burned down the living room and almost burned down the house.

Comment by RHollerith (rhollerith_dot_com) on Overview of strong human intelligence amplification methods · 2024-10-08T23:34:29.583Z · LW · GW

Meditation has been practiced for many centuries and millions practice it currently.

Please list 3 people who got deeply into meditation, then went on to change the world in some way, not counting people like Alan Watts who changed the world by promoting or teaching meditation.

Comment by RHollerith (rhollerith_dot_com) on If I have some money, whom should I donate it to in order to reduce expected P(doom) the most? · 2024-10-06T16:20:18.108Z · LW · GW

I'm not saying that MIRI has some effective plan which more money would help with. I'm only saying that unlike most of the actors accepting money to work in AI Safety, at least they won't use a donation in a way that makes the situation worse. Specifically, MIRI does not publish insights that help the AI project and is very careful in choosing whom they will teach technical AI skills and knowledge.

Comment by RHollerith (rhollerith_dot_com) on Bogdan Ionut Cirstea's Shortform · 2024-10-05T16:01:34.475Z · LW · GW
Comment by RHollerith (rhollerith_dot_com) on If I have some money, whom should I donate it to in order to reduce expected P(doom) the most? · 2024-10-03T13:39:02.673Z · LW · GW

TsviBT didn't recommend MIRI probably because he receives a paycheck from MIRI and does not want to appear self-serving. I on the other hand have never worked for MIRI and am unlikely ever to (being of the age when people usually retire) so I feel free to recommend MIRI without hesitation or reservation.

MIRI has abandoned hope of anyone's solving alignment before humanity runs out of time: they continue to employ people with deep expertise in AI alignment, but those employees spend their time explaining why the alignment plans of others will not work.

Most technical alignment researchers are increasing P(doom) because they openly publish results that help both the capability research program and the alignment program, but the alignment program is very unlikely to reach a successful conclusion before the capability program "succeeds", so publishing the results only shortens the amount of time we have to luck into an effective response or resolution to the AI danger (which again if one appears might not even involve figuring out how to align an AI so that it stays aligned as it becomes an ASI).

There are 2 other (not-for-profit) organizations in the sector that as far as I can tell are probably doing more good than harm, but I don't know enough about them for it to be a good idea for me to name them here.

Comment by RHollerith (rhollerith_dot_com) on Alignment by default: the simulation hypothesis · 2024-09-26T01:51:05.803Z · LW · GW

I'm going to be a little stubborn and decline to reply till you ask me a question without "simulate" or "simulation" in it. I have an unpleasant memory of getting motte-and-baileyed by it.

Comment by RHollerith (rhollerith_dot_com) on Alignment by default: the simulation hypothesis · 2024-09-25T19:14:34.753Z · LW · GW

Essentially the same question was asked in May 2022 although you did a better job in wording your question. Back then the question received 3 answers / replies and some back-and-forth discussion:

https://www.lesswrong.com/posts/vaX6inJgoARYohPJn/

I'm the author of one of the 3 answers and am happy to continue the discussion. I suggest we continue it here rather than in the 2-year-old web page.

Clarification: I acknowledge that it would be sufficiently easy for an ASI to spare our lives that it would do so if it thought that killing us all carried even a one in 100,000 chance of something really bad happening to it (assuming as is likely that the state of reality many 1000s of years from now matters to the ASI). I just estimate the probability of the ASI's thinking the latter to be about .03 or so -- and most of that .03 comes from considerations other than the consideration (i.e., that the ASI is being fed fake sensory data as a test) we are discussing here. (I suggest tabooiing the terms "simulate" and "simulation".)

Comment by RHollerith (rhollerith_dot_com) on rhollerith_dot_com's Shortform · 2024-09-24T23:11:59.484Z · LW · GW

I appreciate it when people repost here things Eliezer has written on Twitter or Facebook because it makes it easier for me to stay away from Twitter and Facebook.

(OTOH, I'm grateful to Eliezer for participating on Twitter because posting on Twitter has much higher impact than posting here does.)

Comment by RHollerith (rhollerith_dot_com) on Why the 2024 election matters, the AI risk case for Harris, & what you can do to help · 2024-09-24T21:06:05.270Z · LW · GW

You spend a whole section on the health of US democracy. Do you think if US democracy gets worse, then risks from AI get bigger?

It would seem to me that if the US system gets more autocratic, then it becomes slightly easier to slow down the AI juggernaut because fewer people would need to be convinced that the juggernaut is too dangerous to be allowed to continue.

Compare with climate change: the main reason high taxes on gasoline haven't been imposed in the US like they have in Europe is that US lawmakers have been more afraid of getting voted out of office by voters angry that it cost them more to drive their big pickup trucks and SUVs than their counterparts in Europe have been. "Less democracy" in the US would've resulted in a more robust response to climate change! I don't see anything about the AI situation that makes me expect a different outcome there: i.e., I expect "robust democracy" to interfere with a robust response to the AI juggernaut, especially in a few years when it becomes clear to most people just how useful AI-based products and services can be.

Another related argument is that elites don't want themselves and their children to be killed by the AI juggernaut any more than the masses do: its not like castles (1000 years ago) or taxes on luxury goods where the interests of the elites are fundamentally opposed to the interests of the masses. Elites (being smarter) are easier to explain the danger to than the masses are, so the more control we can give the elites relative to the masses, the better our chances of surviving the AI juggernaut, it seems to me. But IMHO we've strayed too far from the original topic of Harris vs Trump, and one sign that we've strayed too far is that IMHO Harris's winning would strengthen US elites a little more than a Trump win would.

Along with direct harms, a single war relevant to US interests could absorb much of the nation’s political attention and vast material resources for months or years. This is particularly dangerous during times as technologically critical as ours

I agree that a war would absorb much of the nation's political attention, but I believe that that effect would be more than cancelled out by how much easier it would become to pass new laws or new regulations. To give an example, the railroads were in 1914 as central to the US economy as the Internet is today -- or close to it -- and Washington nationalized the railroads during WWI, something that simply would not have been politically possible during peacetime.

You write that a war would consume "vast material resources for months or years". Please explain what good material resources do, e.g., in the hands of governments, to help stop or render safe the AI juggernaut? It seems to me that if we could somehow reduce worldwide material-resource availability by a factor of 5 or even 10, our chances of surviving AI get much better: resources would need to be focused on maintaining the basic infrastructure keeping people safe and alive (e.g., mechanized farming, police forces) with the result that there wouldn't be any resources left over to do huge AI training runs or to keep on fabbing ever more efficient GPUs.

I hope I am not being perceived as an intrinsically authoritarian person who has seized on the danger from AI as an opportunity to advocate for his favored policy of authoritarianism. As soon as the danger from AI is passed, I will go right back to being what in the US is called a moderate libertarian. But I can't help but notice that we would be a lot more likely to survive the AI juggernaut if all of the world's governments were as authoritarian as for example the Kremlin is. That's just the logic of the situation. AI is a truly revolutionary technology. Americans (and Western Europeans to a slightly lesser extent) are comfortable with revolutionary changes; Russia and China much less so. In fact, there is a good chance that as soon as Moscow and Beijing are assured that they will have "enough access to AI" to create a truly effective system of surveilling their own populations, they'll lose interest in AI as long as they don't think they need to continue to invest in it in order to stay competitive militarily and economically with the West. AI is capable of transforming society in rapid, powerful, revolutionary ways -- which means that all other things being equal, Beijing (who main goal is to avoid anything that might be described as a revolution) and Moscow will tend to want to supress it as much as practical.

The kind of control and power Moscow and Beijing (and Tehran) have over their respective populations is highly useful for stopping those populations from contributing to the AI juggernaut. In contrast, American democracy and the American commitment to liberty makes the US relatively bad at using political power stopping some project or activity being done by its population. (And the US government was specifically designed by the Founding Fathers to make it a lot of hard work to impose any curb or control on the freedom of the American people). America's freedom, particularly economic and intellectual freedom, in contrast, is highly helpful to the Enemy, namely, those working to make AI more powerful. If only more of the world's countries were like Russia, China and Iran!

I used to be a huge admirer of the US Founding Fathers. Now that I know how dangerous the AI juggernaut is, I wish that Thomas Jefferson had choked on a chicken bone and died before he had the chance to exert any influence on the form of any government! (In the unlikely event that the danger from AI is successfully circumnavigated in my lifetime, I will probably go right back to being an admirer of Thomas Jefferson.) It seemed like a great idea at the time, but now that we know how dangerous AI is, we can see in retrospect that it was a bad idea and that the architects of the governmental systems of Russia, China and Iran were in a very real sense "more correct": those choices of governmental architectures make it easier for humanity to survive the AI gotcha (which was completely hidden from any possibility of human perception at the time those governmental architectural decisions were made, but still, right is right).

I feel that the people who recognize the AI juggernaut for the potent danger that it is are compartmentalizing their awareness of the danger in a regrettable way. Maybe a little exercise would be helpful. Do you admire the inventors of the transistor? Still? I used to, but no longer do. If William Shockley had slipped on a banana peel and hit his head on something sharp and died before he had a chance to assist in the invention of the transistor, that would have been a good thing, I now believe, because the invention of the transistor would have been delayed -- by an expected 5 years in my estimation -- giving humanity more time to become collectively wiser before it must confront the great danger of AI. Of course, we cannot hold Shockley morally responsible because there is no way he could have known about the AI danger. But still, if your awareness of the potency of the danger from AI doesn't cause you to radically re-evaluate the goodness or badness of the invention of the transistor, then you're showing a regrettable lapse in rationality IMHO. Ditto most advances in computing. The Soviets distrusted information technology. The Soviets were right -- probably for the wrong reason, but right is still right, and no one who recognizes AI for the potent danger it is should continue to use the Soviet distrust of info tech as a point against them.

(This comment is addressed to those readers who consider AI to be so dangerous as to make AI risk the primary consideration in this conversation. I say that to account for the possbility that the OP cares mainly about more mundane political concerns and brought up AI safety because he (wrongly IMHO) believes it will help him make his argument.)

Comment by RHollerith (rhollerith_dot_com) on My simple AGI investment & insurance strategy · 2024-09-20T17:33:39.711Z · LW · GW

Our situation is analogous to someone who has been diagnosed with cancer and told he has a low probability of survival, but in consolation, he has identified a nifty investment opportunity.

Comment by RHollerith (rhollerith_dot_com) on Which LessWrong/Alignment topics would you like to be tutored in? [Poll] · 2024-09-19T04:30:35.807Z · LW · GW

Applying decision theory to scenarios involving mutually untrusting agents.

Comment by RHollerith (rhollerith_dot_com) on Tapatakt's Shortform · 2024-09-18T21:01:58.575Z · LW · GW

Even people who know English pretty well might prefer to consume information in their native language, particularly when they aren't in a task-oriented frame of mind and do not consider themselves to be engaged in work, which I'm guessing is when people are most receptive to learning more about AI safety.

Comment by RHollerith (rhollerith_dot_com) on Is this a Pivotal Weak Act? Creating bacteria that decompose metal · 2024-09-18T14:51:13.875Z · LW · GW

buy up most of the GPUs or the resources used to produce them

That would backfire IMHO. Specifically, GPUs would become more expensive, but that would last only as long as it takes for the GPU producers to ramp up production (which is very unlikely to take more than 5 years) after which GPU prices would go lower than they would've gone if we hadn't started buying them up (because of better economies of scale).

GPUs and the products and services needs to produce GPUs are not like the commodity silver where if you buy up most of the silver, the economy probably cannot respond promptly by producing a lot more silver. If you could make leading-edge fabs blow up in contrast that would make GPUs more expensive permanently (by reducing investment in fabs) or at least it would if you could convince investors that leading-edge fabs are likely to continue to blow up.

Comment by RHollerith (rhollerith_dot_com) on When can I be numerate? · 2024-09-12T15:27:35.694Z · LW · GW

We innately know a lot about "the physical environment", but not a lot about math (because the math abilities of your ancestors have not been under selection pressure for any evolutionary-significant length of time). Although it is true that neuroscience has found brain circuitry specialized for understanding non-negative integers, it remains the fact that much more of an educated person's knowledge about math must be acquired through deliberate practice, which is slow, than his or her knowledge about stacking boxes or replacing an alternator in a car.

In summary, there is no analog in math of your experience of having your knowledge about the physical environment unlock just because you chose to pay more attention to the details of your the physical environment.

We all have an innate drive to understand (i.e., curiosity) and also an innate drive to try to win arguments. Unlocking those 2 motivations is the closest analog I can think of to of your experience in the factory of getting a skill or innate ability to unlock, but before you can wield math towards profitable ends, you must spend many hundreds of hours reading about math and doing math. The 2 motivations I just described merely make it possible for you to put in those many hundreds of hours with less use of willpower or discipline.

Comment by RHollerith (rhollerith_dot_com) on Is this a Pivotal Weak Act? Creating bacteria that decompose metal · 2024-09-11T21:32:49.545Z · LW · GW

Because of the difficulty of predicting a "safe upper bound" on size of rock below which the risk of human extinction is within acceptable limits, I prefer the idea of destroying all the leading-edge fabs in the world or reducing the supply of electricity worldwide to low enough levels that the AI labs cannot compete for electricity with municipalities who need some electricity just to maintain social order and cannot compete with the basic infrastructure required just to keep most people alive. If either of those 2 outcomes weren't hard enough, we would have to maintain such an improved state of affairs (i.e., no leading-edge fab capability or severely degraded electricity generation capability) long enough (i.e., probably at least a century in my estimation) for there to come into being some other, less drastic way of protecting against reckless AI development.

Neither OP's metal-eating bacteria, the large rock from space nor either of the 2 interventions I just described is feasible enough to be worth thinking about much, IMO (and again the large rock from space carries much higher extinction risk than the other 3).

Comment by RHollerith (rhollerith_dot_com) on Is this a Pivotal Weak Act? Creating bacteria that decompose metal · 2024-09-11T21:07:01.707Z · LW · GW
Comment by RHollerith (rhollerith_dot_com) on Has Anyone Here Consciously Changed Their Passions? · 2024-09-09T18:53:17.969Z · LW · GW

My answer to your first question was, I don't know, but in the interest of comprehensiveness I mentioned a minor exception, which you asked about, so here is more info on the minor exception.

what does encouraging/discouraging sources of intrinsic motivation look like for you?

It looks like making a habit of patiently watching for the desired change. My software environment is organized enough that I can usually arrange for my future self to re-read a written reminder. So, (years ago) I wrote a reminder to watch for any instance where I think my behavior and my decisions are being motivated by interpersonal altruism or I experience pleasure or satisfaction from having achieve an altruistic interpersonal outcome. Note that this did not result in incontrovertible evidence of a significant increase in frequency of altruistic behavior. But I certainly stopped my addiction to the flow motivator (over the course of many years, except I relapse when I'm under stress, but these years it takes a lot of stress) and am pretty sure that the patient watching strategy helped a lot there. (And "just watching" helped me make other kinds of internal mental changes.)

My mind seems to works such that if the only conscious effort I make to effect some internal change is to get into the habit of watching or checking to see if the change has already occurred, my subconscious sometimes seems to figure out a way to effect the change if I watch long enough (months or years).

There are much faster and more potent ways to increase motivation and drive for most people: avoiding all exposure to light between 23:00 and 05:00 every night; getting as much very bright light as possible during the first 3 hours of wakefulness; making sure to get enough tyrosine (a dopamine precursor); deliberate cold exposure; avoiding spending too much of the day in perfectly-safe pleasurable activities; doing enough things you find aversive or outright painful; doing enough risky things. But you didn't ask about that, I don't think: "passion" almost always refers to intrinsic motivation (i.e., basically something that feels good or that a person wants to do for it own sake rather than as part of a plan to achieve some other outcome), whereas the motivation for most of what I (and probably you) do is extrinsic. E.g., I take the time to cook a healthy meal not because I enjoy cooking but rather because I anticipate that eating well will pay dividends mostly in future years in helping me achieve outcomes that I haven't even decided to pursue yet. I took stuff like that to be outside the scope of your question.

Comment by RHollerith (rhollerith_dot_com) on An AI Race With China Can Be Better Than Not Racing · 2024-09-09T17:48:00.701Z · LW · GW

value(SHUT IT ALL DOWN) > 0.2 > value(MAGIC) > 0 = value(US first, no race)=value(US first, race)=value(PRC first, no race)=value(PRC first, race)=value(PRC first, race)=value(extinction)

Yes, that is essentially my preference ordering / assignments, which remains the case even if the 0.2 is replaced with 0.05 -- in case anyone is wondering whether there are real human beings outside MIRI who are that pessimistic about the AI project.

Comment by RHollerith (rhollerith_dot_com) on Has Anyone Here Consciously Changed Their Passions? · 2024-09-09T16:09:24.631Z · LW · GW

No, I haven't made a successful effort to get myself to be passionate about doing X where I was not passionate about X before -- with one minor exception.

I try to factor my motivations into things that seem like complex function adaptations. For example, my motivation for coding projects comes from a strong desire for my being in the flow state and for being dissociated (which might or might not be the same complex function adaptation), but also the desire to "get ahead" or advance my status or position in society and occasionally from curiosity. Curiosity I consider a mildly good intrinsic motivation for me to have (but it does cause me to waste a lot of time doing web searches or listening to long Youtube videos). I wish to strengthen my desire to get ahead; my desire to be in the flow state or to be dissociated I strong try to discourage (because I need to learn ways to keep my nervous system calm other than the flow state and because I was addicted to flow for many many years). I try to encourage what small amount of intrinsic motivation I have to help other people.

So the "one minor exception" I refer to above is that over many years, my efforts to encourage some intrinsic motivators while discouraging others has had effects, the strongest of which has been that these years I mostly succeed in my policy of staying out of deep flow states unless I'm under severe stress. (To reach deep flow, I need to be fairly active, e.g., coding or doing math or playing chess online. My guess is that I'm also entering a flow state when I'm watching an absorbing movie, but if so, passive activities like movie watching never activate deep flow. I should probably mention that these years I never enter a flow state when I am coding AFAICT: my knowing that I'm a more effective at coding when I stay out of the flow state makes it a lot easier for me to stay out of the flow state while coding.) The way I understand motivation to work, by making that one source of intrinsic motivation less influential on my behavior, I am automatically making the other intrinsic motivators more influential. (And one of the major disadvantages of being addicted to something is that it is almost impossible to be intrinsically motivated by anything other than the addictive activity.)

Comment by RHollerith (rhollerith_dot_com) on Is it Legal to Maintain Turing Tests using Data Poisoning, and would it work? · 2024-09-07T14:12:37.609Z · LW · GW

There is a trend toward simplifying model architectures. For example, AlphaGo Zero is simpler than AlphaGo in that it was created without using data from human games. AlphaZero in turn was simpler than AlphaGo Zero (in some way that I cannot recall right now).

Have you tried to find out whether any of the next-generation LLMs (or "transformer-based models") being trained now even bothers to split text into tokens?

Comment by RHollerith (rhollerith_dot_com) on Executable philosophy as a failed totalizing meta-worldview · 2024-09-06T04:06:35.982Z · LW · GW

OK.

Decision theorists holds that for every sequence of observations and every utility function (set of goals), there is exactly one best move or optimal action (namely, the one that maximizes expected utility conditional on the observations). Does trying to use decision theory as much as possible in one's life tend to push one into having a totalizing worldview?

Comment by RHollerith (rhollerith_dot_com) on Executable philosophy as a failed totalizing meta-worldview · 2024-09-05T17:28:21.856Z · LW · GW

It is not clear to me what makes a worldview totalizing. Would Newtonian mechanics be a totalizing worldview? If not, is it a worldview? Is any worldview in physics after Newton non-totalizing? (My guess is no.) Is Greek geometry a la Euclid?

Comment by RHollerith (rhollerith_dot_com) on Is there any rigorous work on using anthropic uncertainty to prevent situational awareness / deception? · 2024-09-04T16:12:21.945Z · LW · GW

People have proposed putting an AI into a simulated environment such that when it thinks it is acting on reality, in actuality it is not.

Is that what you mean by "plausibly need to know exactly where they are in space-time"? If not, what do you mean?

Comment by RHollerith (rhollerith_dot_com) on Reducing global AI competition through the Commerce Control List and Immigration reform: a dual-pronged approach · 2024-09-04T15:18:58.043Z · LW · GW

One disadvantage of your second proposal is that it rewards people who chose to prepare themselves to do AI research or AI development, which in turn incentivizes young people who have not yet chosen a career path to choose AI. (It is impractical to hide from the young people the news that AI researchers have been rewarded with US citizenship.) I would prefer a policy that punishes people for choosing to prepare themselves to be AI researchers, e.g., a policy that puts any Chinese national who studies AI at a university or gets a job as an AI researcher or apprentice AI researcher on a sanctions list maintained by the US government similar to the list of Russian entities who participated in the invasion of Ukraine!

Your first proposal is fine with me, particularly if China retaliates by imposing strict controls on the transfer of AI knowledge to the US: I'm for almost any restriction on the flow of knowledge useful for advancing AI between any 2 populations.

Comment by RHollerith (rhollerith_dot_com) on Jeremy Gillen's Shortform · 2024-09-03T18:49:54.092Z · LW · GW

By "non-world-destroying", I assume you mean, "non-humanity ending".

Well, yeah, if there were a way to keep AI models to roughly human capabilities that would be great because they would be unlikely to end humanity and because we could use them to do useful work with less expense (particularly, less energy expense and less CO2 emissions) than the expense of employing people.

But do you know of a safe way of making sure that, e.g., OpenAI's next major training run will result in a model that is at most roughly human-level in every capability that can be used to end humanity or to put and to keep humanity in a situation that humanity would not want? I sure don't -- even if OpenAI were completely honest and cooperative with us.

The qualifier "safe" is present in the above paragraph / sentence because giving the model access to the internet (or to gullible people or to a compute farm where it can run any program it wants) then seeing what happens is only safe if we assume the thing to be proved, namely, that the model is not capable enough to impose its will on humanity.

But yeah, it is a source of hope (which I didn't mention when I wrote, "what hope I have . . . comes mostly from the hope that someone will figure out how to make an ASI that genuinely wants the same things that we want") that someone will develop a method to keep AI capabilities to roughly human level and all labs actually use the method and focus on making the human-level AIs more efficient in resource consumption even during a great-powers war or an arms race between great powers.

I'd be more hopeful if I had ever seen a paper or a blog post by a researcher trying to devise such a method.

For completeness's sake, let's also point out that we could ban large training runs now worldwide, then the labs could concentrate on running the models they have now more efficiently and that would be safe (not completely safe, but much much safer than any future timeline we can realistically hope for) and would allow us to derive some of the benefits of the technology.

Comment by RHollerith (rhollerith_dot_com) on Jeremy Gillen's Shortform · 2024-09-03T16:26:47.358Z · LW · GW

Such an entity would be useless to us IMHO.

Comment by RHollerith (rhollerith_dot_com) on Jeremy Gillen's Shortform · 2024-09-03T15:17:41.386Z · LW · GW

The whole approach is pretty hopeless IMHO: I mean the approach of "well, the AI will be wicked smart, but we'll just make it so that it doesn't want anything particularly badly or so that what it wants tomorrow will be different from what it wants today".

It seems fairly certain to me that having a superhuman ability to do things that humans want to be done entails applying strong optimization pressure onto reality -- pressure that persists as long as the AI is able to make it persist -- forever, ideally, from the point of view of the AI. The two are not separate things like you hope they are. Either the AI is wicked good at steering reality towards a goal or not. If it is wicked good, then either its goal is compatible with continued human survival or not, and if not, we are all dead. If it is not wicked good at steering reality, then no one is going to be able to figure out how to use it to align an AI such that it stays aligned once it is much smarter than us.

I subscribe to MIRI's current position that most of the hope for continued human survival comes from the (slim) hope that no one builds super-humanly smart AI until there are AI researchers that are significantly smarter and wiser than the current generation of AI designers (which will probably take centuries unless it proves much easier to employ technology to improve human cognition than most people think it is).

But what hope I have for alignment research done by currently-living people comes mostly from the hope that someone will figure out how to make an ASI that genuinely wants the same things that we want -- like Eliezer has been saying since 2006 or so.

Comment by RHollerith (rhollerith_dot_com) on What are the effective utilitarian pros and cons of having children (in rich countries)? · 2024-09-02T14:15:20.342Z · LW · GW

Certainly a strong argument against having a child is it makes it easier for society to deal with climate change. Halving the global population has the same effect on climate as doubling the size of the Earth's atmosphere, allowing it to absorb twice as much CO2 for the same effect on climate. But if people who care enough about society to respond to an argument like this actually do respond to this argument, then the next generation will not include their children, so it will be more selfish than the current generation.

Some people believe that the main impediment to drastically reducing or stopping society's use of fossil fuels is stubborn refusal to see the light by consumers, voters and people in power. Obviously if that is the actual situation we are in, then reducing the human population will not be needed to deal with climate change. But there's a good chance that that is not the situation we are in and that the only way we can stop burning fossil fuels is to suffer a severe drop in the global standard of living if we maintain current global population levels.

Comment by RHollerith (rhollerith_dot_com) on rhollerith_dot_com's Shortform · 2024-09-01T17:34:43.705Z · LW · GW

The different agents at those different timepoints surely have different utility functions, don't they?

IMHO, no.

Comment by RHollerith (rhollerith_dot_com) on rhollerith_dot_com's Shortform · 2024-09-01T02:42:44.811Z · LW · GW

My assertion is that all utility functions (i.e., all functions that satisfy the 4 VNM axioms plus perhaps some additional postulates most of us would agree on) are static (do not change over time).

I should try to prove that, but I've been telling myself I should for months now, but haven't mustered the energy, so am posting the assertion now without proof because an weak argument posted now is better then a perfect argument that might never be posted.

I've never been tempted to distinguish between "the outside-of-time all-timepoints-included utility function" and other utility functions such as the utility function referred to by the definition of expected utility (EU (action) = sum over all outcomes of (U(outcome) times p(outcome | action))).

Comment by RHollerith (rhollerith_dot_com) on Jeremy Gillen's Shortform · 2024-09-01T02:08:26.972Z · LW · GW

If we all die because an AI put super-human amounts of optimization pressure into some goal incompatible with human survival (i.e., almost any goal if the optimization pressure is high enough) it does not matter whether the AI would have had some other goal in some other context.

Comment by RHollerith (rhollerith_dot_com) on rhollerith_dot_com's Shortform · 2024-08-30T05:49:45.849Z · LW · GW

I think you are responding to my "is an abstraction that does not capture the richness" which on reflection I'm not attached to and would not include if I were to rewrite my comment.

Your "seen from outside time" suggests that maybe you agree with my "it never makes sense to model a utility function as changing over time". In contrast, some on LW hold that a utility function needs to change over time (for human potential to remain "open" or some such). I think that that doesn't work; i.e., if it changes over time, I think that it is incompatible with the four axioms of VNM-rationality, so these people should switch to some other term than "utility function" for the same reason that someone using the term "point", "line" or "plane" in a way that is inconsistent with the axioms of geometry should find some other term. (I have no doubt that your "much grander thing" is compatible with the four axioms.)

(In the context of RL, I'm used to hearing it referred to as a reward function.)

Comment by RHollerith (rhollerith_dot_com) on Eli's shortform feed · 2024-08-29T21:53:04.623Z · LW · GW

I’m thinking here mainly of a prototypical case of an isolated farmer family (like the early farming families of the greek peninsula, not absorbed into a polis), being accosted by some roving bandits

The assertion IIUC is not that it never makes sense for anyone to give in to a threat -- that would clearly be an untrue assertion -- but rather that it is possible for a society to reach a level of internal coordination where it starts to make sense to adopt a categorical policy of never giving in to a threat. That would mean for example that any society member that wants to live in dath ilan's equivalent of an isolated farm would probably need to formally and publicly relinquish their citizenship to maintain dath ilan's reputation for never giving in to a threat. Or dath ilan would make it very clear that they must not give in to any threats, and if they do and dath ilan finds out, then dath ilan will be the one that slaughters the whole family. The latter policy is a lot like how men's prisons work at least in the US whereby the inmates are organized into groups (usually based on race or gang affiliation) and if anyone even hints (where others can hear) that you might give in to sexual extortion, you need to respond with violence because if you don't, your own group (the main purpose of which is mutual protection from the members of the other groups) will beat you up.

That got a little grim. Should I add a trigger warning? Should I hide the grim parts behind a spoiler tag thingie?

Comment by RHollerith (rhollerith_dot_com) on Liability regimes for AI · 2024-08-28T23:30:39.621Z · LW · GW

So far, lawmakers at least in the US have refrained from passing laws that impose liability on software owners and software distributors: they have left the question up to the contract (e.g., the license) between the software owner and the software user. But there is nothing preventing them from passing laws on software (or AI models) that trump contract provisions -- something they routinely do in other parts of the economy: in California, for example, the terms of the contract between tenant and landlord, i.e., the lease, hardly matter at all because there are so many state laws that override whatever is in the lease.

Licenses are considered contracts at least in the English-speaking countries: the act of downloading the software or using the software is considered acceptance of the contract. But, like I said, there are tons of laws that override contracts.

So, the fact that the labs have the option of releasing models under open-source-like licenses has very little bearing on the feasibility, effectiveness or desirability of a future liability regime for AI as discussed in the OP -- as long as lawmakers cooperate in the creation of the regime by passing new laws.

Comment by RHollerith (rhollerith_dot_com) on Liability regimes for AI · 2024-08-28T23:08:49.212Z · LW · GW

There's a lot more to the Open Source Definition than, "well, you can read the source". Most of the licenses approved by the Open Source Initiative have also been approved by the Free Software Foundation.

Comment by RHollerith (rhollerith_dot_com) on Liability regimes for AI · 2024-08-28T22:46:27.636Z · LW · GW

There are licenses that only allow you to use the code in non-commercial projects

But they are emphatically not considered open-source licenses by the Open Source Initiative and are not considered Free-Software licenses by the Free Software Foundation, positions that have persisted uninterrupted since the 1990s.

Comment by RHollerith (rhollerith_dot_com) on "Deception Genre" What Books are like Project Lawful? · 2024-08-28T21:26:21.899Z · LW · GW

My question is, Can I download an offline copy of it -- either text or spoken audio?

The audio consists of 195 episodes, each of which can be individually downloaded, but can I get it as a single audio file (of duration 150 hours or so)?

Comment by RHollerith (rhollerith_dot_com) on "Deception Genre" What Books are like Project Lawful? · 2024-08-28T20:56:04.540Z · LW · GW

This almost certainly will not satisfy the desire that motivated your question, but just for the sake of thoroughness, we might notice that the 2014 TV show Ascension is about 70 people who believe they are on an inter-generational space ship going to Alpha Centauri, but they are really on Earth. The writers are mainly interested in exploring class differences, 1960s American society and a murder, and there's very little treatment of "conspiracists frantically trying to keep the deception running".

https://en.wikipedia.org/wiki/Ascension_(miniseries)

I watched it, and cannot even recall the motivation for the deception.

Comment by RHollerith (rhollerith_dot_com) on Shortform · 2024-08-28T00:13:33.225Z · LW · GW

There are. The prestigious universities are examples. (lc and I are Americans.)

Comment by RHollerith (rhollerith_dot_com) on On Nothing · 2024-08-27T23:17:17.420Z · LW · GW

I would've downvoted OP even if it were a shortform. Would that transgress a community norm I'm unaware of?

Also, I was unaware of any norm against small questions and doubt the value of having such a norm.

Comment by RHollerith (rhollerith_dot_com) on rhollerith_dot_com's Shortform · 2024-08-24T01:07:21.655Z · LW · GW

The utility function is an abstraction that does not capture the richness of the behavior of agents in the real world, but the better an agent is at rationality, the more accurate (and comprehensive) the abstraction of the utility function becomes IMHO at describing the agent. I suspect that it never makes sense to model a utility function as changing over time.

Or maybe it makes sense only as a mental shortcut to be used only when we do not have time to make a proper analysis. People of course discover new wants and new preferences as they go through life, but this can be taken into account by saying (or noticing) that the person does not know what their (unchanging) utility function is in every detail, and every now and then he or she learn a previously-unknown detail (or fact) about their utility function.

Comment by RHollerith (rhollerith_dot_com) on Why you should be using a retinoid · 2024-08-23T19:52:03.932Z · LW · GW

Note that this is a reply to myself.

Since the comment I am replying to is being upvoted, I will put a little more effort into explaining. "Getting 20 to 30 minutes of strong sun on my bare arms and legs and eyes every other day" is from neuroscientist Andrew Huberman. This effect of UVB light on the sex hormones seems to have been recently discovered; more information here.

Comment by RHollerith (rhollerith_dot_com) on AI #78: Some Welcome Calm · 2024-08-23T18:51:00.428Z · LW · GW

Should a California voter contact Governor Newsom's office asking him to support SB 1047?

Comment by RHollerith (rhollerith_dot_com) on Where should I look for information on gut health? · 2024-08-23T06:43:57.971Z · LW · GW

The document I linked to contains advice that does not entail buying any products.

Comment by RHollerith (rhollerith_dot_com) on Nathan Young's Shortform · 2024-08-22T15:55:05.138Z · LW · GW

I only ever use words to express a probability when I don't want to take the time to figure out a number. I would write your example as, "Kamela will win the election with p = 54%, win Virginia with p =83% and win North Carolina with p = 43%."

Comment by RHollerith (rhollerith_dot_com) on Where should I look for information on gut health? · 2024-08-21T18:37:26.714Z · LW · GW

Yes, she is still taking products from the company and following advice in the company's publications (e.g., eating jicama, probably other things) so it has been 6 or 7 years for her.

Note that she is in her early 80s, so . . .

Comment by RHollerith (rhollerith_dot_com) on How great is the utility of "saving" endangered languages? · 2024-08-21T02:56:36.110Z · LW · GW

Being bilingual has been linked with a lot of benefits

Being bilingual is AFAIK a strong signal of cognitive competence: given a choice between 2 applicants for a cognitively-demanding job, one bilingual and one not bilingual, I would heavy favor the bilingual one. But that does not mean that investing effort in making a person bilingual increases the person's cognitive competence to any significant degree.

One thing we don't need studies or complex arguments for is the fact that it takes a lot of study and practice to learn a second language -- time and mental energy that can be used to learn other things. Our society has accumulated an impressive store of potent knowledge, knowledge that takes a long time for people to acquire, but which clearly improves their lives and their ability to contribute to society. I'm very skeptical that the benefits of spending an hour learning a second language outweigh the benefits of spending an hour learning, e.g., history, geography, chemistry, physics, statistics, computer programming, practical human physiology, cooking, sewing, woodworking, accounting or the basics of public speaking or performing in front of an audience.

I'm anticipating that you will reply here that there is more to culture than knowledge that has obvious practical benefits. And my reply to that is that I don't see why an hour spent on second-language learning would outweigh the benefits of an hour spent watching Buffy the Vampire Slayer or Desperate Housewives or browsing https://tvtropes.org/. Those 3 things are products of the dominant culture, and I suspect that most of the effort to save endangered languages stems from a perceived need to fight the dominant culture in any way possible (but I don't perceive any need to fight against the dominant culture).

Comment by RHollerith (rhollerith_dot_com) on Where should I look for information on gut health? · 2024-08-21T01:21:35.076Z · LW · GW

I also read a concerning article by a professor that seemed to indicate that bacteria from probiotic foods doesn’t stick around in the intestine

Beneficial practices that have been around a long time accumulate simplistic explanations for why they work. I wouldn't lose interest in fermented foods just because this particular simplistic explanation has been refuted.

Comment by RHollerith (rhollerith_dot_com) on Where should I look for information on gut health? · 2024-08-21T00:55:13.005Z · LW · GW

Usually I do not recommend publications by outfits that sell products (e.g., Life Extension magazine is awful IMHO -- very mercenary) but I make an exception for publications by Microbiome Labs, like this next one:

https://microbiomelabs.co.uk/wp-content/uploads/2023/12/Total-Gut-Restoration-Protocol-Practitioner-Copy.pdf

One of my friends has been helped significantly by Microbiome Labs's protocols.