Posts
Comments
Picture a dynamic logarithmic scale of discomfort stacking with a ‘hard cap’ where every new instance contributes less and less to the total to the point of flatlining on a graph.
Reality is structured such that there tend to be an endless number of (typically very complicated) ways of increasing a probability by a tiny amount. The problem with putting a hard cap on the desirability of some need or want is that the agent will completely disregard that need or want to affect the probability of a need or want that is not capped (e.g., the need to avoid people's being tortured) even if that effect is extremely small.
The problem with routinely skipping dinner is getting enough protein. No matter how much protein you eat in one sitting, your body can use at most 40 or 45 grams. (The rest is converted to fuel -- glucose, fructose or fatty acids, I don't know which.) On a low protein diet, it is difficult to maintain anything near an optimal amount of muscle mass (even if you train regularly with weights) -- and the older you get, the harder it gets. One thing muscle mass is good for is smoothing out spikes in blood glucose: the muscles remove glucose from the blood and store it. Muscle also protects you from injury. Also men report that people (men and women) seem to like them better when they have more muscles (within reason).
But yeah, if you don't have to worry about maintaining muscle mass, routinely skipping meals ("time-restricted eating") is a very easy way to maintain a healthy BMI.
Just because it was not among the organizing principles of any of the literate societies before Jesus does not mean it is not part of the human mental architecture.
There is very little hope here IMO. The basic problem is the fact that people have a false confidence in measures to render a powerful AI safe (or in explanations as to why the AI will turn out safe even if no one intervenes to make it safe). Although the warning shot might convince some people to switch from one source of false hope to a different source, it will not materially increase the number of people strongly committed to stopping AI research, all of which have somehow come to doubt all of the many dozens of schemes published so far for rendered powerful AI safe (and the many explanations for why the AI will turn out safe even if we don't have a good plan for ensuring its safety).
I wouldn't be surprised to learn that Sean Carroll already did that!
Is it impossible that someday someone will derive the Born rule from Schrodinger's equation (plus perhaps some of the "background assumptions" relied on by the MWI)?
Being uncertain of the implications of the hypothesis has no bearing on the Kolmogorv complexity of a hypothesis.
Fire temperature can be computed from the fire's color.
I'm tired of the worthless AI-generated art that writers here put in their posts and comments. Some might not be able to relate, but the way my brain works, I have to exert focus for a few seconds to suppress the effects of having seen the image before I can continue to engage with the writer's words. It is quite mentally effortful.
As a deep-learning novice, I found the post charming and informative.
The statement does not mention existential risk, but rather "the risk of extinction from AI".
Any computer program can be presented in the form of an equation. Specifically, you define a function named step such that step (s, input) = (s2, output) where s and s2 are "states", i.e., mathematical representations of the RAM, cache and registers.
To run the computer program, you apply step to some starting state, yielding (s2, output), then you apply step to s2, yielding (s3, output2), then apply step to s3, and so on for billions of repetitions.
Another reply to your question asserts that equations cannot handle non-determinism. Untrue. To handle it, all we need to do is add another argument to step, rand say, that describes the non-deterministic influences on the program. This is routinely done in formalisms for modelling causality, e.g., the structural equation models used in economics.
So, in summary, your question has some implicit assumptions that would need to be made explicit before I can answer.
I hope owencb won't let this prevent him from continuing to post on this topic.
One of the reasons I think demographic factors aren't as important over the long term as some say they are is that an estimated 50% of the population of Europe died (from the plague) from 1346 to 1353 and yet not long afterwards Europe started pulling ahead the rest of the world, with Gutenberg inventing movable type in 1450 and the Renaissance having spread throughout Europe by 1500. Admittedly the Renaissance's beginning (in Italy) predate the plague, but the point is that the loss of about 50% of the population did not prevent those beginnings from spreading to the rest of the Europe and did not prevent Europe from becoming the most influential region of the world (with the European discovery of the New World in 1492, with the European Magellan making the first circumnavigation of the globe between 1519 and 1522 and with the scientific and political advances, e.g., the empirical method and liberalism, of the European Renaissance having tremendous global influence to the present day).
And even if I were not able to cite the example I just cited, the track record of the class of experts (geographers, political scientists) who maintain that demographics is national destiny is that they are often wrong. Less Wrong does well in general, but could do better at resisting false information and information of unknown truth value that gets repeated over and over on by the internet.
My guess is that banning birth control is not in the Overton window in Russia.
You make a good point. I think however that these demographic factors aren't as important to a country's long-term fate as many recent commentators say it is.
Probably the only feasible way of fixing the demography is mass immigration.
Banning birth control would be another feasible way in my estimation for any government that can survive the resentment caused by the ban -- and the Kremlin probably could survive it.
The Russian regime is not that bad. It has for example caused fewer civilian deaths in Ukraine since 2014 than the Israeli regime has caused in a few months in Gaza -- and Ukraine has 74 times as many people as Gaza has.
If Texas left the United States and started accepting military aid from China, as an American, I would probably want my government to force Texas back into the United States even if that requires invading Texas and causing many casualties.
Also, even if the reputation of the Russian government remains poor, Westerners will continue to treat individual Russians okay because of the fairly strong ethic in the West and especially in the US under which every individual should be judged on their own merits, not by the group they belong to, particularly if they were born into the group. I cannot imagine for example your staying in Russia significantly reducing the probability of your winning a grant from a Western organization to work on AI safety or effective altruism with the one exception that the organization might be prohibited by sanctions imposed by Western governments from sending money into Russia.
On the other hand, Stockholm syndrome is a powerful cognitive bias which might bias you against emigrating.
I drink it or more precisely mix it with my bowl of beans and veggies.
My $300 against your $30 that OpenAI's next round of funding raises less than $40 billion of investor money. In case OpenAI raises no investment money in the next 36 months, the bet expires. Any takers?
I make the same offer with "OpenAI" replaced with "any startup with Sam Altman as a founder".
I'm retracting this comment and these 2 offers. I still think it is very unlikely, but realized that I would be betting against people who are vastly more efficient than I am at using the internet to access timely information and might even be betting against people with private information from Altman or OpenAI.
Someone should publicly bet like I did as an act of altruism to discourage this unprofitable line of inquiry, but I'm not feeling that altruistic today.
Reading this makes me suspect that increasing the scope (range of applicability) of insurance or of “derivative” contracts (e.g., options and futures) is a more potent way to improve the world than promoting prediction markets.
It is good to see people thinking creatively, but a frontier model that becomes superhuman at physics and making plans that can survive determined human opposition is very dangerous even if it never learns how to read or understand any human language.
In other words, being able to interact verbally with humans is one avenue by which an AI can advance dangerous plans, but not the only avenue. (Breaking into computers would be another avenue where being able to communicate with humans might be helpful, but certainly not necessary.)
So, do you have any ideas on how to ensure that your Sumerian-schooled frontier model doesn't become superhuman at physics or breaking into computer?
OpenAI raised money recently at a valuation of $100 billion whereas the last time Commonwealth Fusion raised money it did so at a valuation of "$7.2—10.8b (Dealroom.co estimates Dec 2021)". Also, OpenAI is only one of dozens of well-funded organizations in the space.
Source of the latter fact: https://app.dealroom.co/companies/commonwealth_fusion_systems
If the supply of government bonds is reduced, most of the money that would've gone into gov bonds will go into other investments (because if you want to, e.g., save for retirement, the unavailability of gov bonds is not going to stop you from finding some way to save for retirement). Investment has a different effect on the economy than consumption (e.g., choosing to have a kid) does, and your policy proposal replaces consumption with (private) investment.
No it does not imply constancy or consistency over time because the 4 axioms do not stop us from adding to the utility function a real-valued argument that represent the moment in time that the definition refers to.
In other words, the 4 axioms do not constrain us to consider only utility functions over world states: utility functions over "histories" are also allowed, where a "history" is a sequence of world states evolving over time (or equivalently a function that takes a number representing an instant in time and returning a world state).
>Does this mean that a cheap "pseudo-capnometer" can be created which . . . ?
I doubt it, but don't know for sure because I don't know anything about the mechanisms by which people outgas the VOCs.
The $100 CO2 monitors do not measure CO2, they measure VOCs, which in typical home and office settings closely correlates with CO2 (because humans emit both at a relatively constant rate and humans are the main sources of both in typical home and office settings).
I typically see 35-45 mmHg of partial pressure carbon dioxide being cited as the good range
That's about 5% (since atmospheric pressure is about 760 mmHg) or 50,000 ppm. Being in a room with that high a concentration of co2 is immediately dangerous to life and health, which is a good illustration of the fact that this post (your post) is about co2 in exhaled air, which is distinct from co2 in inhaled air or ambient air, where for example 5,000 ppm of co2 "is the permissible exposure limit for daily workplace exposures" (source).
What does "the lr" mean in this context?
An LLM can be strongly super-human in its ability to predict the next token (that some distribution over humans with IQ < 100 would write) even if it was trained only on the written outputs of humans with IQ < 100.
More generally, the cognitive architecture of an LLM is very different from that of a person, and IMO we can use our knowledge of human behavior to reason about LLM behavior.
In a world without any global network of computers, human beings would probably be most potent resource available to an AI that had just acquired superhuman capabilities (beyond the resources needed just to keep the AI running). Spy agencies are skilled at getting human beings to act against their own interests and the interests of their fellow human beings; an AI with superhuman capabilities would probably be much better at it.
The bigger picture is that no one is going to create a very powerful AI and not try to put it to profitable use, and all the ways I can think of to put it to profitable use entail giving it access to resources it can use to take over.
My memory is not that good. I do recall that it is in the chapter "Other ways: alternatives to many-worlds".
My probability that quantum Bayesianism is onto something is .05. It went down a lot when I read Sean Carroll's book Something Deeply Hidden. .05 is about as extreme as my probabilities get for the parts of quantum physics that are not settled science since I'm not an expert.
You don't want to warn us that it is behind a paywall?
Suppose I start a dialog knowing I will never choose to publish it. Would the LW team welcome that or tend to consider it a waste of resources because nothing gets published?
How prepared is LW for an attack? Those who want AI research to proceed unimpeded have an incentive to sabotage those who want to slow it down or ban it and consequently have an incentive to DDoS LW.com or otherwise make the site hard to use. What kind of response could LW make against that?
Also, how likely is it that an adversary will manage to exploit security vulnerabilities to harvest all the PMs (private messages) stored on LW?
I have some idea about how much work it takes to maintain something like LW.com, so this random person would like to take this opportunity to thank you for running LW for the last many years.
I wish people would stop including images of text on LW. I know this practice is common on Twitter and probably other forums, but we aspire a higher standard here. My reasoning: (1) it is more tedious to compose a reply when one cannot use copying-pasting to choose exactly which extent of text to quote (2) the practice is a barrier to disabled people using assistive technologies and people reading on very narrow devices like smartphones.
I have yet to interact with a state-of-the-art model (that I know of), but I do know from browsing Hacker News that many are running LLaMA and other open-source models on their own hardware (typically Apple Silicon or desktops with powerful GPUs).
You don't say what kind of harms you consider worst. I consider extinction risk the worst harm, and here is my current personal strategy. I don't give OpenAI or similar companies any money: if I had to use use AI, I'd use an open-sourced model. (It would be nice if there were some company that offered a paid service that I could trust not to advance the state of the art, but I don't know of any though yes, I understand that some companies contribute more to extinction risk than others.) I expect that the paid services will eventually (and soon probably) get so good that it is futile to continue to bother with open-sourced models, and that to compete I will need eventually to give money to a company offering a paid service. I plan to try to put that day off as long as possible, which I consider useful, not futile: suppose it takes OpenAI 4 generations of services (where GPT-4-based services is generation 1) to become a very successful, very profitable company. each of those generations is equally important in expectation to OpenAI's eventual wild success. (An iMac was a vastly better computer than the first Macintosh, but every major product launch between the first Mac and the first iMac was roughly equally important to the eventual wild success of Apple.) Thus if I can hold off giving OpenAI any money till they're offering 4th-generation services, I will have withheld from OpenAI 75% of the "mojo" (revenue, evidence that I'm a user that they can in aggregate show to investors) I might have given them (before they become so successful that nothing I do could possibly have any effect), but enhanced my productivity almost as much as if I had used all 4 generations of OpenAI's service (because of how good the 4th generation will be).
If I use any AI services, open-source or not, I don't tell anyone about it (to prevent my contributing to the AI field's reputation for usefulness) except for people who take AI extinction risk seriously enough that they'd quickly vote to shut down the whole field if they could.
Like Eliezer says, this is not what a civilization that might survive AI research looks like, but what I just wrote is my current personal strategy for squeezing as much dignity as possible from the situation.
I don't think I've ever seen an endorsement of the flow state that came with non-flimsy evidence that it increases productivity or performance in any pursuit, and many endorsers take the mere fact that the state feels really good to be that evidence.
>you're in relentless, undisturbed pursuit
This suggest that you are confusing drive/motivation with the flow state. I have tons of personal experience of days spent in the flow state, but lacking motivation to do anything that would actually move my life forward.
You know how if you spend 5 days in a row mostly just eating and watching Youtube videos, it starts to become hard to motivate yourself to do anything? Well, the quick explanation of that effect is that watching the Youtube videos is too much pleasure for too long with the result that the anticipation of additional pleasure (from sources other than Youtube videos) no longer has its usual motivating effect. The flow state can serve as the source of the "excess" pleasure that saps your motivation: I know because I wasted years of my life that way!
Just to make sure we're referring to the same thing: a very salient feature of the flow state is that you lose track of time: suddenly you realize that 4 or 8 or 12 hours have gone by without your noticing. (Also, as soon as you enter the flow state, your level of mental tension, i.e., physiological arousal, decreases drastically--at least if you are chronically tense, but I don't lead with this feature because a lot of people can't even tell how tense they are.) In contrast, if you take some Modafinil or some mixed amphetamine salts or some Ritalin (and your brain is not adapted to any of those things) (not that I recommend any of those things unless you've tried many other ways to increase drive and motivation) you will tend to have a lot of drive and motivation at least for a few hours, but you probably won't lose track of time.
The US NatSec community doesn't know that the US (and Britain) are with probability = .99 at least 8 years ahead of China and Russia in AI?
It's hard to control how capable the AI turns out to be. Even the creators of GPT-4 were surprised, for example, that it would be able to score in the 90th percentile on the Bar Exam. (They expected that if they and other AI researchers were allowed to continue their work long enough that eventually one of their models would be able to do, but had no way of telling which model it would be.)
But more to the point: how does boxing have any bearing on this thread? If you want to talk about boxing, why do it in the comments to this particular paper? why do it as a reply to my previous comment?
Yudkowsky's suggestion is for preventing the creation of a dangerous AI by people. Once a superhumanly-capable AI has been created and has had a little time to improve its situation, it is probably too late even for a national government with nuclear weapons to stop it (because the AI will have hidden copies of itself all around the world or taken other measures to protect itself, measures that might astonish all of us).
The OP in contrast is exploring the hope that (before any dangerous AIs are created) a very particular kind of AI can be created that won't try to prevent people from shutting it down.
Can you explain why you think that "Microsoft has gained approximately $100B in market capitalization?" I see a big dip in stock price late Thursday, followed by a recovery to exactly the start price 2 hours later.
Agree in general, but there is an ecosystem of mostly-small colleges where teaching has higher priority, and most ambitious American students and their parents know about it. Note for example that Harvard, Yale, Princeton and Stanford do not appear in the following list of about 200 colleges:
https://www.usnews.com/best-colleges/rankings/national-liberal-arts-colleges
Someone writes anonymously, "I feel compelled as someone close to the situation to share additional context about Sam and company. . . ."
https://www.reddit.com/r/OpenAI/comments/17xoact/comment/k9p7mpv/