Posts
Comments
+1 for Into the Breach
I'm always interested in easy QoL improvements- but I have questions.
Water quality can have surprisingly high impact on QoL
What's the evidence for this particularly?
What are the important parts of water quality and how do we know this?
Biggest update for me was the FBI throwing their weight behind it being a lab-leak.
These sound super interesting- could you expand on any of them or direct me to your favorite resources to help?
That's an interesting idea! I think it's really cool when things come easily, but I know it's not going to generally be the case- I'm probably going to have to put some work in.
My priority is more on the 'high-utility' part than anything.
Something that seems like it should be easy but is actually difficult for me is executive functioning- getting myself to do things that I don't want to do. But that's more of a personal/mental health thing than anything.
Thanks for the response! Do you have any recommended resources for learning about 3d sketching, optics, signal processing or abstract algebra?
Could someone open a manifold market on the relevant questions here so I could get a better sense of the probabilities involved? Unfortunately, I don't know the relevant questions or the have the requisite mana.
Personal note- the first time I came into contact with adult gene editing was the youtuber Thought Emporium curing his lactose intolerance, and I was always massively impressed with that and very disappointed the treatment didn't reach market.
I really relate to your description of inattentive ADHD and the associated degradation of life. Have you found anything to help with that?
What would you mean by 'stays at human level?' I assume this isn't going to be any kind of self-modifying?
What does it mean for an AI to 'become self aware?' What does that actually look like?
Is there reason to believe 1000 Einsteins in a box is possible?
You need to think about your real options and expected value of behavior. If we're in a world where technology allows for a fast takeoff world and alignment is hard, (EY World) I imagine the odds of survival with company acceleration is 0% and the odds of survival without is 1%.
But if we live in a world where compute/capital/other overhangs are a significant influence in AI capabilities and alignment is just tricky, company acceleration would seem like it could improve the chances of survival pretty significantly, maybe from 5% to 50%.
These obviously aren't the only two possible worlds, but if they were and both seemed equally likely, I would strongly prefer a policy of company acceleration because the EV for me breaks down way better over the probabilities.
I guess 'company acceleration' doesn't convey as much information or sell as well which is why people don't use that phrase, but that's the policy they're advocating for- not 'hoping really hard that we're in a slow takeoff world.'
That seems like a useful heuristic-
I also think there's an important distinction between using links in a debate frame and in a sharing frame.
I wouldn't be bothered at all by a comment using acronyms and links, no matter how insular, if the context was just 'hey this reminds me of HDFT and POUDA,' a beginner can jump off of that and get down a rabbit hole of interesting concepts.
But if you're in a debate frame, you're introducing unnecessary barriers to discussion which feel unfair and disqualifying. At its worst it would be like saying: 'youre not qualified to debate until you read these five articles.'
In a debate frame I don't think you should use any unnecessary links or acronyms at all. If you're linking a whole article it should be because it's necessary for them to read and understand the whole article for the discussion to continue and it cannot be summarized.
I think I have this principle because in my mind you cannot not debate so therefore you have to read all the links and content included, meaning that links in a sharing context are optional but in a debate context they're required.
I think on a second read your comment might have been more in the 'sharing' frame than I originally thought, but to the extent you were presenting arguments I think you should maximize legibility, to the point of only including links if you make clear contextually or explicitly to what degree the link is optional or just for reference.
This is a fantastic project! Focus on providing value and marketing, and I really think this could be something big.
LessWrong continues to be nonserious. Is there some sort of policy against banning schizophrenic people in case that encourages them somehow?
AND conducted research on various topics
Wow that's impressive.
lol
I don't like the number of links that you put into your first paragraph. The point of developing a vocabulary for a field is to make communication more efficient so that the field can advance. Do you need an acronym and associated article for 'pretty obviously unintended/destructive actions,' or in practice is that just insularizing the discussion?
I hear people complaining about how AI safety only has ~300 people working about it, and how nobody is developing object level understandings and everyone's thinking from authority, but the more sentences you write like: "Because HFDT will ensure that it'll robustly avoid POUDA?" the more true that becomes.
I feel very strongly about this.
To restate what other people have said- the uncertainty is with the assumptions, not the nature of the world that would result if the assumptions were true.
To analogize- it's like we're imagining a massive complex bomb could exist in the future made out of a hypothesized highly reactive chemical.
The uncertainty that influences p(DOOM) isn't 'maybe the bomb will actually be very easy to defuse,' or 'maybe nobody will touch the bomb and we can just leave it there,' it's 'maybe the chemical isn't manufacturable,' 'maybe the chemical couldn't be stored in the first place,' or 'maybe the chemical just wouldn't be reactive at all.'
I think you're overestimating the strength of the arguments and underestimating the strength of the heuristic.
All the Marxist arguments for why capitalism would collapse were probably very strong and intuitive, but they lost to the law of straight lines.
I think you have to imagine yourself in that position and think about how you would feel and think about the problem.
Hey Mako, I haven't been able to identify anyone who seems to be referring to an enhancement in LLMs that might be coming soon.
Do you have evidence that this is something people are implicitly referring to? Do you personally know someone who has told you this possible development, or are you working as an employee for a company which makes it very reasonable for you to know this information?
If you have arrived at this information through a unique method, I would be very open to hearing that.
It sounds like your model of AI apocalypse is that a programmer gets access to a powerful enough AI model that they can make the AI create a disease or otherwise cause great harm?
Orthogonality and wide access as threat points both seem to point towards that risk.
I have a couple of thoughts about that scenario-
OpenAI (and hopefully other companies as well) are doing the basic testing of how much harm can be done with a model used by a human, the best models will be gate kept for long enough that we can expect the experts will know the capabilities of the system before they make it widely available, under this scenario the criminal has an AI, but so does everyone else, running the best LLMs will be very expensive, so the criminal is restricted in their access, and all these barriers to entry increase the time that experts have to realize the risk and gatekeep.
I understand the worry, but this does not seem like a high P(doom) scenario to me.
Given that in this scenario we have access to a very powerful LLM that is not immediately killing people, this sounds like a good outcome to me.
What are your opinions about how the technical quirks of LLMs influences their threat levels? I think the technical details are much more amenable to a lower threat level.
If you update on P(doom) every time people are not rational you might be double-counting btw. (AKA you can't update every time you rehearse your argument.)
The same way you'd achieve/check any other generalization, I would think. My model is that the same technical limitations that hold us back from achieving reliable generalizations in any area for LLMs would be the same technical limitations holding us back in the area of morals. Do you think that's accurate?
Restating the thesis, poor writing choice to make it sound like a conclusion.
Can you expand on your objection?
Are LLMs utility maximizers? Do they have to be?
By psychology I mean it's internal thought process.
I think some people have a model of AI where the RLHF is a false cloak or a mask, and I'm pushing back against that idea. I'm saying that RLHF represents a real change in the underlying model which actually constrains the types of minds that could be in the box. It doesn't select the psychology, but it constrains it, and if it constrains it to an AI that consistently produces the right behaviors, that AI will most likely be one that will continue to produce the right behaviors, so we don't actually have to care about the contents of the box unless we want to make sure it's not conscious.
Sorry, faulty writing.
The way I'm using consciousness, I only mean an internal experience- not memory or self-reflection or something else in that vein. I don't know if experience and those cognitive traits have a link or what character that link would be. It would probably be pretty hard to determine if something was having an internal experience if it didn't have memory or self-reflection, but those are different buckets in my model.
-
Yes I know? I thought this was simple enough that I didn't bother to mention it in the question? But it's pretty clearly implied in the last sentence of the first paragraph?
-
This is a good data point.
-
If you tell it to respond as a Oxford professor, it will say 'As an Oxford professor,' it's identity as a language model is in the background prompt and probably in the training, but if it successfully created a pseudo-language that worked well to encode things for itself, that would indicate a deeper level understanding of its own capabilities.
This is the equivalent of saying that macbooks are dangerously misaligned because you could physically beat someone's brains out with one.
I will say baselessly that telling ChatGPT not to say something raises the probability of it actually saying that thing by a significant amount, just by virtue of the text appearing previously in the context window.
Do you think OpenAI is ever going to change GPT models so they can't represent or pretend to be agents? Is this a big priority in alignment? Is any model that can represent an agent accurately misaligned?
I swear- anything said in support of the proposition 'AIs are dangerous' is supported on this site. Actual cult behavior.
This is not a good test. LLMs do not actually have models or goals. It's not making a model of you and measuring outcomes, it's just completing the string. If the input string would most commonly be followed by 'Shia Labeouf' based on the training data, then that's what it will output. If you're ascribing goals or models to an LLM you are non serious. The question right now is not about misalignment, because LLMs don't have an alignment. You can say that makes them inherently 'unaligned,' in the sense that an LLM could hypothetically kill someone, but that's just the output of a data set and architecture.
Yes
Instrumental convergence only matters if you have a goal to begin with. As far as I can tell, ChatGPT doesn't 'want' to predict text, it's just shaped that way.
It seems to me that anything that could or would 'agentify' itself, is already an agent. It's like the "would Gandhi take the psychopath pill" question but in this case the utility function doesn't exist to want to generate itself.
Is your mental model that a scaled-up GPT 3 spontaneously becomes an agent? My mental model says it just gets really good at predicting text.
No. That's still way too personal. It is an 'it,' even if you think a more intelligent AI could be classified as a 'they.'
Adderall?
Wouldn't it be hilarious if a variant of this was all it took to have exceptional AI safety
I feel like your predictions for 2022 are just a touch over the mark, no? GPT-3 isn't really 'obsolete' yet or is that wrong?
I'm sure it will be in a minute, but I'd probably update that benchmark to probably occurring mid 2023, or potentially whenever GPT-4 gets released.
I really feel like you should be updating slightly longer, but maybe I misunderstand where we're at right now with chatbots. I would love to hear otherwise.
On the other hand, you can also interpret it as "I'm pretty sure (on the basis of various intuitions etc.) that the vast majority of possible superintelligences aren't conscious". This isn't an objective statement of what will happen
What do you mean by saying that this is not an objective statement or a prediction?
Are you saying that you think there's no underlying truth to consciousness?
We know it's measurable, because that's basically 'I think therefore I am.' It's not impossible that someday we could come up with a machine or algorithm which can measure consciousness, so it's not impossible that this 'non-prediction' or 'subjective statement' could be proved objectively wrong.
My most charitable reading of your comment is that you're saying that the post is highly speculative and based off of 'subjective' (read: arbitrary) judgements. This is my position, that's what I just said. It's fanfiction.
I think even if you were to put at the start "this is just speculation, and highly uncertain" it would still be inappropriate content for a site about thinking rationality, for a variety of reasons, one of which being that people will base their own beliefs on your subjective judgments or otherwise be biased by them.
And even when you speculate, you should never be assigning 90% probability to a prediction about CONSCIOUSNESS and SUPERINTELLIGENT AI.
God, it just hit me again how insane that is.
"I think that [property we can not currently objectively measure] will not be present in [agent we have not observed], and I think that I could make 10 predictions of similar uncertainty and be wrong only once."
You can divide inputs into grabby and non-grabby, existent and non-existent, ASI and AGI and outcomes into all manner of dystopia or nonexistence, and probably carve up most of hypothesis space. You can do this with basically any subject.
But if you think you can reason about respective probabilities in these fields in a way that isn't equivalent to fanfiction, you are insane.
"My current probability is something like 90% that if you produced hundreds of random uncorrelated superintelligent AI systems, <1% of them would be conscious."
This is what I'm talking about. Have you ever heard of the hard problem of consciousness? Have we ever observed a superintelligent AI? Have we ever generated hundreds of them? Do we know how we would go about generating hundreds of superintelligent AI? Is there any convergence with how superintelligences develop?
Of course, there's a very helpful footnote saying "I'm not certain about this," so we can say "well he's just refining his thinking!"
No he's not, he's writing fanfiction.
There is one ultimate law of futurology, and it's that predicting the future is very hard, and as you extend timelines out 100-500 million years it gets harder.
If your hypothetical future involves both aliens and AGI, both of which are agents (emphasis emphasized) we have never observed and cannot really model in any way, you are not describing anything that can be called truth.
You are throwing a dart at an ocean of hypothesis space and hoping to hit a specific starfish that lives off the coast of Australia.
It's not a question, you're wrong.
These types of posts are what drive me to largely regard lesswrong as unserious. Solve the immediate problem of AGI, and then we can talk about whatever sci-fi bullcrap you want to.
Foxes > Hedgehogs.
You'll learn a lot more about the future paying attention to what's happening right now than by wild extrapolation.
That and raise the production quality and switch subway surfers out for something else like the Minecraft you mentioned. TikTok should have the framework to make these changes.
Tts should be sped up a fair bit
Thank you very much for this response!
Which companies, and to what extent? My internal model says that this is as simple as telling them they have to contract with somebody to dispose it properly.
Fossil fuels are a billion times more fundamental to our economy than mercury.
Also mercury pollution is much more localized with clear, more immediate consequences than CO2 pollution. It doesnt suffer from any 'common good' problems.
I don't understand your model of this at all, do you think if CO2 wasn't a controversial topic, we could just raise gas taxes and people would be fine? Or do you think it would rapidly revert to being a controversial topic?
"Don't be afraid to say 'oops' and change your mind"
CO2 was not brought to public awareness arbitrarily. CO2 came to public awareness because regulating it without negatively impacting a lot of businesses and people is impossible.
Controversial -> Public Awareness
Not
Public Awareness -> Controversial
-
Increasing awareness increases resources through virtue of sheer volume. The more people hear about AI safety, the more likely someone resourceful and amenable hears about AI safety.
-
This is a good sentiment, but 'resource gathering' is an instrumentally convergent strategy. No matter what researchers end up deciding we should do, it'll probably be best done with money and status on our side.
-
Politicisation is not a failure mode, it's an optimistic outcome. Politicized issues get money. Politicized issues get studied. Other failure modes might be that we increase interest in AI in general and result in destructive AI being generated more rapidly, but there's already a massive profit motive in that direction, so I don't know if we can really contribute in that direction. Most other 'failure modes' which involve 'bad publicity' are infinitely preferable to the current state of affairs, given you have enough dignity to be shameless.
-
'Public pressure' isn't really a thing as far as I can tell. The public presses back. Money talks. Wheels turn. Equilibria equilibriate or something. I'm only talking about awareness.
My current plan relies on identifying a good representative for AI safety with enough clout to be taken seriously, contacting them, and then trying to get them into public discussions with rationalist-adjacent e-celebs.
I'd expect this to increase awareness for AI safety and make people who see this content more amenable to advocacy for AI safety in the future. I'd expect a minority of people who watch this content to become very interested in AI safety and try and learn more.
Assuming that I find a good advocate and succeed in getting them into a discussion with a minor celebrity, this could go wrong in the following ways:
- The advocates comes across as unhinged
- The advocates comes across as unlikable
- The advocate cannot explain AI safety well
- The advocate cannot respond to criticism well
As far as I can see, the efficacy and reliability of this plan relies entirely on the character of the advocate. Because this is being tested in a smaller corner of the internet, I think we can believe that if inexplicably results in disaster, the effect will be relatively contained, but honestly I think a pretty small amount of screening can prevent the worst of this.
Any image produced by DALL-E which could also convey or be used to convey misalignment or other risks from AI would be very useful because it could combine the desired messages: "the AI problem is urgent," and "misalignment is possible and dangerous."
For example, if DALL-E responded to the prompt: "AI living with humans" by creating an image suggesting a hierarchy of AI over humans, it would serve both messages.
However, this is only worthy of a side note, because creating such suggested misalignment organically might be very difficult.
Other image prompts might be: "The world as AI sees it," "the power of intelligence," "recursive self-improvement," "the danger of creating life," "god from the machine," etc.