Posts
Comments
SASH isn't official (we're waiting on funding).
Here is TARA :)
https://www.lesswrong.com/posts/tyGxgvvBbrvcrHPJH/apply-to-be-a-ta-for-tara
I think it will take less than 3 years for the equivalent of 1,000,000 people to get laid off.
If transformative AI is defined by its societal impact rather than its technical capabilities (i.e. TAI as process not a technology), we already have what is needed. The real question isn't about waiting for GPT-X or capability Y - it's about imagining what happens when current AI is deployed 1000x more widely in just a few years. This presents EXTREMELY different problems to solve from a governance and advocacy perspective.
E.g. 1: compute governance might no longer be a good intervention
E.g. 2: "Pause" can't just be about pausing model development. It should also be about pausing implementation across use cases
Yep
E.g. 50,000 travel agents lose their jobs in 25/26
That's pretty high. What use cases are you imagining as the most likely?
I'm not sure of an org that deals with ultra-high net worth individuals (longview?), but someone should reach out to Bryan Johnson. I think he could be persuaded to invest in AI Safety (skip to 1:07:15)
I spent 8 years working in strategy departments for Ad Agencies. If you're interested in the science behind brand tracking, I recommend you check out the Ehrenberg-Bass Institutes work on Category Entry Points: https://marketingscience.info/research-services/identifying-and-prioritising-category-entry-points/
What is your AI Capabilities Red Line Personal Statement? It should read something like "when AI can do X in Y way, then I think we should be extremely worried / advocate for a Pause*".
I think it would be valuable if people started doing this; we can't feel when they're on an exponential, so its likely we will have powerful AI creep up on us.
@Greg_Colbourn just posted this and I have an intuition that people are going to read it and say "while it can do Y it still can't do X"
*in the case you think a Pause is ever optimal.
I'd like to see research that uses Jonathan Haidt's Moral Foundations research and the AI risk repository to forecast whether there will soon be a moral backlash / panic against frontier AI models.
Ideas below from Claude:
# Conservative Moral Concerns About Frontier AI Through Haidt's Moral Foundations
## Authority/Respect Foundation
### Immediate Concerns
- AI systems challenging traditional hierarchies of expertise and authority
- Undermining of traditional gatekeepers in media, education, and professional fields
- Potential for AI to make decisions traditionally reserved for human authority figures
### Examples
- AI writing systems replacing human editors and teachers
- AI medical diagnosis systems challenging doctor authority
- Language models providing alternative interpretations of religious texts
## Sanctity/Purity Foundation
### Immediate Concerns
- AI "thinking" about sacred topics or generating religious content
- Artificial beings engaging with spirituality and consciousness
- Degradation of "natural" human processes and relationships
### Examples
- AI spiritual advisors or prayer companions
- Synthetic media creating "fake" human experiences
- AI-generated art mixing sacred and profane elements
## Loyalty/Ingroup Foundation
### Immediate Concerns
- AI systems potentially eroding national sovereignty
- Weakening of traditional community bonds
- Displacement of local customs and practices
### Examples
- Global AI systems superseding national control
- AI replacing local business relationships
- Automated systems reducing human-to-human community interactions
## Fairness/Reciprocity Foundation
### Immediate Concerns
- Unequal access to AI capabilities creating new social divides
- AI systems making decisions without clear accountability
- Displacement of workers without reciprocal benefits
### Examples
- Elite access to powerful AI tools
- Automated systems making hiring/firing decisions
- AI wealth concentration without clear social benefit
## Harm/Care Foundation
### Immediate Concerns
- Potential for AI systems to cause unintended societal damage
- Psychological impact on children and vulnerable populations
- Loss of human agency and autonomy
### Examples
- AI manipulation of emotions and behavior
- Impact on childhood development
- Dependency on AI systems for critical decisions
## Key Trigger Points for Conservative Moral Panic
1. Religious Content Generation
- AI systems creating or interpreting religious texts
- Automated spiritual guidance
- Virtual religious experiences
2. Traditional Value Systems
- AI challenging traditional moral authorities
- Automated ethical decision-making
- Replacement of human judgment in moral domains
3. Community and Family
- AI impact on child-rearing and education
- Disruption of traditional social structures
- Replacement of human relationships
4. National Identity
- Foreign AI influence on domestic affairs
- Cultural homogenization through AI
- Loss of local control over technology
## Potential Flashpoints for Immediate Backlash
1. Education System Integration
- AI teachers and tutors
- Automated grading and assessment
- Curriculum generation
2. Religious Institution Interaction
- AI pastoral care
- Religious text interpretation
- Spiritual counseling
3. Media and Information Control
- AI content moderation
- Synthetic media generation
- News creation and curation
4. Family and Child Protection
- AI childcare applications
- Educational AI exposure
- Family privacy concerns
I think there is a 10-20 per cent chance we get digital agents in 2025 that produce a holy shit moment as big as the launch of chatgpt.
If that happens I think that will produce another round of questions that sounds approximately like “how were we so unprepared for this moment”.
Fool me once, shame on you…
I am 90% sure that most AI Safety talent aren't thinking hard enough about what Neglectedness. The industry is so nascent that you could look at 10 analogous industries, see what processes or institutions are valuable and missing and build an organisation around the highest impact one.
The highest impact job ≠ the highest impact opportunity for you!
AI Safety (in the broadest possible sense, i.e. including ethics & bias) is going be taken very seriously soon by Government decision makers in many countries. But without high quality talent staying in their home countries (i.e. not moving to UK or US), there is a reasonable chance that x/c-risk won’t be considered problems worth trying to solve. X/c-risk sympathisers need seats at the table. IMO local AIS movement builders should be thinking hard about how to keep talent local (this is an extremely hard problem to solve).
Big AIS news imo: “The initial members of the International Network of AI Safety Institutes are Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States.”
H/T @shakeel
I beta tested a new movement building format last night: online networking. It seems to have legs.
V quick theory of change:
> problem it solves: not enough people in AIS across Australia (especially) and New Zealand are meeting each other (this is bad for the movement and people's impact).
> we need to brute force serendipity to create collabs.
> this initiative has v low cost
quantitative results:
> I purposefully didn't market it hard because it was a beta. I literally got more people that I hoped for
> 22 RSVPs and 18 attendees
> this says to me I could easily get 40+
> average score for below question was 7.27, which is very good for a beta test
I used Zoom, which was extremely clunky. These results suggest to me I should;
> invest in software designed for this use case, not zoom
> segment by career stream (governance vs technical) and/or experience (beginner vs advanced)
> run it every second month
I have heaps of qualitative feedback from participants but don't have time to share it here.
Email me if interested: yanni@aisafetyanz.com.au
Is anyone in the AI Governance-Comms space working on what public outreach should look like if lots of jobs start getting automated in < 3 years?
I point to Travel Agents a lot not to pick on them, but because they're salient and there are lots of them. I think there is a reasonable chance in 3 years that industry loses 50% of its workers (3 million globally).
People are going to start freaking out about this. Which means we're in "December 2019" all over again, and we all remember how bad Government Comms were during COVID.
Now is the time to start working on the messaging!
Three million people are employed by the travel (agent) industry worldwide. I am struggling to see how we don't lose 80%+ of those jobs to AI Agents in 3 years (this is ofc just one example). This is going to be an extremely painful process for a lot of people.
I should have been clearer - I like the fact that they're trying to create a larger tent and (presumably) win ethics people over. There are many reasons not also not like the ad. I would also guess that they have an automated campaign running with several (maybe dozens) of pieces of creative. Without seeing their analytics it would be impossible to know which performs the best, but it wouldn't surprise me if it was this one (lists work).
I really like this ad strategy from BlueDot. 5 stars.
90% of the message is still x / c-risk focussed, but by including "discrimination" and "loss of social connection" they're clearly trying to either (or both);
- create a big tent
- nudge people that are in "ethics" but sympathetic to x / c-risk into x / c-risk
(prediction: 4000 disagreement Karma)
Thanks for the heads up! Should be there now :)
[IMAGE] Extremely proud and excited to watch Greg Sadler (CEO of Good Ancestors Policy) and Soroush Pour (Co-Founder of Harmony Intelligence) speak to the Australian Federal Senate Select Committee on Artificial Intelligence. This is a big moment for the local movement. Also, check out who they ran after.
I think how delicately you treat your personal Overton Window should also depend on your timelines.
My instinct as to why people don't find it a compelling argument;
- They don't have short timelines like me, and therefore chuck it out completely
- Are struggling to imagine a hostile public response to 15% unemployment rates
- Copium
I always get downvoted when I suggest that (1) if you have short timelines and (2) you decide to work at a Lab then (3) people might hate you soon and your employment prospects could be damaged.
What is something obvious I'm missing here?
One thing I won't find convincing is someone pointing at the Finance industry post GFC as a comparison.
I believe the scale of unemployment could be much higher. E.g. 5% ->15% unemployment in 3 years.
Thanks for the feedback! I had a feeling this is where I'd land :|
I’d like to quickly skill up on ‘working with people that have a kind of neurodivergence’.
This means;
- understanding their unique challenges
- how I can help them get the best from/for themselves
- but I don’t want to sink more than two hours into this
Can anyone recommend some online content for me please?
Hey thanks for your thoughts! I'm not sure what you mean by "obvious business context", but I'm either meeting friends / family (not business) or colleagues / community members (business).
Is there a third group that I'm missing?
If you're actually talking about the second group, I wonder how much your and their productivity would be improved if you were less lax about it?
It just occurred to me that maybe posting this isn't helpful. And maybe net negative. If I get enough feedback saying it is then I'll happily delete.
I don't know how to say this in a way that won't come off harsh, but I've had meetings with > 100 people in the last 6 months in AI Safety and I've been surprised how poor the industry standard is for people:
- Planning meetings (writing emails that communicate what the meeting will be about, it's purpose, my role)
- Turning up to scheduled meetings (like literally, showing up at all)
- Turning up on time
- Turning up prepared (i.e. with informed opinions on what we're going to discuss)
- Turning up with clear agendas (if they've organised the meeting)
- Running the meetings
- Following up after the meeting (clear communication about next steps)
I do not have an explanation for what is going on here but it concerns me about the movement interfacing with government and industry. My background is in industry and these just seem to be things people take for granted?
It is possible I give off a chill vibe, leading to people thinking these things aren't necessary, and they'd switch gears for more important meetings!
About a year and a half ago I listened to Loch Kelly for the first time. I instantly glimpsed the nondual nature of reality. Besides becoming a parent, it is the most important thing that has ever happened to me. The last year and a half has been a process of stabilising this glimpse and it is going extremely well. I relate deeply to the things that Sasha is describing here. Thanks for sharing ❤️
The Australian Federal Government is setting up an "AI Advisory Body" inside the Department of Industry, Science and Resources.
IMO that means Australia has an AISI in:
9 months (~ 30% confidence)
12 months (~ 50%)
18 months (~ 80%)
Something bouncing around my head recently ... I think I agree with the notion that "you can't solve a problem at the level it was created".
A key point here is the difference between "solving" a problem and "minimising its harm".
- Solving a problem = engaging with a problem by going up a level from which is was createwd
- Minimising its harm = trying to solve it at the level it was created
Why is this important? Because I think EA and AI Safety have historically focussed (and has their respective strengths in) harm-minimisation.
This applies obviously the micro. Here are some bad examples:
- Problem: I'm experiencing intrusive + negative thoughts
- Minimising its harm: engage with the thought using CBT
- Attempting to solve it by going meta: apply meta cognitive therapy, see thoughts as empty of intrinsic value, as farts in the wind
- Problem: I'm having fights with my partner about doing the dishes
- Minimising its harm: create a spreadsheet and write down every thing each of us does around the house and calculate time spent
- Attempting to solve it by going meta: discuss our communication styles and emotional states when frustration arises
But I also think this applies at the macro:
- Problem: People love eating meat
- Minimising harm by acting at the level the problem was created: asking them not to eat meat
- Attempting to solve by going meta: replacing the meat with lab grown meat
- Problem: Unaligned AI might kill us
- Minimising harm by acting at the level the problem was created: understand the AI through mechanistic interpretability
- Attempting to solve by going meta: probably just Governance
I think it is good to have some ratio of upvoted/agreed : downvotes/disagreed posts in your portfolio. I think if all of your posts are upvoted/high agreeance then you're either playing it too safe or you've eaten the culture without chewing first.
Hi! I think in Sydney we're ~ 3 seats short of critical mass, so I am going to reassess the viability of a community space in 5-6 months :)
It sounds like you're saying you feel quite comfortable with how long you wait, which is nice to hear :)
I'm pretty confident that a majority of the population will soon have very negative attitudes towards big AI labs. I'm extremely unsure about what impact this will have on the AI Safety and EA communities (because we work with those labs in all sorts of ways). I think this could increase the likelihood of "Ethics" advocates becoming much more popular, but I don't know if this necessarily increases catastrophic or existential risks.
Not sure if this type of concern has reached the meta yet, but if someone approached me asking for career advice, tossing up whether to apply for a job at a big AI lab, I would let them know that it could negatively affect their career prospects down the track because so many people now perceive such as a move as either morally wrong or just plain wrong-headed. And those perceptions might only increase over time. I am not making a claim here beyond this should be a career advice consideration.
More thoughts on "80,000 hours should remove OpenAI from the Job Board"
- - - -
A broadly robust heuristic I walk around with in my head is "the more targeted the strategy, the more likely it will miss the target".
Its origin is from when I worked in advertising agencies, where TV ads for car brands all looked the same because they needed to reach millions of people, while email campaigns were highly personalised.
The thing is, if you get hit with a tv ad for a car, worst case scenario it can't have a strongly negative effect because of its generic content (dude looks cool while driving his 4WD through rocky terrain). But when an email that attempts to be personalised misses the mark, you feel it (Hi {First Name : Last Name}!).
Anyway, effective altruism loves targeting. It is virtually built from the ground up on it (RCTs and all that, aversion to PR etc).
Anyway, I am thinking about this in relation to the recent "80,000 hours should remove OpenAI from the Job Board" post.
80k is applying an extremely targeted strategy by recommending people work in some jobs at the workplaces creating the possible death machines (caveat 1, caveat 2, caveat 3, caveatn).
There are so many failure modes involved with those caveats that makes it virtually impossible to not go catastrophically poorly in at least a few instances and take the mental health of a bunch of young and well-meaning EAs with it.
[IMAGE] there is something about the lack overlap between these two audiences that makes me uneasy. WYD?
I think an assumption 80k makes is something like "well if our audience thinks incredibly deeply about the Safety problem and what it would be like to work at a lab and the pressures they could be under while there, then we're no longer accountable for how this could go wrong. After all, we provided vast amounts of information on why and how people should do their own research before making such a decision".
The problem is, that is not how most people make decisions. No matter how much rational thinking is promoted, we're first and foremost emotional creatures that care about things like status. So, if 80k decides to have a podcast with the Superalignment team lead, then they're effectively promoting the work of OpenAI. That will make people want to work for OpenAI. This is an inescapable part of the Halo effect.
Lastly, 80k is explicitly targeting very young people who, no offense, probably don't have the life experience to imagine themselves in a workplace where they have to resist incredible pressures to not conform, such as not sharing interpretability insights with capabilities teams.
The whole exercise smacks of nativity and I'm very confident we'll look back and see it as an incredibly obvious mistake in hindsight.
A piece of career advice I've given a few times recently to people in AI Safety, which I thought worth repeating here, is that AI Safety is so nascent a field that the following strategy could be worth pursuing:
1. Write your own job description (whatever it is that you're good at / brings joy into your life).
2. Find organisations that you think need thing that job but don't yet have it. This role should solve a problem they either don't know they have or haven't figured out how to solve.
3. Find the key decision maker and email them. Explain the (their) problem as you see it and how having this role could fix their problem. Explain why you're the person to do it.
I think this might work better for mid-career people, but, if you're just looking to skill up and don't mind volunteering, you could adapt this approach no matter what stage you're at in your career.
TIL that the words "fact" and "fiction" come from the same word: "Artifice" - which is ~ "something that has been created".
When I was ~ 5 I saw a homeless person on the street. I asked my dad where his home was. My dad said "he doesn't have a home". I burst into tears.
I'm 35 now and reading this post makes me want to burst into tears again. I appreciate you writing it though.
I'm pleased to announce the launch of a brand new Facebook group dedicated to AI Safety in Wellington: AI Safety Wellington (AISW). This is your local hub for connecting with others passionate about ensuring a safe and beneficial future with artificial intelligence / reducing x-risk.To kick things off, we're hosting a super casual meetup where you can:
- Meet & Learn: Connect with Wellington's AI Safety community.
- Chat & Collaborate: Discuss career paths, upcoming events, and training opportunities.
- Share & Explore: Whether you're new to AI Safety or a seasoned expert, come learn from each other's perspectives.
- Expand the Network: Feel free to bring a friend who's curious about AI Safety!
Join the group and RSVP for the meetup here
https://www.facebook.com/groups/500872105703980
one of my more esoteric buddhist practices is to always spell my name in lower case; it means I am a regularly reminded of the illusory nature of Self, while still engaging in imagined reality (Parikalpitā-svabhāva) in a practical way.
When AI Safety people are also vegetarians, vegans or reducetarian, I am pleasantly surprised, as this is one (of many possible) signals to me they're "in it" to prevent harm, rather than because it is interesting.
Hey mate thanks for the comment. I'm finding "pretty surprised" hard to interpret. Is that closer to 1% or 15%?
Hi Ann! Thank you for your comment. Some quick thoughts:
"I would consider, for the sake of humility, that they might disagree with your assessment for actual reasons, rather than assuming confusion is necessary."
- Yep! I have considered this. The purpose of my post is to consider it (I am looking for feedback, not upvotes or downvotes).
"They also happen to have a have a p(doom from not AGI) of 40% from combined other causes, and expect an aligned AGI to be able to effectively reduce this to something closer to 1% through better coordinating reasonable efforts."
- This falls into the confused category for me. I'm not sure how you have a 40% p(doom) from something other than unaligned AGI. Could you spell out for me what could make such a large number?
Hi Richard! Thanks for the comment. It seems to me that might apply to < 5% of people in capabilities?