What is everyone doing in AI governance

post by Igor Ivanov (igor-ivanov) · 2023-07-08T15:16:10.249Z · LW · GW · 0 comments

Contents

  What is this post about, and how to use it
    This is a flawed list
  Major non-profit organizations
    Centre for the Governance of AI (GovAI)
      Areas of work
      Mechanisms of influence:
      Other notable things
    Center for AI Safety (CAIS)
      Areas of work
      Mechanisms of influence 
    Existential Risk Observatory (ERO)
      Areas of work
      Mechanisms of influence
      Other notable things
    Centre for Long-term Resilience (CLTR)
      Areas of work
      Mechanisms of influence 
      Other notable things
    Future of Life Institute (FLI)
      Areas of work
      Mechanisms of influence
      Other notable things
    The Future Society (TFS)
      Areas of work
      Mechanisms of influence
      Other notable things
    Rethink Priorities (RP)
      Areas of work
      Mechanisms of influence
      Other notable things
    UK Foundation Model Task Force 
  Less-influential non-profit organizations 
      Center for Human Compatible AI (CHAI)
      Campaign for AI Safety
      PauseAI
      Future of humanity institute (FHI)
      Global Catastrophic Risk Institute (GCRI)
      Center on Long-Term Risk (CLR)
      BlueDot impact
  Governance teams at AI labs
      The public policy team at OpenAI
      Areas of work
      The policy team at Anthropic
      Areas of work
      The policy team at Google DeepMind 
      The governance team at Conjecture
None
No comments

What is this post about, and how to use it

This is a list of AI governance organizations with a brief description of their directions of work, so a reader can get a basic understanding of what they are doing and explore them in-depth on their own.

There is another recent post [EA · GW] on a similar topic that explores AI governance research agendas with a lesser focus on the areas of work of specific governance organizations.


This is a flawed list

I made this list using publicly available data, as well as by talking with people from the field. I am sure that I missed something important, especially in the section on the governance teams of AI labs since they are less open than nonprofits, but the best way to get the right answer on the internet is to post the wrong one. 
 
At some point, adding each new bit of information started taking more and more time, and I decided to stop so this post is reasonably useful and will not cause me to struggle.


DM me if you see mistakes or think I forgot something important.


 

Major non-profit organizations

Centre for the Governance of AI (GovAI)


Areas of work

  1. Scientific research on AI policies
  2. Educational and fellowship programs 
  3. Some efforts to improve coordination among AI governance orgs


 

Mechanisms of influence:

  1. Many prominent AI governance specialists are alumni or former employees of GovAI including heads of policy at Deepmind, OpenAI, and Anthropic. So GovAI has strong connections among decision-makers
  2. GovAI produced a lot of academic research on policies. Mostly academic, to a lesser extent - applied


 

Other notable things

GovAI has an established brand of a respectable organization

 


 

Center for AI Safety (CAIS)

 

Areas of work

  1. Field building
    1. Courses and fellowships for starting a career in technical AI safety and philosophy
    2. Offer compute resources for AI safety researchers including compute cluster
    3. Organize competitions for AI safety researchers
  2. Research
    1. Technical research on AI safety 
    2. Research on future societal risks from AI


 

Mechanisms of influence 

  1. CAIS nurtures technical researchers and philosophers



 

Existential Risk Observatory (ERO)

 

Areas of work

  1. Raise awareness on AI x-risks in the press (dozens of publications including TIME magazine) and on conferences
  2. Consultations with Dutch MPs, lobbying in the Dutch government


 

Mechanisms of influence

Lobbying and publicity in media


 

Other notable things

They do not just focus on the risks from AI, but on other potentially existential risks as well: e.g. pandemics and nuclear war



 

Centre for Long-term Resilience (CLTR)

 

Areas of work

  1. Developing policy proposals on extreme risks
  2. Lobbying policies on extreme risks in a number of jurisdictions with a focus on the UK. They describe themselves as a glue between research on extreme risks and actionable policies


 

Mechanisms of influence 

CLTR has solid connections among UK political decision-makers. These decision-makers know CLTR and trust them.


 

Other notable things

  1. CLTR not just focuses on the risks from AI, but also biological risks like pandemics of bioweapons
  2. Right now there is the window of opportunity for shaping AI policies, so CLTR's influence is now at its maximum



 

Future of Life Institute (FLI)

 

Areas of work

  1. Policy advocacy in the US, the EU, and the UK (EU AI act, NIST framework in the US, policies against integration of AI in Nuclear launch)
  2. Education (Podcast, live events, videos)
  3. Grantmaking for fellowships and research


 

Mechanisms of influence

As an outsider, the only projects I was able to identify as ongoing are grantmaking and public outreach projects


 

Other notable things

They do not just focus on the risks from AI, but on other potentially existential risks as well: biotechnology, nukes, climate change



 

The Future Society (TFS)

 

Areas of work

  1. Applied research for AI policies in the EU (including the EU AI Act)
  2. Organizer of The Athens Roundtable on AI and Rule of Law - the largest AI governance conference
  3. Enabling coordination among policy-makers, AI developers, standard-setting bodies, and AI safety community members to enable "distributed competence" for policy-making and effective mechanisms of policy enforcement

 


Mechanisms of influence

  1. Solid network among all sorts of important actors in AI regulation
  2. Several employees are experts in international regulatory bodies
  3. A platform for coordination among all important actors in AI regulation


 

Other notable things

On the paper, TFS is quite influential as they are like a large squid with their tentacles in almost every major AI safety actor, but the real extent of their influence is unclear to me.



 

Rethink Priorities (RP)

 

Areas of work

  1. Research on the Chinese AI Safety strategy and Chinese-western relations
  2. Consultations with the US policy-makers on AI regulations
  3. Launching an incubator for the Longtermist entrepreneurs


 

Mechanisms of influence

Unclear

 

 

Other notable things

  1. Focus not just on AI safety, but also on animal welfare, climate change and global health
  2. Relatively small influence in the US policy making
  3. Planning to do research on the safety policies for AI labs, as well as compute governance (governance of computational resources. e.g. chips)
  4. Publish a newsletter



 

UK Foundation Model Task Force 

No website yet


By the time this post is published, there is little information on this organization, but it has solid support from the UK government as well as from all major AI labs and has the ambition to become the main international body for AI regulation.




 

Less-influential non-profit organizations
 

The organizations I consider less influential are described briefly. There are no objective criteria on which organization is major and which is not. This classification is based only on my impression.
 

Center for Human Compatible AI (CHAI)

Mostly conceptual research on ML, AI safety, and cognitive sciences, but Stuart Russel, the founder of CHAI is a high-profile public figure who is active in media.


 

Campaign for AI Safety

Mostly, research on public opinion of AI risks that is useful for shaping narratives around topics.


 

PauseAI

Protests to pause AI development, as well as some media and politicians outreach.


 

Future of humanity institute (FHI)

Conceptual research on AI governance and technical AI safety, as well as strategic thinking in general. Last activity on their website is dated 2021.

 

Global Catastrophic Risk Institute (GCRI)

Minor policy advising as well as public outreach on AI safety. Focuses mostly not on AI risks, but other global risks.


 

Center on Long-Term Risk (CLR)

It was hard for me to determine whether CLR is actively doing any work on AI governance, but it seems like they have some influence on AI labs and doing academic research on cooperation.


 

BlueDot impact

Runs AI safety fundamentals courses


 

 

Governance teams at AI labs


I expect this part of the list to be way less complete compared to the previous ones since AI labs are generally way less open to the public about their work. If you know more non-confidential information than me, feel free to DM me.

 

The public policy team at OpenAI

Areas of work

  1. Coordination among leading AI labs to slow down AI progress and ensure safety
  2. Advocating for the creation of an international regulatory body for AI that controls the development and deployment of AI models. Something similar to IAEA
  3. Development of instruments for the democratic public oversight over the governance of AGI
  4. Research on the effects of AI systems on society

Sources: [1] [2] [3]


 

The policy team at Anthropic

Areas of work

  1. Support the development of safety evaluations for AI systems
  2. Enforce pre-registration of large training runs by regulators
  3. Empower third-party audit of models, as well as red-teaming

Sources: [1] [2]


 

The policy team at Google DeepMind
 

Although I found some sources describing Deepmind's AI policy work ([1] [2] [3]) I was unable to create a comprehensive image of what are they doing so I decide to not describe it here to nod mislead readers.
 

 

The governance team at Conjecture


One of the team's goals is advocating for narratives on AI x-risks among general the public and politicians. The members of the team also launched the Stop AGI project which is aimed to achieve similar goals.

0 comments

Comments sorted by top scores.