AI Safety & Entrepreneurship v1.0

post by Chris_Leong · 2025-04-26T14:37:11.037Z · LW · GW · 0 comments

Contents

  For the most up-to-date version, see the Wiki page.
  Articles:
  Incubation Programs:
  AIS Friendly General Program:
  Communities:
  VC:
  Organisational Support:
  Other:
  Dustbin - Things that previously existed
None
No comments

Why did I create both a post and a wiki article?

Posts are best for making sure people see the initial version of the post, whilst Wiki articles are best for long-term maintenance. Posting an article that links to a Wiki page provides the best of both worlds.

For the most up-to-date version, see the Wiki page [? · GW].

Articles:

There should be more AI safety organisations [LW · GW]

Why does the AI Safety Community need help founding projects? [LW(p) · GW(p)]

AI Assurance Tech Report

AI Safety as a YC Startup [LW · GW]

Alignment can be the ‘clean energy’ of AI [LW · GW]

AI Tools for Existential Security [EA · GW]

Incubation Programs:

Def/acc at Entrepreneur First - this is a new program, which focuses on "defensive" tech, which will produce a wide variety of startups, some of which may help with reducing x-risk, but others may include AI applications like autonomous weapons or diagnostic medical software, which, while they generally won't increase x-risk, may not reduce it either.

Catalyze - AI Safety Incubation

Seldon Accelerator: “We take on AGI risk with optimism”

AE Studio has expressed interest in helping seed neglected approaches, incl. ideas that take the form of business ideas

AIS Friendly General Program:

Founding to Give [EA · GW]: General EA incubator

Entrepreneur First - They run def/acc, so this could be a decent alternative

Halycon Futures: “We’re an entrepreneurial nonprofit and VC fund dedicated to making AI safe, secure and good for humanity.” They do new project incubation, grants, investments, ect.

Communities:

Entrepreneurship Channel EA Anyway

AI Safety Founders: Website, Discord, LinkedIn

VC:

AI Safety Seed Funding Network [EA · GW]

Lionheart Ventures - “We invest in the wise development of transformative technologies.”

Juniper Ventures - Invests in companies “working to make AI safe, secure and beneficial for humanity”. Published the AI Assurance Tech Report

Polaris Ventures - “We support projects and people aiming to build a future guided by wisdom and compassion for all”

Mythos Ventures - “an early-stage venture capital firm investing in prosocial technologies for the transformative AI era.”

Metaplanet - “We support novel and evidence-based innovation that could produce an outsized return for the benefit of humankind.”

Anthology Fund - Backed by Anthropic. One of the five key areas is “trust and safety tooling”

Menlo Ventures - Lead the GoodFire round (article)

Safe AI Fund

Organisational Support:

Ashgro - fiscal sponsorship

Rethink Priorities Special Projects - provides fiscal sponsorship or incubation

Other:

Hackathon for AI Safety Startups - Ran once by Apart Research, may run again

Constellation Residency - Year-long position

Nonlinear - Free coaching for people who are running an AI safety startup or who are considering starting one

Dustbin - Things that previously existed

Longtermist Entrepreneurship Project [? · GW] - Retrospective [? · GW]

0 comments

Comments sorted by top scores.