Become a PIBBSS Research Affiliate

post by Nora_Ammann, DusanDNesic · 2023-10-10T07:41:02.037Z · LW · GW · 6 comments

Contents

  Goals and key features of the Affiliate Program
  Requirements of the Affiliate Program
  What are we looking for in an ideal affiliate?
  About PIBBSS and our research priorities
  How to apply
None
6 comments

TL;DR: PIBBSS is hiring research affiliates! 

PIBBSS is a research initiative facilitating work that draws on the parallels between intelligent behavior in natural and artificial systems, and leveraging these insights towards making AI systems safe, beneficial and aligned.

We want to support excellent researchers pursuing “PIBBSS-style” AI alignment research by providing them with tailored longer-term support: a full-time salary, a lively research community, operational support and more. The initial commitment is 6 months, with potential extensions to 12 or more. 

To apply, click here. To be considered as part of the next iteration of affiliates, apply by November 5th.

---

At PIBBSS (“Principles of Intelligent Behavior in Biological and Social Systems”), our mission is to facilitate AI alignment research that draws on the parallels between intelligent behavior in natural and artificial systems. (For more details, see the section "About PIBBSS and our research priorities" below.)

Since PIBBSS’ inception in 2022, we have run two iterations of our 3-months research fellowships, as well as a reading group [LW · GW], an ongoing speaker series, and a number of research retreats exploring topics in line with PIBBSS’ research interests. Our alumni have gone on to pursue AI alignment research at places like OpenAI, Anthropic, the Alignment of Complex Systems research group, and academia, as well as through independent funding. 

While we continue to be excited about the value of those activities, we also believe that there is more value to be had! In particular, we see potential in a program geared towards supporting longer-term, more sustained research efforts in line with PIBBSS’ research bet. 

As such, we are delighted to launch our new PIBBSS Affiliate Program. 

Goals and key features of the Affiliate Program

At its core, our goal in running this program is to counterfactually help produce substantial, high-quality and legible research progress on problems in AI risk, alignment and governance. The affiliate program aims to do this by supporting promising individual researchers in a highly tailored fashion to scope, develop, and progress on their personal research agendas. 

Concrete success scenarios might look like (but are not limited to) affiliates publishing insightful research results; joining the efforts of existing research groups or founding new ones; identifying, scoping, testing and building traction on novel or neglected research avenues; maturing their personal research agenda and secure independent funding; etc.

The PIBBSS Affiliate Program has the following key features: 

• Affiliates receive a salary of 5,000 USD/month/FTE. We are open to paying higher salaries to exceptional candidates. 
  • We are, by default, looking for affiliates who will be joining the program full-time. That said, we are open to adapting to affiliates’ circumstances where this seems appropriate, such as joining the program in a part-time fashion. We also have the capacity to offer support to people who do not want to leave their current research positions.
• Affiliates benefit from a range of support structures aimed at helping them focus on and advance in their research endeavors, such as:
  • Quarterly research retreats bringing together affiliates as well as other alignment researchers frothe the m our extended network 
  • Personalized research and strategic support, typically in the form of bi-weekly calls with your point person (with option to adapt frequency of nature of support to the ffiliate’s needs and preferences)
  • Access to the PIBBSS research community and extended network, including joining our slack space, tailored introductions, opportunities to participate in workshops, etc. 
  • Financial and administrative support in order to e.g. visit relevant research conferences, get access to base models and compute where required for research, etc.
  • Access to office space depending on the needs/preferences of affiliates.
  • Note: A key feature of the program is to provide highly tailored support to our affiliates, according to what counterfactually helps them most make progress on their research. As such, the above list, while representative, is not necessarily fully precise or comprehensive.
• The initial duration of the affiliateship is 6 months, at which point we review affiliates for a possible extension to 12 months. In exceptional circumstances, we will make initial offers of 12 months. 
  • Our vision is for the affiliate program to provide a serious long-term home for promising researchers pursuing PIBBSS-style AI alignment research at time scales of 1 year and more, including the possibility of permanent positions. 
• We want to build on and strengthen the supportive, curious and dedicated culture that PIBBSS has built over the last two years. We care about good epistemic practices while navigating the intricacies of making sense of a complex world and particularly emphasize the value of epistemic pluralism [AF · GW] in doing so. 

 At the current point in time, we orient to the affiliate program with an experimental lens - testing the hypothesis that this program can provide significant value, and learning about how exactly we can support our affiliates most effectively. Depending on what we learn, we might scale up or scale down, streamline the program or make it more tailored to individual affiliates, aim to have affiliates for time-limited or for open-ended periods, or do other things which increase the impact of the program. 

Requirements of the Affiliate Program

The affiliate program is a very open-ended program with limited requirements. Here is what we asked from our affiliates:

• Submitting a quarterly report reflecting on your research progress and prospective research plan. 
  • The reports are meant to help the affiliate by providing an accountability structure and encouraging them to zoom out and reflect on a regular basis. They also help us understand what affiliates have been working on and whether there is anything we can do to support them better going forward.
• Participating in our ~quarterly research retreats 
• A genuine commitment to making progress on questions related to AI safety, alignment and governance, and constructive, truth-seeking and kind participation in our shared epistemic culture 

What are we looking for in an ideal affiliate?

Affiliates are selected for being in a strong position to pursue promising PIBBSS-aligned research directions in AI alignment in a highly self-directed fashion. As such, we are looking for (a mix of) the following characteristics:

• Scholarly competence/caliber in your domain(s) of expertise 
• Strong prior understanding of AI Risk & Alignment
• Evidence of good research judgment/taste
• Evidence for good research productivity, including strong self-management skills
• Promisingness of your proposed line of research
• Ability to counterfactually benefit from what PIBBSS can offer 

Note that, in our experience, great PIBBSS-style researchers have often had non-standard backgrounds. As such, we wish to encourage individuals with varied backgrounds and perspectives to apply. 

About PIBBSS and our research priorities

PIBBSS is a research initiative facilitating work that draws on the parallels between intelligent behavior in natural and artificial systems, and leveraging these insights towards making AI systems safe, beneficial and aligned.

In essence, we believe that the study of complex natural and social systems can help us better understand the risk involved in implementing such behaviors in artificial substrates and improve our ability to develop AI systems such that they reliably display desired safety- and governability-relevant properties. (We describe in some more detail the nature of and some background assumptions going into our epistemic bet in this write up [AF · GW], as well as in this talk). 

To provide more concrete pointers towards the type of research we’re excited about, here is a sample of the work that we have supported in the past: 

• Aiming to develop a principled understanding of the dynamics emerging from multi-agent interactions between AI systems, such as by drawing on the study of collective behavior in army ants or models from evolutionary biology and ecology 
• Exploring how to include understanding of multi-agent AI dynamics in developing model evaluations [forthcoming] 
• Exploring novel avenues for interpretability, such as A Geometric Approach to Interpretability and Singular Learning Theory
• Building towards naturalized theories of agency, such as by exploring the role of abstractions in the phenomenon of life;  testing the natural abstraction hypothesis in the game of life [AF · GW]; thinking about the characteristics of agents that apply the intentional stance on themselves (“auto-intentional agents”); or applying a dynamical systems perspective on goal-oriented behavior 
• Tackling problems in decision theory, such as formalizing updateless decision theory, robustness to self-modification, or trying to bridge expected utility maximization and optimization [AF · GW]
• Developing more realistic account of values and practical reasoning, and exploring their implications for our understanding of the AI alignment problem [forthcoming] 
• Evaluating the potential and limits of existing legal tools for reducing catastrophic risks from AI
• Testing the potential for productive epistemic exchange between AI alignment and other fields/topics, e.g. artificial life, basal cognition, morphological computation, active inference, critical social theory, etc.
• Looking at philosophy of sciences (and the special sciences) to find ways to make our.  [LW · GW]research. efforts. more [LW · GW]. productive [LW · GW].

Our fellows and collaborators have brought expertise from many domains to bear on the question of AI alignment, including but not limited to:

• Ecology
• Evolutionary and developmental biology
• Systems biology
• Biophysics
• Neuroscience
• Cognitive science
• Linguistics 
• Legal theory
• Political theory
• Economic theory
• Social theory
• Complex systems studies 
• Cybernetics
• Network science
• Physics
• Philosophy of Science

How to apply

Applications are now open. You can apply by following this link. 

We accept new affiliates on a ~quarterly basis. To be considered as part of the next iteration, apply by November 5th.

We expect to hire 2-6 (median 4) affiliates over the coming 6 months.

About the application process: The details of the application process may vary between candidates in virtue of which information we think will most help us make well-informed decisions. In most cases, the initial stage (application form) is followed by one or more interviews to discuss your research in more depth. We may also reach out to your references, or in some cases, ask you to submit a work task of ~2-5h. 

6 comments

Comments sorted by top scores.