AI Safety Fundamentals Fellowship

Applications for the Fall 2024 program are now closed. Thank you to all who applied!

CAISH runs a 6-week in-person reading group on AI safety, covering topics like neural network interpretability, learning from human feedback and goal misgeneralisation in reinforcement learning agents. We meet for 2 hours weekly in small groups led by experienced facilitators, with free dinner provided. Meet safety researchers in Cambridge and attend your choice of workshops, ranging from coding assignments to AI policy discussions.

The fellowship is designed for students and professionals of various backgrounds, with the option for participants to specialise in either technical AI safety or AI goverance & policy. See the description below for further details. 

The curriculum

The course is offered over a period of 6 weeks, and gives students the option to specialise in either technical AI safety or AI governance.

The first 4 weeks of the curriculum is based on the AGI Safety Fundamentals program developed by AI safety researcher Richard Ngo (Governance team at OpenAI, previously AGI safety team at DeepMind).

Following this, fellows will take part in a series of concurrent workshops over the course of 2 weeks, spanning a wide range of topics (see below). Participants have the opportunity to tailor their experience by selectively attending the workshops that align with their interests.

During these two weeks, two workshop tracks are offered: (1) the Technical Track, designed for students with science, engineering, and maths backgrounds, and (2) the Governance Track, designed for participants with political science, public policy, international studies, or similar backgrounds.

Course details

  • People who are interested in making advanced AI safe and in engaging with either technical or governmental problems in this field.

    The Technical Track of the course may particularly appeal to people with computer science, maths, physics, engineering, and quantitative backgrounds.

    The Governance Track may particularly appeal to public policy, internationl relations, or political science students, or those with similar background.

    We consider applicants from all academic backgrounds, even if you do not fit into one of the categories above.

  • Participants on the Technical Track are expected to be familiar with the basic concepts of ML, such as stochastic gradient descent, neural networks, and reinforcement learning. Those without previous ML experience should be willing to learn the basic concepts (e.g. read the Week 0 Prerequisites materials on ML). We may group participants with similar experience levels.

    Participants intending to pursue the Governance Track should consider reviewing machine learning basics, but this is not required.

  • We expect to accept 25-50% of applicants

  • Meetings will be hosted in our office in Cambridge. For the first 4 weeks, each weekly session lasts for 2 hours and consists of a mixture of reading and discussion on the materials. The final 2 weeks will reuqire attending workshops that may be hosted at differing times. For either cohort meetings or workshops, no preparation is required, as all reading will be done in-session. Your facilitator will guide the conversation and answer questions. Dinner provided.

    Accepted applicants will be asked for their time availability and assigned to a cohort time accordingly.

  • CAISH members with research experience in AI safety, usually a mix of undergrad finalists, Master’s and PhDs. If you are a PhD/postdoc, you will most likely be facilitated by a PhD working on direct AI safety research.

  • Yes! We’ve had a mix of undergrads, Master’s, PhDs, and postdocs. We will likely group people with similar backgrounds and experience levels.

  • Please contact hello@cambridgeaisafety.org to discuss other ways of getting involved in AI safety through CAISH! We have more advanced machine learning programs for those who’ve done the Intro fellowship.

Previous participants have worked at...