Skip to content
Alignment Ecosystem Development
Share
Explore
This is a table of projects to improve the alignment ecosystem which could benefit from volunteer support.
Search
Help wanted?
Type
Status
Name
Help wanted?
Description
Chat
Code
Status
Type
Stack
Manager
Team
Contact
1
Yes
will soon be rebuilt as an entry point to AI Safety. The main skillset needed right now is design.
Open
Unscoped
AED Partnered
TBD
plex#1874, Søren Elverlin #9083
Yes
A living document listing AI Safety training programs, with dates and applications. Looking for someone to add self-study options.
Open
Launched
AED
Plex
Plex
JJ
plex
3
Yes
stampy.ai / is a single point of access for learning about AGI safety created by Rob Miles's volunteer team. Accessibility testing volunteer wanted!
Open
Launched
AED Partnered
React/Remix (UI), Python (Discord bot), Coda + Google Docs (edit interface)
Plex
TJ Porter
Plex
Sudo
Aprillion
ccstan99
Augustus
etc
Rob Miles
plex
4
AI Safety Net
Yes

Open
Paused
Unknown
plex#1874
5
Mobility Fund
Yes
A fund to help people in lower and middle income countries travel to connect with core members of EA and AI Safety.
Open
AED Partnered
HP#2176
6
FAST - Frugal Ai Safety Talent
Yes

Open
Under development
Unknown
Tutor Vals #6237
7
Research for Rob Miles videos
Yes

Open
Soft Launched
AED Partnered
W.#7124
Yes
Ryan Kidd:
Could someone make a launch page specifically for these entities, plus any I've missed, complete with small descriptions of their cruxes and aims?
Alignment research employers: , , ,
,
,
, , , , ,
, , , , ;
Academic alignment research hubs: , , , ;
Independent research funders: .
Also would be sweet to have a feed of the and the .
I have a very specific vision:
As professional-looking as the and the ;
Targeted to people who specifically want to get employed/funded post MATS + Refine + ARENA + REMIX + MLAB + WMLB + PIBBSS (i.e., doesn't include random Discords or Slacks or programs graduates have exceeded);
Curated and up-to-date.
Ideally, this would happen by the first week of Feb (to capture MATS 3.0 alumni).
Open
Under development
AED
softr+airtable
Plex
Plex
nemo
ryan kidd
hello@plex.ventures
Yes
EigenTrust is a mechanism for scaling trust by allowing individuals to leverage their network's combined experience. First, peer vetting of alignment research contributions at scale. Then, the world!
Open
Soft Launched
AED
Python (server and bot), Remix/TypeScript (website)
TJ Porter
TJ Porter
Sudo
TJ
10
Yes
Databases of research and project ideas towards alignment, with features for fostering collaboration and connecting teams.
Open
Launched
Independent
11
Yes
EleutherAI is a grassroots collective of researchers working to open source AI research projects. They have many active alignment channels.
Open
Launched
Independent
12
Forum Magnum
Unknown
The open-source codebase that the AI Alignment Forum, Effective Altruism Forum, and LessWrong run on, supporting alignment discussion.
Open
Launched
Independent
13
Unified AI Safety Feed
Yes
A feed for AI Safety content, personalized to optimize for intellectual growth. Should use sources from the alignment dataset.
Open
Unscoped
AED
TBD
plex
Maybe
Some approaches to solving alignment go through teaching ML systems about alignment and getting research assistance from them. Training ML systems needs data, but we might not have enough alignment research to sufficiently fine tune our models, and we might miss out on many concepts which have not been written up. Furthermore, training on the final outputs (AF posts, papers, etc) might be less good at capturing the thought processes which go into hashing out an idea or poking holes in proposals which would be the most useful for a research assistant to be skilled at.
It might be significantly beneficial to capture many of the conversations between researchers, and use them to expand our dataset of alignment content to train models on. Additionally, some researchers may be fine with having their some of their conversations available to the public, in case people want to do a deep dive into their models and research approaches.
The two parts of the system which I'm currently imagining addressing this are:
An email where audio files can be sent, automatically run through , and added to the .
Clear instructions for setting up a tool which captures audio from calls automatically (either a general tool or platform-specific advice), and makes it as easy as possible to send the right calls to the dataset platform.
Open
Paused
AED
Luca
plex, michael trazzi would be a good person to talk to (he's already hiring ppl to edit otter docs), and daniel filan
15
AI Safety PhD cohorts
Yes
The idea is to collect and connect everyone interested in AIS who are applying for a PhD each year.

Find all aspiring AIS researchers who wants to start a PhD in 2024. Put them in the same slack or discord. Organize the effort to look in to different programs and share information with each other. Those who wants to can coordinate where they apply to end up at the same university, and thus be sure to have at least one other AI Safety researcher around. E.g: find all aspiring AIS researchers who wants to start a PhD in 2024. Put them in the same slack or discord. Organize the effort to look in to different programs and share information with each otherer. Those who wants to can coordinate where they apply to end up at the same university, and thus be sure to have at least one other AI Safety researcher around. We have and a with logo ready for this.
Open
Unscoped
AED
Discord, Carrd
16
Yes

Open
Launched
Independent
Gather Town
Sasha
EAGather Team
Contact info
Yes
A living document listing AI Safety communities.
Open
Launched
AED
18
AlignmentSearch
Yes
Conversational agent informed by the Alignment Research Dataset.
Open
Soft Launched
Independent
Python + Flask + Node.js + OpenAI embeddings/ChatGPT.
BionicD0LPH1N#5326
19
Enhancing AI safety ecosystem via debates
Yes
Our epistemics and mutual understanding could be improved with regular debates/adversarial collaborations between alignment researchers who disagree on particular topics. I think about something similar to but in audio (+video) format and open to people’s suggestions about who should we “pitch against” whom and what topic we’d like them to discuss. Spencer Greenberg’s podcast has several episodes that can serve as an example:
.
Open
Unscoped
AED
bagginsmatthew@gmail.com
Yes
Reading What We Can is a web page which has reading challenges designed to upskill people rapidly on alignment topics.
Open
Launched
Independent
Maybe

Open
Launched
AED Partnered
Airtable / Softr
Plex
Luca
22
AGI Safety Fundamentals Video Playlist
Yes

Open
Paused
AED
YouTube Playlist
Plex
ccstan99
plex, ccstan99
23
Maybe

Open
Under development
AED
24
AI Safety Homepage
Unknown
A homepage for AI safety, linking out to the relevant parts of the ecosystem.
Open
Paused
Independent
25
Alignment Project Factory
No

Open
Paused
Independent
26
List of existing research teams
Yes
I imagine a spreadsheet, similar to this one. Each entry is a research team.
Some columns:
Start date (when the team was formed)
End date (e.g. AISC teams has an end date, but for others this will be empty)
Open to join (Options: Yes, No, Maybe)
How to get in touch (This is a text box with instruction for how to contact and/or join the team. E.g. “to collaborate, please join our discord [link]”.)
Research (what the team is doing)
Affiliation (e.g: AISC, SERI MATS, PIBBSS, Independent, Mixed, etc.)
Current number of members (can be just one, but only if they are open for people to join, otherwise it’s not a team)
Why is this useful?
To find collaborators
For inspiration for more people to start teams. Teams are great!
Don’t add teams with out their permission. This should be a list of teams that wants the visibility and are happy to be contacted.
If this is created it needs to be maintained. I don’t think this will be much work, but it will be necessary.
Suggestion created by Linda Linsefors ()
I will not lead this project. Don’t reach out to me to offer to help unless you are also willing to take ownership. However if you are willing to take ownership, then I can help you.
Open
Unscoped
Unknown
linda.linsefors@gmail.com
27
Maybe
Needs a community manager.
Open
Paused
AED Partnered
Wordpress plugins ( & ), community management
28
Test

Open
There are no rows in this table

Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.