2-year review: concluding SI's inception

· 19 min read
By , , and

Overview

In April 2021, we launched the Simon Institute for Longterm Governance (SI). In this post, we provide a review of SI’s work since inception. We outline:

  • Why we founded SI 
  • Our approach to impact
  • Our first two years in numbers
  • Our impact so far 
  • Our plans moving forward

Why We Founded SI

The Simon Institute for Longterm Governance (SI) was founded in April 2021, with the goal of addressing critical gaps in the multilateral risk governance system, and integrating the interests of future generations into policymaking processes. From the beginning, SI saw the need for a multilateral system capable of adapting to an increasingly interconnected and technology-driven world, where new risks and opportunities were emerging. In particular, advances in artificial intelligence and biotechnology were increasing the prospects of drastic technological breakthroughs and risks which the multilateral system was not equipped to govern. Combined with prevailing short-term thinking amongst policymakers, a fresh approach to multilateral risk governance was needed.

SI was named after Herbert Simon, an American polymath and political scientist renowned for his work in decision making, organizational theory, and complex systems. Herbert Simon foresaw the increasing role that technology and artificial intelligence would play in our daily lives, and understood that technology’s potential benefits and harms heavily rely on the efforts we make today. Furthermore, he emphasized the importance of designing, and not just predicting, a sustainable future. In his own words, 

“I don’t believe that predicting the future is really what we’re about. After all, we ourselves, or at least the younger ones among us, are going to be a part of the future. So being a part of it, our task is not really to predict it. Our task is to design it, to design a sustainable, and acceptable world. And then to devote our efforts to bringing that future about now.”

In its first two years, SI focused on fostering long-term governance; governance that includes future generations’ interests in policymaking processes. In practice, this involves advocating for institutions that simultaneously promote development, mitigate risks of rapid technological progress, and consider the long-term implications of policies enacted today. To date, SI has contributed to the UN Our Common Agenda’s five most important processes for reducing existential risk, developed a future-proofing framework to inform the High-Level Advisory Board on Effective Multilateralism, and delivered the first and most extensive United Nations (UN) report on existential risk and rapid technological change. Additionally, SI has published research papers, interview analyses, frameworks, training exercises and table-top exercises to equip policymakers and multilateral actors with insights and tools for better understanding risk, uncertainty, and long-term thinking.  

For its first 13 months, SI operated without full time staff, with its first full time team only starting in June 2022. Despite this short time-frame, SI’s early successes in fostering long-term governance provide solid proof of concept and evidence of impact. 

Our Approach

At its core, SI believes that policymaking at the national and multilateral level is key for effective risk governance and for safeguarding the interests of future generations. Governments can encourage responsible technological development, implement regulations, and enact preventative measures for curtailing risks. Multilateral fora, such as the UN, play a crucial role in managing global risks coherently across jurisdictions by offering platforms for dialogue, negotiation, and information sharing. As such, the multilateral development of governance structures capable of effectively managing risks, particularly those from technological progress, is vital.

While affecting change at the multilateral level is a lengthy and complex endeavor, SI concentrates its efforts on informing key political processes. SI generates resources, produces expert briefings, and disseminates information outlining risks associated with rapid technological change, concrete solutions for reducing risks, and strategies for incorporating the interests of future generations in policymaking processes. Through presentations and focused meetings, SI also ensures that key stakeholders are well-informed, and are able to contextualize specialized knowledge. 

SI also works to bridge the gap between researchers studying technological and existential risks and policymakers working on governing these risks. SI facilitates workshops, meetings, and events that offer a platform for exchange, collaboration, and knowledge sharing; crucially helping to bridge gaps between theory and practice. Finally, SI supports the education of policymakers, facilitating research and exercises around decision-making under uncertainty, prioritization training, and effective group deliberation. These activities aim to equip policymakers with the skills and tools needed to handle complex governance issues in real time more effectively. 

Initially, SI set out to inform agenda-setting processes with evidence and foresight exercises, while developing better institutional designs. Luckily, and to SI’s surprise, the political discourse prioritized risks from rapid technological change much faster than expected. Welcoming these shifts in attention, SI has increasingly pivoted to supplying subject matter expertise on risk governance, artificial intelligence (AI) , and biotechnology. Going forward, SI will continue to adapt its activities in light of the multilateral system’s biggest needs.

Our First Two Years in Numbers

In brief, SI’s first two years have been marked by: 

  • 39 publications and submissions
  • 44 events, workshops and talks organized, with a total of  800+ attendees
  • 10 public statements at UN conferences and roundtables 
  • ~1500 meetings with external actors from the UN system, academia, civil society, private sector, and government 
  • 95 calls with talented young professionals on careers in long-term governance
  • 4 full time employees on-boarded and 3 additional full time employees signed on
  • $1.5MM funding raised

Our Impact so Far

Below, we outline five key impact stories so far, along with 19 counterfactual impact points –  instances where we believe SI’s contributions have made a significant difference. While measuring impact in the realm of policymaking can be difficult, we concentrate on notable instances of information exchange, resource production, and coordination, in order to demonstrate the tangible effect of our work. 

SI is increasing the chances of a successful UN “Summit for the Future” 

In 2021, in response to a wave of global shocks, including Covid-19 and the growing climate crisis, the UN Secretary General issued Our Common Agenda (OCA) – a blueprint for renewed international cooperation. As part of OCA, the Secretary General announced plans for an inaugural “Summit for the Future” for September 2024, with the goal of strengthening the Sustainable Development Goals, and revitalizing the multilateral system. The summit is also expected to result in the adoption of an action-oriented “Pact for the Future”, detailing practical measures to enshrine and promote the rights of future generations. 

In the lead up to this Summit, SI has been working to bring key scientific and policy stakeholders together. Coordination plays a central role in ensuring that key stakeholders are included in summit-related discussions, practical and theoretical knowledge gaps on scientific and policy issues are filled, and actors are aligned on key matters. Through a series of workshops, meetings, and events, SI has been able to bring together key scientific and policy stakeholders; often resulting in joint contributions to multilateral processes, invitations to write briefs, or new speaking opportunities:

  • In October 2021, SI co-organized a two-day workshop with the Centre for the Study of Existential Risk (CSER) to inform policymakers and existential risk researchers on science-policy interfaces for risk reduction and OCA processes. 
  • In September 2022, SI held a two-day workshop with 25 researchers and funders to inform them about UN processes, and coordinate OCA activities. This led to coordinated contributions to the Global Digital Compact and Global Risk and Opportunities Report. 
  • In October and November 2022, SI went to New York to meet key actors working towards the Summit of the Future. SI met with the UN Foundation, the permanent missions of Sweden, Denmark, Luxembourg and the Netherlands, the Executive Office of the Secretary General, the UN Office for Disaster Risk Reduction, the UNU Centre for Policy Research, the UN Crisis Response Group, and the Office of the Tech Envoy.
  • In January 2023, SI organized a two-day workshop with 28 existential risk researchers and UN actors to coordinate efforts around rapid technological change governance across the disarmament, disaster risk reduction, health and development domains. As a result: 
  • The University of Oxford initiated an alliance for future generations;
  • The right hand of the co-facilitator of the Global Digital Compact (GDC) prioritized understanding and making progress on AI governance via the GDC; 
  • The UN Foundation and Executive Office of the Secretary General (EOSG) invited SI to give a workshop to Member State delegations on future proofing for rapid tech change governance
  • SI was provided with an opportunity to respond to EOSG policy briefs;
  • EOSG invited one of the workshop participants, Cecil Abungu, to make an expert statement in front of the UN General Assembly. 
  • In March 2023, SI organized a 1.5 day workshop with the UN Foundation and the International Chamber of Commerce to develop effective messages emphasizing the significance of technological development for future generations at the 2024 Summit of the Future. Participants were also invited to discuss the future of multilateralism with Micheal Møller, the 12th director-general of the United Nations Office at Geneva.

Without SI’s contributions… 

  1. Key international organizations, like UNDRR, would likely not have explicitly stated an interest in focusing on global catastrophic risks. 
  2. Existential risk researchers would engage less frequently and effectively with the UN.
  3. SI would have less bandwidth with UN actors – with in-person meetings and workshops helping to build mutual understanding and exchange knowledge. 
  4. UN actors would be less informed about risks from AI and biotechnology. 
  5. Key actors informing the Summit of the Future would work independently rather than collaboratively towards shared goals. 

Alongside the coordination of key actors ahead of the Summit, SI has also been working to inform key documents and processes leading up to the “Pact for the Future”. If successful, this pact could serve as a key turning point in the multilateral system, laying groundwork for international strategies to safeguard future generations and effectively govern rapid technological change. If unsuccessful, key framings or recommendations on existential risk and future generations might become diluted, politicized, or misused. With expertise in both of these domains, SI has been working to positively shape this Pact.

  • SI informed the forthcoming UN Declaration for Future Generations via a written submission and a statement; facilitating academic input; as well as direct support of the co-facilitators leading the process. This led to a highly aligned Elements Paper, offering guidance for the drafting of the Declaration, with a focus on unborn generations, existential risk and emerging technology. 
  • SI contributed to the report of the High-Level Advisory Board on Effective Multilateralism (HLAB) by conducting a series of future-proofing exercises for them and supplying state of the art references on AI development and governance.
  • SI contributed to the development of the Global Digital Compact through a submission on foundation models, direct exchange with the Tech Envoy and his Office, as well as a joint submission with the Future of Life Institute.

Without SI’s contributions… 

  1. The Elements Paper for the UN Declaration for Future Generations would focus mostly on youth, excluding unborn generations, and would not mention existential risk or emerging technologies.  
  2. Likely fewer mentions of existential risks and rapid technological change would have made it into the HLAB report, as direct contact with the Board was necessary. 
  3. It is likely that no other submissions to the Global Digital Compact would have emphasized the importance of foundation model governance.

Going forward, SI plans to continue engaging with processes stemming from Our Common Agenda, and positively shaping the Summit for the Future and the resulting Pact for the Future. This engagement will center on informing key UN processes such as the Futures Lab, Declaration for Future Generations, Envoy for Future Generations, Global Risk Report, Global Digital Compact, among others. Moreover, SI aims to pursue a balanced strategy, contributing to these processes not only by outlining key risks associated with rapid technological change but also fostering positive narratives about the future and technology. 

SI informed the mid-term review of the Sendai Framework for Disaster Risk Reduction

The Sendai Framework for Disaster Risk Reduction (2015-2030) is the sole international framework dedicated to coordinating efforts around the reduction of risks stemming from all types of hazards. While its scope encompasses technological hazards as well as extreme risks, discussions around Sendai tend to focus on natural hazards. Throughout 2022 and early 2023, in light of changing risk landscapes, the framework was under review. This review period provided a unique opportunity to raise awareness about technological and existential risks – two key risks that are commonly left out of discussions when discussing disaster risk reduction. 

With expertise in both domains, SI leveraged this review period to produce accessible knowledge about advanced rapid technological change, raise awareness about these issues at the multilateral level, and outline concrete plans for the multilateral system to address these risks effectively:

  • In Spring 2022, SI’s Co-CEO Max Stauffer was invited to join the Swiss working group on disaster risk reduction as a youth delegate. At the Global Platform on Disaster Risk Reduction, Max gave two statements on existential risk and long-term governance. 
  • In Summer 2022, the UNDRR contracted SI to deliver the first ever thematic study on existential risk and rapid technological change. In the study, SI outlines 30 strategies for reducing existential risk, and effectively governing developments in artificial intelligence and biotechnology at multilateral and national levels. 
  • The UNDRR invited SI to give two presentations on risks from AI and emerging technological risks to their private sector and member state networks in New York. 
  • The Sendai Framework’s mid-term review report cited SI’s thematic study six times. 
  • The zero draft of the political declaration of the mid-term review mentions: “We recognize also that greater attention is needed in all countries to strengthen understanding of new, emerging, and future risks, including the long-term impacts of climate change and the adverse impacts of rapid technological change and artificial intelligence.”
  • In May 2023, SI discussed risks from rapid technological change at a High-Level Meeting on Disaster Risk Reduction in New York, participating in a panel on foresight and frontier risk, and a leaders’ roundtable with ministers and representatives from Microsoft and OpenAI. The summary presented to the UN General Assembly contains multiple references and quotes that highlight SI’s contributions (p. 13ff).

Without SI’s contributions…

  1. The UNDRR would focus less on existential risks and risks from artificial intelligence and biotechnology, and more on natural hazards and other forms of digital transformation.  
  2. The Sendai Framework’s mid-term review report and zero draft of the political declaration would likely not emphasize risks from rapid technological change and artificial intelligence. 
  3. The mid-term review and declaration would lack an explicit mention of existential risk and rapid technological change. 
  4. The high-level meeting would likely not have any panelists addressing existential risk and rapid technological change. 

While the Sendai review period is now over, SI remains committed to raising awareness about existential risks and risks from rapid technological change in both the disaster risk reduction community and the broader multilateral network. SI aims to do so by providing technical expertise and process design input to policy actors, empowering Low and Middle Income Countries to play a greater role in technology governance, and acting as a bridge between policy and science networks to foster information exchange. SI also hopes to provide specialized technical briefings to member state actors, in order to ensure the accurate transmission of information about the risks and benefits of advanced technologies, with a particular focus on AI. 

SI increased awareness about engineered pandemics around the Biological Weapons Convention 

The Biological Weapons Convention (BWC) is the most important international instrument to uphold norms of non-development and non-proliferation of biological weapons, and remains a key forum to discuss risks from pandemics and the misuse of biotechnology at the multilateral level. During the 2021 Meeting of States Parties and the 2022 9th Review Conference of the Biological Weapons Convention, SI coordinated several dinners, events and engagements around the theme of engineered pandemics as the top international biosecurity priority – a point that is often forgotten in the breadth of discussions surrounding the BWC. These events aimed to increase awareness of solutions for engineered pandemics and facilitate interactions between biosecurity researchers and policymakers. 

  • SI organized a total of 11 events focusing on engineered pandemics and other global catastrophic biological risks (GCBRs), and sponsored 20 young researchers’ participation in DiploFoundation’s BWC-crash course. SI welcomed delegations from expert organizations, such as Cambridge and Oxford, the Open Philanthropy foundation, the iGEM foundation, and the Nuclear Threat Initiative (NTI), providing physical and online spaces for networking; as well as personal introductions to biosecurity experts and diplomats. This led to many new connections between guests and seasoned experts and deepened understanding of relevant diplomatic dynamics key to making progress on biosecurity. 
  • SI provided biosecurity researchers at the University of Oxford with practical insights on the multilateral system and organized three dinners to connect them to researchers, young diplomats and senior biosecurity leadership. A prominent grantmaker in the field of biosecurity notably valued one of these dinners at $50,000. 

Without SI’s contributions… 

  1. No in-person side events would have raised awareness for Global Catastrophic Biological Risks and facilitated crucial knowledge exchange and networking.
  2. Academics and grantmakers would be less informed about international diplomacy, and have fewer connections in the field – particularly from the World Health Organization (WHO), the United Nations Institute for Disarmament Research (UNIDIR), the US, Germany, and civil society. 
  3. Visiting expert delegations to the BWC Review Conference would have had a harder time getting oriented and meeting fewer multilateral biosecurity policy veterans.

Going forward, SI plans to reduce its focus on biosecurity, in order to consolidate activities in other domains. SI hopes to transfer its existing contacts and accumulated knowledge in this field to other organizations working towards the safe governance of advanced biotechnologies, for example, to the newly launched International Biosecurity and Biosafety Initiative for Science (IBBIS). Moreover, SI remains open to adapting its approach and strategies according to the shifting needs of the global governance landscape.

SI contributed to building the field of long-term governance

Over the past two years, the SI team has dedicated a large portion of its work towards building the academic field of long-term governance. In practice, this involved researching current risk governance and disaster response practices among policymakers, training these policymakers in decision-making amidst uncertainty, and producing new knowledge on emerging technological risks. SI’s contributions to the field include developing foundational knowledge and tools, publishing research, and offering career, strategy, and grant-making advice to individuals, funders, and organizations interested in risk governance and long-term governance. 

  • SI published the FAIR framework to inform the High-Level Advisory Board on Effective Multilateralism and to help institutions design future-proof policies
  • SI published a paper on policy networks and attention dynamics in Complexity. 
  • SI developed the framework of long-term institutional fit to operationalise long-term governance. 
  • SI published an interview analysis of decision-making in the face of uncertainty 
  • SI published a short guide for robust decision-making and an overview of fields to strengthen decision-making. 
  • SI published the first and most extensive UN report on existential risk and rapid technological change. 
  • SI collaborated with the UN Office for Disaster Risk Reduction and the International Science Council on a briefing note and conducted one of the largest expert surveys on identifying hazards with global catastrophic and existential potential. 
  • SI developed policy prioritization training on multi-criteria decision analysis and a table-top exercise on governing extreme pandemics.
  • SI provided research directions and career advice to ~95 talented young professionals, and gave policy-related funding advice to 5 key existential risk grantmakers. 
  • SI was commissioned to recommend future directions for the International Risk Governance Council (IRGC) – a Swiss foundation which established the field of risk governance in the early 2000s. 

Without SI’s work…

  1. The field of long-term governance would have mostly theoretical and academic knowledge foundations.
  2. Existential risk researchers and grantmakers would not have a specialized actor to talk to about long-term governance and multilateralism. 

Looking ahead, SI plans to shift its focus away from foundational work building the field of long-term governance, and contributions to theoretical discourse, towards greater involvement and active support in multilateral processes. 

SI became a professional organization filling an underserved niche

Finally, over the past two years, the starkest progress SI made was to establish itself as a functional and recognized organization. SI went from an abstract project with no office and no funding to an institutionalized, resourced organization with promising signs of value-add to multilateral processes. 

  • SI raised $1.5MM
  • SI hired 3 staff members and finalized the hiring process for 3 other staff members
  • SI secured office space at the Maison de la Paix, and built a local hub through opening up this office to select collaborators. 

Without SI… 

  1. There would be no organization explicitly working on long-term governance in Geneva and in New York, and no organization working at the interface between existential risk research and policy fora.
  2. There would be less hope in the broad network that the multilateral system could succeed at reducing existential risk from rapid technological change.

Our Plans Moving Forward

For its first two years, SI was primarily dedicated to addressing a range of gaps in the multilateral risk governance system. Moving forward, SI plans to shift its focus more specifically towards technology governance at the multilateral level. In particular, SI aims to contribute to the development of a multilateral regime complex for technological progress, designed to benefit present and future generations alike. Rapid advancements in technology, and particularly in artificial intelligence, highlight the need for improved coordination between big tech and policy actors, a greater diversity of voices in tech governance discussions, and a multilateral system that not only keeps up with national regulations, but also plays a leading role in global enforcement and standard setting.

In practice, these plans involve several strategic approaches. First, SI aims to empower Low and Middle-Income Countries to participate in tech governance discussions, working to both understand their unique needs and ensuring they are included in key dialogues. Second, SI aims to support multilateral institutions, Member States, and other policy actors by offering technological and process design expertise through recommendations, expert briefings, and continued contributions to key processes and reports. Lastly, SI hopes to continue serving as a bridge between policy actors and technology experts, and work towards establishing a robust, long-term governance community in Geneva.

If you’re interested in discussing our plans in more detail, collaborating, or supporting our work, please reach out to konrad@simoninstitute.ch.