How we will evaluate our impact

· 2 min read
By

Context

Given the difficulty of measuring effects on policy networks, we anchor our strategy in a detailed theory of change. While organizational long-term adaptability and strategic capacity will be most indicative of the value of our work,1 we highly value early indicators of the counterfactual impact of our work.

How we collect data

We gather data on counterfactual scenarios by reviewing projects pre- and post-execution and including control groups in experiments wherever possible. We regularly conduct research interviews to better understand our audience and send out an annual impact survey to everyone we have interacted with. For our policy support activities, we strive to conduct pre-engagement surveys on top of the routine post-engagement surveys.

Key metrics

Policy support

  1. change in processes and tools used and
  2. change in the understanding of long-term issues.

Field building

  1. change in political engagement tactics and
  2. change of research focus.

Research

Number of peer-reviewed academic publications.

First signs of impact

As a result of our research leading up to the conception of SI, we already have:

  • Advised the European Commission’s Joint Research Centre on evidence-based policymaking
  • Advised a Czech think tank on prioritization dynamics in policy networks
  • Substantially contributed to a workshop on decision-making support tools for Dutch civil servants
  • Advised a Norwegian think tank on strategy and the state of the art in policy theory
  • Contributed to the strategy of a Swiss institute bridging academia and international organizations
  • Advised dozens of future policy practitioners and researchers on career strategy and priority areas

Footnotes

1 Teles, S., & Schmitt, M. (2011). The elusive craft of evaluating advocacy. Stanford Social Innovation Review, 9(3), 40-43.