Centre for Research on Impact Evaluation (CRIE) 

The Community of Practice on Counterfactual Impact Evaluation of ESF interventions (CoP-CIE-ESF) was launched by CRIE and DG Employment, Social Affairs and Inclusion (DG EMPL) in 2016. This initiative aims to foster the collaboration between DG EMPL, CRIE, the ESF Managing Authorities and other stakeholders to conduct CIE of ESF funded interventions by sharing experiences, knowledge and promoting joint activities.

• The 2024 COP-CIE-ESF took place in Sofia on 12 June 2024. See the meeting agenda, and the flash report.  

  The meeting was structure in four parts:  

  1. Keynote speech by Rafael de Hoyos from the World Bank on “Increasing the Effectiveness of Social Policies in the EU”. 
  2. Managing authorities’ presentations & group discussions on promoting CIE within the evaluation governance framework of Member States. 
  3. Managing authorities’ presentations sharing relevant evaluation experiences. 
  4. Two case studies on Counterfactual Impact Evaluation planning using real ESF-funded policies. 
      

Here is the list of the COP-CIE-ESF meetings held in the past.

• The main focus of the 2023 COP-CIE-ESF meeting was to respond to the request from the members of the Community of Practice to finalize the evaluation plans of ESF+ and to discuss how to best prepare for evaluations in areas different from active labour market policy, such as interventions in the fields of health, education, poverty and social inclusion.
  The meeting took place in Dubrovnik on 25 May 2023. See the meeting agenda, and the flash report 

  The meeting was structured in four main parts:  

  1. Keynote speech by Martina Pezer on the use of microsimulation models for planning ESF interventions
  2. Managing authorities’ presentations & group discussion on evaluation plans 
  3. Managing authorities’ presentations sharing relevant evaluation experiences 
  4. Managing authorities’ presentations & group discussion on evaluation in the field of education, health, poverty and social inclusion. 
      

• The 2022 COP-CIE-ESF meeting was held in Brussels & online on May and 18th. See the meeting agenda and the flash report

  The agenda included:

  • Keynote speech by Professor Jochen Kluve (Head of the Evaluation Unit at KfW Development Bank and Professor of Economics at the School of Business and Economics, Humboldt-Universität zu Berlin): “Recent developments in CIE in Europe”
  • Small group discussions on the concluded evaluations of the past programming period, focusing on:
    1. how to assess the quality of evaluation reports,
    2. how to communicate results of concluded evaluation
    3. how MAs intend to prepare the report summarizing the findings of evaluations, due by end of 2022 
        
  • Small group discussion on the current programming periods possible concrete actions that could be done in preparation of the evaluation plans and evaluations that will have to be carried out in the next years, focusing on :
    1. Choice of what to evaluate and what to evaluate using CIE
    2. Evaluation process and design of intervention
    3. Data needs
    4. Contractors 
        

• The 2021 COP-CIE-ESF meeting was held online on June 16th and 18th. See the meeting agenda and the flash report.

  The agenda included:

  • Theme:  which steps are in between the approval of the Operational Programme (OP) and the execution/outsourcing of the evaluation?
  • World café and plenary discussion:
    1. How do you decide what to evaluate?
    2. How do you decide how to evaluate?
    3. What are the conditions that make CIE feasible for given interventions?
    4. how to approach the new actions under ESF plus?
        

• The 2020 COP-CIE-ESF meeting was held online on November 10^th and 12^th. See the meeting agenda and the flash report and a more detailed report, which summarise the main points touched.

  The agenda included:

  • Theme: experiences made by the Managing Authorities (MAs) in the past 7 years to carry out CIE of ESF funded interventions.
  • World café and Plenary discussion on: (i) selection of interventions and methods; (ii) data; (iii) evaluation process; (iv) contractors; and (v) communication of results and engagement of policy makers.
     

• 2019 COP-CIE- ESF meeting - Dublin, Ireland on 7 June 2019, hosted by the Irish Managing Authority of the ESF and EGF Policy and Operations Unit in the Department of Education and Skills and the Department of Employment Affairs and Social Protection. See the meeting agenda and the flash report.

  The agenda included:

  • Theme: data provision for evaluation
  • World Café: interactive working tables on data requirements for CIE
  • Keynote lecture by Prof. Johan Vikström, Uppsala University and Institute for Evaluation of Labour Market and Education Policy (IFAU)
  • Presentations of current evaluation practices in Member States
     

• 2018 CoP-CIE-ESF meeting - Athens, Greece, 14 June 2018
  hosted by the Evaluation Unit, Greek Ministry of Economy and Development, General Secretariat of NSRF, National Coordination Authority

  The agenda included:

  • Theme: how to design ESF interventions and programmes in light of future CIE
  • Policy design lab: interactive working tables
  • Keynote lecture by Professor Enrico Rettore, University of Trento and Research Institute for the Evaluation of Public Policies (IRVAPP)
  • Presentations of Data Fitness Initiative for CIE (D4C) 2017 and 2018 projects, and other CIE conducted in several Member States
     

• 2017 Autumn Meeting of the CoP-CIE-ESF - Brussels, Belgium, 8 December 2017

  The agenda included:

  • Theme: how to combine different data sources
  • Presentations of Data Fitness Initiative for CIE (D4C) 2016 and 2017 projects
  • Training sessions on evaluation in practice with computer-based examples from D4C projects
     

• 2017 Spring Meeting of the CoP-CIE-ESF - Riga, Latvia, 8-9 June 2017
  hosted by the Evaluation Unit of the Ministry of Finance of Latvia

  The agenda included:

  • Theme: how to combine CIE with other evaluation methods
  • Keynote lectures by Professor Lena Tsipouri, University of Athens and Professor Claudio Lucifora, Catholic University of Milan
  • Presentations of Data Fitness Initiative for CIE (D4C) 2016 and 2017 projects
  • Training sessions on evaluation in practice (Theory Based and CIE methods)
     

• First Meeting of the CoP-CIE-ESF - Ispra, Italy, 9-10 June 2016

  The agenda included:

  • Launch of a Community of Practice on CIE
  • Keynote lecture by Professor Jochen Kluve, Humboldt-Universität zu Berlin
  • Presentations on administrative data and data linkage
  • Presentations on Youth Employment Initiative (YEI) evaluation