This material may help you:

1. videotutorial on 'Tips and tricks on how to plan intervention design';
2. case studies discussed at the 2018 Community of Practice on Counterfactual Impact Evaluation of ESF interventions (CoP-CIE-ESF), 14 June 2018.

This material may help you:

1. videotutorial on 'Data collection, Data access and Data merging'.

This material may help you:

1. video on administrative data for evaluation: 'What is administrative data?';
2. guidance document on 'How to use administrative data in ESF evaluation';
3. checklist on 'How to use administrative data in ESF evaluation';
4. book on microdata for policy evaluation: 'Data-Driven Policy Impact Evaluation - How Access to Microdata is Transforming Policy Design' freely available.

This material may help you:

1. Sample Size webtool to compute the sample size required for ESF indicators;
2. Sample Size videotutorial explaining how to use the webtool.

If you are planning to compute these indicators using administrative registries, please refer to the 'Data Preparation' section.

Here you can find supporting material to draft a Terms of Reference (ToR) document in order to outsource a CIE study:

1. guideline on how to write a ToR in view of a CIE;
2. video tutorial on how to write a ToR;
3. checklist document to ensure all necessary elements are present in the ToR document and to review its overall quality;
4. standard template with the requested information for each section. 

Finding the right method to perform a CIE depends on how the intervention was designed, and on the data available to evaluators. The following material may help with this step:

1. videotutorial on how to choose the method, based on intervention design: Tips and tricks on how to plan intervention design
2. videotutorial on how to choose the method, based on available data: Data collection, Data access and Data merging.

Here you can find material on various methods for performing a CIE study.

1. An overview of all available methods

• guidance document Design and commissioning of Counterfactual Impact Evaluations (available in 23 languages)

2. Propensity Score Matching (PSM)

• video How to evaluate policy;
• example from the Data Fitness Initiative for CIE - report Counterfactual Impact Evaluation of Work Experience Laureati e Laureate – WELL (Work Experience for Graduates) (see section 5).

3. Difference-in-Difference (DiD)

• video Difference-in-Differences method for policy evaluation.

4. Regression Discontinuity Design (RDD)

• video Regression Discontinuity Design for policy evaluation;
• example from the Data Fitness Initiative for CIE - report Vocational training and labour market outcomes: Evidence from Youth Guarantee in Latvia (see section 6).

5. More advanced CIE methods: Synthetic Control, Sequence Analysis, Dynamic Matching

• Advanced counterfactual evaluation methods - Guidance document.

6. Counterfactual Evaluation Archive (CEA)

• CEA is an online database which classifies published articles and working papers using counterfactual impact evaluation methods to assess the impact of active labour market policies within EU28. Use filters to search for studies regarding the country, the CIE method or the group you are interested in.

After the evaluation is completed, it is important to communicate the relevant findings to the stakeholders involved. Here you can find material to help you with this stage.

1. video tutorial 'Communicating the Results of Counterfactual Impact Evaluations', to learn how to communicate results to policy makers;
2. CoP - CIE - ESF meetings to share your experience with other Managing Authorities.

The Counterfactual Evaluation Archive (CEA) classifies published articles and working papers using counterfactual impact evaluation methods to assess the impact of active labour market policies within EU-28. Use filters to search for studies regarding the country, the CIE method or the group you are interested in.

Here you can find examples of evaluations of ESF-funded interventions, carried out by CRIE in collaboration with the ESF Managing Authorities in the context of the Data Fitness Initiative for CIE:

1. Counterfactual Impact Evaluation of “Work Experience Laureati e Laureate – WELL” (Work Experience for Graduates) illustrating the evaluation of a regional on-the-job training programme implemented in Italy;
2. Vocational training and labour market outcomes: Evidence from Youth Guarantee in Latvia illustrating the evaluation of a vocational training programme implemented in Latvia within the Youth Guarantee scheme;
3. Active Labour Market Policies in Flanders. Evaluation of the ESF “Work Experience for Young Persons” programme, illustrating the evaluation of an ALMP implemented in Flanders;
4. The evaluation of the Youth Employment Initiative in Portugal using Counterfactual Impact Evaluation methods.
5. Counterfactual Impact Evaluation (CIE) of the "JobPlus" programme implemented in Ireland in 2013, in collaboration with the Department of Employment Affairs and Social Protection of Ireland.
6. Evaluation of the Support to Schools Project in Czech Republic
7. Evaluation of the higher education grant system for less privileged students in Portugal
8. Meta-analysis of the ESF counterfactual impact evaluations

Here you can find a collection of examples of ESF-funded intervention evaluations:

1. Guidance document on ESF evaluation examples.