Skip to main content
Knowledge4Policy
Knowledge for policy
Supporting policy with scientific evidence

We mobilise people and resources to create, curate, make sense of and use knowledge to inform policymaking across Europe.

  • Blog post | Last updated: 08 Jul 2022
Can and should science-for-policy systems be mapped and measured?

The Joint Research Centre, engaged on a long-term project on how to support evidence use in policymaking across the EU, invited me to spend some time thinking about the role of the science system, and how we might know what a ‘good’ science system looks like. Partly this is about mapping science systems and trying to understand what makes them work; and partly about using indicators to find out how they are functioning.

Kathryn Oliver Kathryn Oliver Evidence-Informed Policy Making
A flow chart showing the science advice system in the UK

There are two main problems with setting up an indicator dashboard to assess science systems. Firstly, some relatively straightforward technical issues: We often measure what is measurable, which isn’t necessarily the same as what is useful to know; and the potential for perverse incentives, or even just over-attention to what is measurable, is always a risk.

The second challenge is more existential. Does the science system have a goal? Visualising and mapping science systems perhaps inevitably gives the impression that the system was designed, and so can be controlled and adjusted, and operates towards a shared objective. None of this is true of evidence/policy systems, which tend to evolve organically. And members of this system have their own agendas and interests - not all researchers produce policy-relevant research, for instance. So, no – in itself, the system does not have a clear goal.

However, the JRC project is about supporting evidence use in policy. And the literature on evidence-informed policymaking does have a relatively shared understanding of what this means: deliverying relevant, robust evidence to people with the capacity to absorb and fully understand it in order that it might inform their decision-making. This is not a simple goal - big questions about what evidence is produced, what evidence counts for policy, how evidence and expertise reaches decision-makers, and how they can and do respond to this evidence all need to be considered; and all these questions have political, socio-technical and philosophical aspects.

It is possible, though, to think about which types of organisations and individuals are involved in evidence production, mobilisation and use in decision-making; and to examine their connections and activities. In fact, we can start to think about the total set of actors and connections within the entire system through which scientific knowledge is acquired, synthesised, translated, presented for use, and applied in the policymaking process. This is my working definition of the science-for-policy system.

In the table below, I set out my preliminary thoughts about what these different groups might do in service of evidence-informed policymaking, and how we might know if they’re doing it (well). I am not yet sure, however, how these signs of activity might translate into measurable indicators.

 

Type of Actor

Activities – what might they need to do?

Possible indicators

Funders

 
  • To foster a culture of professionalisation in evidence production and mobilisation, supporting and promoting skills and careers to engage effectively and ethically
  • To engage with, respond to, and shape governmental knowledge needs and researcher-led priorities
  • To support and use research into effective knowledge systems
  • Engage regularly and effectively with the research communities and with relevant policy teams  to support policy-relevant research through shared problem-framing
  • Offer appropriate training and capacity building to research organisations and teams, who include support staff
  • Dedicated research and knowledge exchange funding for research into evidence production and use
  • Balance between responsive and policy challenge-led research funding

Research organisations
  • To support a diverse workforce equipped with skills and expertise relevant to policy challenges through relevant training, and capacity building opportunities
  • To reward and enable research engagement with policy and practice organisations
  • Support diverse and skilled research workforce by offering appropriate degree and other training and capacity-building support
  • Reward and recognise policy-relevant research and engagement activities to policy
  • Supporting a diverse workforce with capabilities in knowledge exchange with policy

Researchers
  • Produce interesting and novel research evidence relevant to policy challenges
  • To seek to support effective knowledge systems
  • Appropriately skilled and diverse workforce
  • Produce interesting and novel research, or relevant syntheses
  • Routinely engage with evidence users as a legitimate part of their work

Intermediaries
  • To support effective functioning of the system through providing convening and network opportunities
  • Synthesise and mobilise evidence in range of ways
  • Have skilled workforce able to synthesise and mobilise evidence in range of ways, network, and support and advocate for evidence use
  • Recognised and resourced specialist units or organisations dedicated to knowledge exchange
  • Support shared problem-framing and deliberation between policymakers and stakeholders

Policymakers
  • Internal capacity and capability to engage with shared problem framing, identify and seek out a range of diverse evidence types, and able to assess and comprehend evidence findings and implications
  • To state clearly their knowledge needs
  • Have clear and transparent evidence-informed  mechanisms to seek diverse and appropriate evidence
  • Clearly stated knowledge needs
  • Capabilities within government to assess and absorb evidence of different kinds
  • Transparent mechanisms to solicit and engage with evidence and experts
  • Internal reflection and scrutiny of advisory systems

Parliament, media and other scrutiny bodies
  • To provide effective scrutiny of effective systems functioning, include gaps between knowledge needs and produced research evidence, workforce diversity, and assessment systems
  • To enable government to identify what it does not yet know it needs to know
  • Existence of, and mandate for independent, evidence-capable body(ies) to scrutinise science-for-policy system and its elements, and hold to account
  • Science and evidence capabilities for general scrutiny of evidence use in policy-making at national, regional and local levels

Although there are probably some missing groups, activities, and indicators, mapping these out shows at least three interesting things:

  1. Whether the groups and organisations involved agree on the goal of evidence-informed policymaking, or want to act in its service, all groups currently contribute towards this goal
  2. Mapping out the connections in this way offers potential points of intervention which could help improve the system's functioning
  3. However, interventions seeking to improve evidence use usually target one element or relationship, rather than at a systems level. This tends to add to a chaotic mass of activity, rather than improving systems functioning.

More thinking on these activities, goals, and indicators is needed, particularly around what we want our science-for-policy systems to do, for whom, and how.