Expert reflexivity is discussed as a part of the research and thought process. In this blog, Justyna Bandola-Gill, Marlee Tichenor and Sotiria Grek discuss reflexivity as a strategic tool of experts who approach it epistemically, care-fully and instrumentally. As such, this blog considers and encourages debates over the role of values in expert advice.

Producing and making use of data and metrics in policy making have important limitations – from practical issues with missing or incomplete data to political challenges of navigating both the intended and unintended consequences of implementing monitoring and evaluation programmes. But how do experts producing quantified evidence make sense of these challenges and how do they navigate working in imperfect statistical environments? In our recent study, drawing on over 80 interviews with experts working in key International Organisations, we explored these questions by looking at the concept of expert reflexivity.

We soon discovered that experts working with data and statistics approach reflexivity not only as a thought process but also as an important strategic resource they use to work effectively – to negotiate with different actors and their agendas, build consensus and support diverse groups of stakeholders. What is even more important, reflexivity is a complex and multifaceted process and one that is often not discussed explicitly in expert work. We aimed to capture this diversity by categorising experts’ actions and perceptions into three types of reflexivity: epistemic, care-ful and instrumental. Experts mix and match these different modes, depending on their goals, preferences, strategic goals or even personal characteristics.

Epistemic reflexivity regards the quality of data and measurement and allows for a reflection on how well (or how ineffectively) metrics represent real-life problems. Here, the experts discussed how they negotiate the necessary limits to data and metrics with the awareness of the far-reaching implications of publishing official numbers.  They recognised that data and metrics do not mirror reality and critically reflected on what aspects of measured problems – such as health, poverty or education – get misrepresented in the process of measurement. And sometimes, it actually meant advising against measurement to avoid producing and reproducing uncertainty.

Care-ful reflexivity allows for imbuing quantified practices with values and care for the populations affected by the measurement. Experts positioned themselves as active participants in the process of solving challenges and advocating for disadvantaged groups (and did so via numbers). This type of reflexivity was also mobilised to make sense of the key challenge of expertise, one that would be familiar to anyone advocating for evidence-informed decision-making:  our interviewees acknowledged that the production of numbers very rarely leads to change. The key motivator to keep going despite this, was the duty of care for the populations on whose behalf the numbers spoke. Experts believed that being ‘care-ful’ required them to monitor levels of different forms of inequalities, even if it was just to acknowledge the problem and expose it rather than solve it.

Instrumental reflexivity reflects political games that experts play. Experts understood the performativity of numbers really well – they knew measurement affects how problems are understood and acted upon. And they mobilised these performative effects to achieve specific goals – build supportive coalitions of actors and persuade policymakers to focus on specific problems or act on them. Therefore, instrumental reflexivity involves strategizing and mobilising political values of quantification in explicit ways to highlight the exact political effects the measurement (even if imperfect) might achieve.

There are two key implications of our study. First, we highlighted that experts are not only aware of the limitations of the knowledge they produce but also, they actively and thoughtfully make sense of them and reflexively navigate them. They use reflexivity to make sense of less-than-perfect data environments and to work efficiently and according to their values. Second, we point to the interwoven processes of quantification and qualification in expert practices.  Reflexivity allows the assignment of political values (values with ‘heart and soul’) back to the measurement of statistical values, to enlist participation, facilitate inclusion and support experts in their daily struggle for sense-making.

This raises a question: is reflexivity a skill that could be developed and improved? In our interviews, experts discussed time and again how their ability to both critically and pragmatically reflect on the limitations of their knowledge environment improved over time. On the other hand, these skills were often implicit and rarely discussed openly.

As we open this ‘black box’ of expert reflexivity, we end this piece with a reflection on convergences between the findings of this study and the recently published Competence Framework ‘Science for Policy’ for researchers. Epistemic reflexivity is reflected in competence Cluster A – Understanding Policy. This type of reflexivity highlights the expectation to understand policymakers’ ‘evidence needs’ but also expands on it by pointing to the ‘bigger picture’ of the evidence puzzle, including what is missing and made invisible and what kinds of effects different forms of measurement might have. Care-ful reflexivity corresponds to skills in Cluster E – Collaborating, as it both requires and enhances different sensitivities as well as emotion and empathy. Care-ful reflexivity is especially important in this context as it guides the attention towards excluded voices in collaborative practices. Policy advising is often (and unsurprisingly) an elite-oriented exercise and care-ful reflexivity is precisely a space for considering whose voices are not being heard and who is getting represented in data and metrics. Furthermore, it might offer a space for thinking about the goals of evidence-informed interventions, going beyond their instrumental utility and highlighting a broader range of values and goals that might motivate and guide the researchers. And finally, instrumental reflexivity could support effective skill development in Cluster B as it is oriented towards explicitly acknowledging and engaging in politics in order to achieve the broader goals.

Overall, reflexivity could be seen as a skill that underpins many (if not all) of the skills and practices at the science-policy boundary and engaging in an open and transparent discussion over this largely ‘embodied’ set of skills might be an important step towards more effective but also more democratic science-policy interactions.

You can read the original research in Global Social Policy: Bandola-Gill, J., Grek, S., & Tichenor, M. (2023). The rise of the reflexive expert? Epistemic, care-ful and instrumental reflexivity in global public policy. Global Social Policy. https://doi.org/10.1177/14680181221145382

Justyna Bandola-Gill is an Assistant Professor in Social Policy at the University of Birmingham

Marlee Tichenor is a Lecturer in Medical Anthropology at the Durham University

Sotiria Grek is a Professor in European and Global Education Governance at the University of Edinburgh