Skip to main content
Knowledge4Policy
Knowledge for policy

Supporting policy with scientific evidence

We mobilise people and resources to create, curate, make sense of and use knowledge to inform policymaking across Europe.

  • Projects and activities | Last updated: 15 May 2025

ValuesML - Unravelling Expressed Values in the Media for Informed Policy Making

Using machine learning to empower policymakers with a deeper comprehension of citizens' values and aspirations concerning policy matters.

Evidence-Informed Policy Making

Brief me

The ValuesML project was an integral part of a broader JRC initiative with the goal to empower policymakers with a deeper comprehension of citizens' values and aspirations concerning policy matters. Recognizing the pivotal role of values and identities in shaping citizens' perspectives on policies, our project aimed to dig deeper into this dimension.

Through an innovative approach utilizing machine learning, the ValuesML project endeavoured to explore expressed values within news media and political manifestos. By establishing meaningful connections between these values, policies, significant events, and political landscapes, we sought to provide policymakers with a more comprehensive understanding of the underlying factors at play when formulating effective policies.

Collaborative and multidisciplinary, the project brought together over 70 researchers, research assistants, and students of the realm of values under the lead of the JRC. Leveraging the vast data resources of Europe Media Monitor (EMM) and a collection of political manifestos, we curated a gold standard dataset as a foundational element for training a more sophisticated algorithm for values detection. Among others, the algorithm has been integrated into the full EMM system to extend the ability to deduct from the vast news landscape what matters to citizens and how.

The project was run in collaboration with the JRC's Text Mining & Analytics Competence Centre (TMA-CC), which is specialised in making sense of vast amounts of text through computing and analytics. For more information, see: Political Intelligence | Knowledge for policy (europa.eu).

Additionally, in an open competition hosted as part of the CLEF conference, researchers and practitioners from all over the world submitted their trained algorithms to collaboratively inquire which algorithm works best. For more information, see https://touche.webis.de/clef24/touche24-web/human-value-detection.html and https://clef2024.imag.fr/.

Publications

  1. Values in News and Political Manifestos: Annotation Guidelines
  2. Johannes Kiesel, Çağrı Çöltekin, Maximilian Heinrich, Maik Fröbe, Milad Alshomary, Bertrand De Longueville, Tomaž Erjavec, Nicolas Handke, Matyáš Kopp, Nikola Ljubešić, Katja Meden, Nailia Mirzakhmedova, Vaidas Morkevičius, Theresa Reitis-Münstermann, Mario Scharfbillig, Nicolas Stefanovitch, Henning Wachsmuth, Martin Potthast, and Benno Stein. Overview of Touché 2024: Argumentation Systems. In Lorraine Goeuriot et al., editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction. 15th International Conference of the CLEF Association (CLEF 2024), volume 14959 of Lecture Notes in Computer Science, pages 308–332, September 2024. Springer. [bib] [clef notes touche] [code] [copylink] [data] [doi] [ecir abstract imageclef] [ecir abstract touche] [event] [imageclef] [poster] [slides]
  3. Aisha Nur Aydin, Shaden Shaar, and Claire Cardie. Edward Said at Touché: Human Value Detection Using Transformers and Upsampling. In Guglielmo Faggioli, Nicola Ferro, Petra Galuscakova, and Alba García Seco Herrera, editors, Working Notes Papers of the CLEF 2024 Evaluation Labs, volume 3740 of CEUR Workshop Proceedings, pages 3379–3383, September 2024. [bib] [copylink] [publisher]
  4. Praveen K, Darshan R K, Chinta Tejdeep Reddy, and Anand Kumar M. SCaLAR NITK at Touché: Comparative Analysis of Machine Learning Models for Human Value Identification. In Guglielmo Faggioli, Nicola Ferro, Petra Galuscakova, and Alba García Seco Herrera, editors, Working Notes Papers of the CLEF 2024 Evaluation Labs, volume 3740 of CEUR Workshop Proceedings, pages 3407–3413, September 2024. [bib] [copylink] [publisher]
  5. Sotirios Legkas, Christina Christodoulou, Matthaios Zidianakis, Dimitrios Koutrintzes, Maria Dagioglou, and Georgios Petasis. Hierocles of Alexandria at Touché: Multi-task & Multi-head Custom Architecture with Transformer-based Models for Human Value Detection. In Guglielmo Faggioli, Nicola Ferro, Petra Galuscakova, and Alba García Seco Herrera, editors, Working Notes Papers of the CLEF 2024 Evaluation Labs, volume 3740 of CEUR Workshop Proceedings, pages 3419–3432, September 2024. [bib] [code] [copylink] [publisher] [slides]
  6. Ranjan Mishra and Meike Morren. Eric Fromm at Touché: Prompts vs FineTuning for Human Value Detection. In Guglielmo Faggioli, Nicola Ferro, Petra Galuscakova, and Alba García Seco Herrera, editors, Working Notes Papers of the CLEF 2024 Evaluation Labs, volume 3740 of CEUR Workshop Proceedings, pages 3433–3446, September 2024. [bib] [copylink] [publisher] [slides]
  7. Víctor Yeste, Mariona Coll-Ardanuy, and Paolo Rosso. Philo of Alexandria at Touché: A Cascade Model Approach to Human Value Detection. In Guglielmo Faggioli, Nicola Ferro, Petra Galuscakova, and Alba García Seco Herrera, editors, Working Notes Papers of the CLEF 2024 Evaluation Labs, volume 3740 of CEUR Workshop Proceedings, pages 3503–3508, September 2024. [bib] [code] [copylink] [publisher] [slides]
  8. Hamza Yunis. Arthur Schopenhauer at Touché 2024: Multi-Lingual Text Classification Using Ensembles of Large Language Models. In Guglielmo Faggioli, Nicola Ferro, Petra Galuscakova, and Alba García Seco Herrera, editors, Working Notes Papers of the CLEF 2024 Evaluation Labs, volume 3740 of CEUR Workshop Proceedings, pages 3509–3517, September 2024. [bib] [code] [copylink] [publisher] [slides]

Team

Project coordinators

From the JRC:

  • Emmanuel Cartier
  • Bertrand De Longueville
  • Theresa Reitis-Münstermann
  • Mario Scharfbillig
  • Paula Schulze Brock
  • Nicolas Stefanovitch

From Bauhaus-Universität Weimar: Johannes Kiesel

Language Leads

  • Murat Ardag, Bremen International Graduate School of Social Sciences
  • Sharon Arieli, The Hebrew University Business School
  • Ella Daniel, Tel Aviv University
  • Henrik Dobewall, Finnish Institute for Health and Welfare
  • Johannes Karl, Dublin City University
  • Anna Krasteva, New Bulgarian University
  • Thomas Peter Oeschger, University of Basel
  • Georgios Petasis, NCSR "Demokritos"
  • Luana Russo, Maastricht University
  • Antonella Seddone, University of Turin
  • Joanne Sneddon, The University of Western Australia
  • Aurelia Tamo-Larrieux, Maastricht University
  • Hester van Herk, Vrije Universiteit Amsterdam

Annotators and Curators

  • Sandrine Astor, Pacte research centre, School of Political Studies Univ. Grenoble Alpes
  • Petra Auer, Free University of Bozen-Bolzano
  • Livia Aulino, Università degli Studi Suor Orsola Benincasa
  • Nazan Avcı, Middle East Technical University Northern Cyprus Campus
  • Anat Bardi, Royal Holloway University of London
  • Fiorella Battaglia, University of Salento, Lecce & Ludwig-Maximilians-Universität, München
  • Constanze Beierlein, Hamm-Lippstadt University of Applied Sciences
  • Maya Benish-Weisman, The Hebrew University of Jerusalem
  • Giuliano Bobba, UNITO
  • Christina Christodoulou, NCSR "Demokritos"
  • Dr Patricia (Trish) Collins, Edith Cowan University
  • Irene Coppola, ReCEPL-Università degli Studi di Napoli Federico II-
  • Ahmet Çoymak, Abdullah Gul University
  • Maria Dagioglou, National Centre for Scientific Research "Demokritos"
  • Mafalda Dâmaso, Erasmus University Rotterdam
  • Giuseppe Di Vetta, Scuola Superiore Sant’Anna
  • Einat Elizarov, The University of Haifa
  • Naama Erlich, Ben-Gurion University Of The Negev
  • Peculiar Tochukwu Ezeigwe-Ephraim, ISCTE - Instituto Universitário de Lisboa
  • Ronald Fischer, IDOR - D'Or Institute for Research and Education
  • Maria Cristina Gaeta, Research Centre of European Private Law (ReCEPL), Suor Orsola Benincasa University of Naples
  • Lucilla Gatt, Università degli STudi Suor Orsola Benincasa
  • Noam Gerera, Tel Aviv University
  • Sjoukje Goldman, Amsterdam University of Applied Sciences
  • Frederic Gonthier, Sciences Po Grenoble - School of Political Studies, Grenoble Alpes University, France
  • Stefanie Habermann, Royal Holloway University
  • Mina Hristova, Bulgarian Academy of Sciences
  • Luigi Izzo, Suor Orsola Benincasa
  • Panos Kapetanakis, NCSR 'DEMOCRITOS'
  • Dora Katsamori, National Centre for Scientific Research 'Demokritos'
  • Reşit Kışlıoğlu, Middle East Technical University Northern Cyprus Campus
  • Chaya Koleva, Policy and Citizens' Observatory
  • Roberta Koleva, New Bulgarian University
  • Joshua Lake, University of Western Australia
  • Ingmar Leijen, Vrije Universiteit Amsterdam
  • Mael Mesplou, Sciences Po Grenoble - School of Political Studies, Grenoble, France
  • Meike Morren, University Amsterdam
  • Vanina Ninova, Policy and Citizens' Observatory
  • Elif Sandal Önal, Bielefeld University
  • Berna Öney, Carl von Ossietzky Universität Oldenburg
  • Shani Oppenheim Weller, Hadassah Academic College
  • Duygu Ozturk, Istanbul Medipol University
  • Ioannis Elissaios Paparrigopoulos, NCRS Demokritos
  • Chiron Pierre, Pacte Research Centre, School of Political Studies Univ. Grenoble Alpes
  • Vladimir Ponizovskiy, Ruhr-Universität Bochum
  • Tim Reeskens, Tilburg University
  • Maria Francesca Romano, Scuola Superiore Sant'Anna
  • Torven Schalk, Te Herenga Waka - Victoria University of Wellington
  • Ricarda Scholz-Kuhn, University of Basel
  • Oscar Smallenbroek, Joint Research Centre
  • Alina Starovolsky-Shitrit, Tel Aviv University
  • Ömer Topuz, Abdullah Gul University
  • Christin-Melanie Vauclair, Iscte - University Institute of Lisbon
  • Demet Islambay Yapali, Independent Researcher
  • Sheng (Charice) Ye, East China University of Science and Technology
  • Emilia Zankina, Temple University

 

Dataset

The full dataset of the project is available for free here: https://zenodo.org/records/13283288

 

Testing

Want to test your text or speech for values? Try it out here: Values classifier

VALUES CLASSIFIER