Section E
This last category of the Toolbox is focused on the transversal and crucial element of the RRI bottom-up demand-driven approach used by CHERRIES, i.e., evaluation and monitoring. What has been described in the various categories of the Toolbox is a complex process of innovation and change that requires to be periodically assessed and monitored, in order to steer its direction towards a more inclusive and sustainable healthcare innovation ecosystem and to overcome possible obstacles and problems. In CHERRIES, a specific challenge for monitoring and evaluation activities concerns the contextualisation of RRI at the territorial level and in healthcare innovation, and the adoption of a co-creation approach also in this field.
Usually, activities, projects, initiatives, are assessed according to criteria like efficiency, efficacy, impact, and sustainability. The resources collected here, identified in the framework of various projects and experiences, present different tailored approaches to assessment and monitoring, adding several other criteria to the four mentioned above.
In the case of CHERRIES (but also of other similar projects), evaluation and assessment have to be applied to three different processes: the project progress in itself; the process of RRI bottom-up demand-driven innovation in healthcare experimented in the three ongoing pilots in Örebro (Sweden); Murcia (Spain), and Cyprus; the process of institutional change for embedding the RRI approach in the innovation process and in the governance of the organisations involved in the pilots. In this regard, a thorough and continuous assessment and monitoring is an important element for a successful governance of the initiative.
These are the reasons why this section of the Toolbox is devoted to monitoring and assessment/ evaluation. The collected resources have been divided into three subcategories:
E1
Assessment of the embedment of RRIThis first subcategory contains different resources and tools for monitoring and assessing the embedment of RRI in different kind of organisations (such as RPOs, universities and higher education institutions, SMEs and industries, etc.) or for evaluating at what level a project or an innovation process is adopting and implementing RRI keys or dimensions, on the basis of different criteria.
This subcategory is linked with the subcategory A4 Starting the process, of which it represents the further and recurring development. As stated in A4, institutional change and RRI embedment are aimed at creating new institutional space, rules, and norms for a closer exchange between science and society. In this framework, assessment and monitoring are important aspects for assuring the governance processes of research and innovation organisations.
Some of the resources presented were devised and can still be used to accompany the process of institutional change of research organizations (universities, RPOs, higher education institutions, etc.) toward more responsible and inclusive research, for example by the design and implementation of road maps, action plans, grounding actions, etc. Some resources are specifically addressed to SMEs and Industries.
A great part of the resources collected in this subcategory have been devised, used, and promoted by European projects, like MoRRI and SuperMoRRI, RRI Tools, New HoRRizon, Compass, Prisma, Karim, Orbit, STARBIOS2, etc, but there are also some that have been developed in an extra-European project (like InFieri Project, in Canada). Some of these resources are well known, such as the MORRI list of 36 indicators. Most of the resources adopt a formative approach, using self-reflexing tools or matrixes that can be used also periodically for evaluating the institutional change process and reflecting on lessons learned from its implementation. The implementation of assessment methods might require to carry out workshops, meetings, field visits, or the organization of focused living labs. Few resources describe external evaluation procedures. One resource is focused on the relationships between formative and external evaluation.
Among the selected resources, few have a more general character. The first one is a checklist made by UNESCO defining standards for responsible research systems and organisations to be applied in all countries, whose application is also tied to the EU project RRING. The second one is a paper describing the state of art of the reflection on the definition of RRI assessment and evaluation methods and criteria for the projects funded in the framework of HORIZON2020. This reflection – and in particular the assessment of the implementation of “territorial RRI” – is in its initial stage.
This is a UNESCO document that codifies the goals and value systems by which science operates, adopted by all its member states in November 2017. The UNESCO Recommendation on Science and Scientific Researchers (RSSR) entails ten key priority areas for its global implementation and four-yearly monitoring. They re-affirmed legal commitments to guarantee scientific freedom, ensure public engagement with science, support the ‘human right to science’, establish equitable and sustainable workforces and pipelines, and many other valuable standards and norms that are meant to guide science equally everywhere. They set out a scientists’ bill of rights and agreed on scientists’ autonomy, responsibility, freedoms, and minimum working conditions. These standards are now meant to apply to researchers worldwide, whether in public, private, or higher education.
Part of interest: The entire document.
Target groups: RPOs, Innovation Business, Higher education institutions, CSOs
This is the Introduction of the book “Assessment of Responsible Innovation. Methods and practices” (by van de Poel, 2020). It explores methods and practices for the assessment of RRI. RRI aims to encourage societal actors to work together during all phases of the research and innovation (R&I) process to better align R&I and its outcomes with the values, needs, and expectations of society. Assessing the benefits and costs of RRI is thus crucial for furthering the responsible conduct of science, technology, and innovation.
Part of interest: The entire article.
Target groups: RPOs, Payors, Innovation Business, Higher education institutions, CSOs
The European Commission has started in the framework of the topic H2020-14-2018-2019 a common reflection on evaluation and assessment procedures and methods to be adopted in supporting the development of territorial RRI. The challenge is to set up a shared way to assess territorial RRI impacts. The resource is the presentation by Cristina Morcone (EC officer) at the starting event of this reflection. The presentation describes the project evaluation criteria for excellence, impact (regarding the MoRRI indicators and the Sustainable Development Goals), and implementation.
Part of interest: The entire document.
Target groups: RPOs, Innovation Business, Higher Education institutions, Policymakers, Payers
EU MoRRI project (carried out from 2014-2018) provides scientific evidence, data analysis, and policy intelligence to support the European Commission in relation to RRI. In particular, MoRRI operationalises the RRI concept and its six keys, develops a sound conceptual framework and associate methodologies; and tests the potential of this methodology to allow monitoring the current state and short-term evolution of RRI. Then, MoRRI defined a list of 36 indicators that have to be applied for assessing the impacts of RRI practices. The link contains the publications of the reports with the indicators. In compliance with the aims of MoRRI, all indicators target the country level, even though most of them are based on data aggregated from the level of institutions or individuals.
Part of interest: The entire document and in particular the list of the 36 indicators.
Target groups: Innovation business, Policymakers, RPOs, Higher Educations Institutions, CSOs
The thinking Tool (New HoRRIzon Project, Deliverable D6.1, 2018) offers practical guidance for researchers who wish to mature the societal readiness of their work. The primary goal is to help researchers to align their project activities with societal needs and expectations. The thinking Tool asks reflective questions to stimulate thinking about how to integrate ideas about RRI into research practice, at different stages in the project life.
Part of interest: The entire tool.
Target groups: RPOs (mainly), Innovation Business, Higher Education institutions
This self-assessment tool is one of the main tools of the RRI Tools platform, which contains also hundreds of resources and documents on RRI (to be used also for training on RRI). For each RRI policy agenda, the Tool includes tailored questions useful for starting a self-reflection, considering who is the respondent (research community, policymakers, education community, business & industry, civil society organisations). The use of the Tool helps the team in designing a process or project in line with RRI principle or for assessing the compliance of their activity/project with the RRI approach. The questions of the self-reflection Tool can be also downloaded, for working offline. The platform contains also a guide on how to use the tool.
Part of interest: The entire tool.
Target groups: RPOs, Innovation Business, Higher Education Institutions, CSOs
This is a paper (by Kalpazidou Schmidt, 2019) on the monitoring and assessment procedures designed and used in the framework of STARBIO2 Project actions plans implemented in biosciences research organisations. Monitoring and assessment criteria were the following: effectiveness, efficiency, relevance, sustainability, transferability, and impact. The assessment and monitoring procedures have been based on a collection of documents and information and of bilateral meetings. The document is Note #12 of the “RRI Implementation in Bioscience organisations” presented in the Guidelines from the STARBIOS2 project.
Part of interest: The entire document.
Target groups: RPOs, Innovations business, Higher education institutions
The NUCLEUS project focuses on identifying key factors for the successfully embedment of RRI in academic practices. The Implementation Roadmap (Deliverable D3.6, 2017) introduces steps and actions to install 10 embedded Nuclei and 20 mobile Nuclei as innovative and reflective RRI test-beds. NUCLEUS approach foreseen the use of a self-assessment tool by the Nucleus, starting from the beginning of the process and to carry out a SWOT analysis of the situation. Setting up a RRI Nucleus foreseen 8 actions, the last one is devoted to embed ongoing reflection, analyse processes, and procedures for monitoring and evaluating progress during the implementation (phase 8).
Part of interest: See in particular, Pp. 42-43 on monitoring and evaluation; Pp. 63-67 on the use of MoRRI indicators and the appendices.
Target groups: Policymakers, Patients and CSO, RPOs, Innovation Business, Higher Education Institutions, Intermediaries, Providers
The COMPASS Responsible Innovation self-check Tool aims to help Small and Medium-Sized Enterprises in highly innovative sectors to learn how to effectively integrate Responsible Innovation practices (RI) into their company and innovation management. For accessing it is necessary to register. The self-check Tool is described in the paper “The COMPASS self-check Tool. Enhancing organisational learning for responsible innovation through self-assessment”.
Part of interest: The entire tool.
Target groups: Innovation Business
This document (2019) provides guidelines to develop long-term strategies (roadmaps) to innovate responsibly, integrating technical, ethical, social, environmental, and economic issues into research and innovation practices. The focus is on transformative and enabling technologies. The road mapping includes 6 steps: commitment and leadership; context analysis; materiality, experiment and engage; validate; road map design. The sixth step on validation is aimed at the evaluation and validation of the added value of the Road map in terms of its impact on the product development and on the company on the basis of some criteria. The resource includes also: the RRI key performance indicators.
Part of interest: See in particular, Section 6.6. “Validation”, the table on ISO standards and RRI, the list of KPIs, the list of criteria for the evaluation exercise.
Target groups: Innovation business, Start-ups
The RRI self-assessment Tool for innovation and research organisations can be used as a starting point for an institutional change process. ORBIT set up a Framework to be adopted to include RRI principles in ICT activities of SMEs. The Framework is articulated in four areas: Anticipate, Reflect, Engage, Act − AREA. The Tool is tied with the implementation of the Framework. The Tool requires free registration.
Part of interest: The entire tool.
Target groups: RPOs, Innovation Business
This is the Responsible Innovation RI In-depth Assessment Tool template set up in the framework of the Interreg ROSIE Project for helping the SMEs to reflect on responsible innovation themes, their level of implementation within the organisations, and on the definition of a plan for the improvement of innovation practices to make them more sustainable, socially accountable, and competitive. The tool guides the reflection by providing questions organised according to the EU RRI Policy Agendas: Ethics, Gender Equality, Governance, Open Access, Public Engagement, and Science Education.
Part of interest: The entire document.
Target groups: RPOs, Innovation Business, Start-ups
This paper (by Lehoux, Silva, Oliveira, and Rivard, 2020) is focused on the relationship between formative self-assessment tools and summative external assessment approach in helping entrepreneurs to integrate RRI principles into their practices. Usually, the summative external approach received little attention. This study addresses this gap by applying the Responsible Innovation in Health (RIH) Tool, which adopted an external assessment approach, to 16 health innovations from Canada and Brazil.
Part of interest: The entire article.
Target groups: Innovation business, RPOs, Higher education institutions
The Responsible Innovation in Health RIH Assessment Tool was developed and validated by InFieri Project to assess responsibility in health innovation. In particular, it might be used by policymakers and providers to assess whether an innovation might be qualified as a Responsible Innovation in Health (RIH). A User Guide has been developed to facilitate the application of the Tool. Drawing on RRI and health policy research, the RIH Tool entails a three-step process assessment of an innovation: screening (with respect of the RIH criteria), assessment of the presence of responsibility features through nine attributes; and rating, determining the outcomes of the assessment with the help of scorecards.
Part of interest: The entire document.
Target groups: Policymakers, Providers, Innovation business, RPOs, Professionals, Patients
Responsibility Navigator is a self-reflecting tool for supporting decision-makers within research organisation (RPOs, RFOs, Industries, etc) towards more responsiveness and accountability. The tool, set up in the framework of ResAgora Project, supports those decision-makers as ‘change agents’ to work as ‘institutional entrepreneurs’, seeking to lead the R&I performed in Europe in the direction of more responsiveness. The Res-AGorA Responsibility Navigator offers support and guidance for reflecting on and intervening in decision making and negotiation processes to fund and orientate R&I activities, whereby these processes can be located within or between organisations.
Part of interest: The entire tool.
Target groups: RPOs, Innovation Business, Payers, Higher education institutions
This is the Interactive tool to assess the institution’s support needs for public engagement, provided by the UK National Co-ordination Centre for Public Engagement. The EDGE Tool was created to help universities assess their current support for public engagement, and to identify areas where they would like to see change. The self-assessment tool identifies three macro areas of public engagement: the area of purpose (including the issues of mission, leadership, and communication); the area of the process (including the issues of support, learning, and recognition); and the area of people (including staff, students and public). The tool identifies also the following dimensions: Embryonic; Developing; Gripping; Embedding.
Part of interest: The entire tool.
Target groups: Policymakers, Providers, RPOs, Business Innovation, High Education Institutions
The resource addresses the central question as to whether citizen engagement in knowledge production could enable inclusive health policymaking. Building on non-health work fields, the authors describe different types of citizen engagement in scientific research, or ‘Citizen Science’. The article describes the challenges that Citizen Science poses for public health, and how these could be addressed. The resource provides also a draft framework to enable the evaluation of Citizen Science in practice, consisting of a descriptive typology of different kinds of Citizen Science and a causal framework that shows how Citizen Science in public health might benefit both the knowledge produced as well as the ‘Citizen Scientists’ as active participants.
Part of interest: The entire document.
Target groups: RPOs, CSOs, Policymakers,
This is a paper (by Souliotis et al., 2016) describing the application Health Democracy Index (HDI) to assess the level of participation of patient associations in policy decision-making in Cyprus. The questionnaire used was comprised of two parts, a socio-demographic section, and the Health Democracy Index. The HDI shows to be efficacy in assessing the participation of patient associations.
Part of interest: The entire document.
Target groups: RPOs, CSOs, Policymakers, Providers, Professionals
E2
Assessment of Gender equality in organizationsGender equality assessment is an area in which a large number of experiences, studies, analysis, and reflections have been made during the last 10 years. Furthermore, in this regard, many desired and designed institutional changes have been accomplished. These ten years of work on gender equality assessment led to the production of tools and methods that might be transferred also in other areas. For this reason and in consideration of the relevance of gender issues for implementing an inclusive, responsible and sustainable healthcare research and innovation ecosystem, a specific subcategory was devoted to gender equality assessment. It has to be considered that also some of the resources contained in the subcategory E1 on RRI assessment dealt with gender equality.
Most of the resources contain tools and guidelines to assess gender equality in research organisations, providing monitoring tools to be used for promoting and supporting institutional change processes by accompanying the implementation of road maps, action plans, grounding actions, etc. In addition, some resources concern useful self-reflecting tools. Most of the resources have been defined and implemented in the framework of EU projects; one of the tools is taken from GEAR – Gender equality in academia and research, built by EIGE – European Institute for gender equality. Some resources contain action plans or road maps for gender equality. Three resources are also focused on the issue of the assessment of the existing situation in a research organization from a gender equality point of view (gender audit). One of those describes the experience of a survey on gender equality carried out in a research organization in the UK, which is interesting for the items investigated; the second one is a framework questionnaire for gender audit and assessment set up in the framework of the EU project ACT; the third one is a self-reflexive tool of the GRACE project. The last resource contains training material useful for the sustainability of gender equality plans.
LIBRA is an EC-funded project which brought together ten research institutes in life sciences in ten European countries that realised ten Gender Equality Plans. In this framework, LIBRA set up a Guide for evaluating the gender situation in Faculty and promoting a fair and gender-inclusive situation. The Guide provides also practical advice to remove gender bias starting from the recruitment process, for raising the number of female scientists in top-level positions.
Part of interest: The entire document.
Target groups: RPOs, Higher Education Institutes, Innovation Business
The Workplace Gender Equality Agency’s (‘WGEA’ or ‘Agency’) Gender Equality Diagnostic Tool (2019) helps to analyse the status of gender equality and pinpoint gender equality gaps within an organisation. It can be used with the Gender Equality Strategy Guide to assist with the development of a strategy for addressing inequalities.
Part of interest: The entire document.
Target groups: Policymakers, Patients and CSOs, Providers, RPOs, Innovation Business, Intermediaries, Payers
The Gender Equality in Academia and Research (GEAR) Tool provides universities and research organisations with practical advice and tools through all stages of institutional change: from setting up a gender equality plan to evaluating its real impact. The tool has been defined by EIGE – European Institute for Gender Equality. The Tool contains also examples and resources. The GEAR foresees the implementation of six steps: 1. Getting started; 2 analysing and assessing the state-of-play in the institution; 3. Setting up a gender equality plan; 4 implementing a gender equality plan; 5. Monitoring progress and evaluating a gender equality plan; 6. What comes after the gender equality plan? A Guide for using GEAR is available on the platform.
Part of interest: See in particular Step 2 and Step 5.
Target groups: RPOs, Innovation Business, Higher Education institutions
This “Reflection Tool” is a six-page document that can be printed and used to facilitate fruitful reflection and discussion on vision, goals, and implementation of RRI initiatives in smaller groups. It was initially developed as part of the GRACE Project where six research performing and funding organisations develop and implement Grounding Actions to strengthen responsible practices in their organisation or network. The Tool can, however, be used by anyone who wants to experiments with such efforts for instance at the beginning of a new project. It is meant to help a working group in its general reflection on what it wishes to achieve, in setting measurable success criteria for the sake of monitoring and evaluation, and in project management by planning the steps of implementation, foreseeing potential obstacles, and reflecting on needed resources.
Part of interest: The entire document.
Target groups: RPOs, Innovation Business, Higher Education institutions
The Guidelines (Trigger Project, Deliverable D7.5, 2017) provided orientations and analyses to manage what may happen when, in a given research organisation, a gender action plan is launched (be it promoted by a specific project team, the HR Department, the Rector, the Head of a department or other internal stakeholders). The Guidelines took into account also a wider debate on institutional change towards gender equality involving representatives of other 8 EC-funded structural change projects. The Guidelines are articulated in four macro-areas: transformational agent; activation and mobilisation; making an impact; and sustainability.
Part of interest: See in particular area #3 “Making an impact” and 4 “Sustainability”.
Target groups: RPOs, Higher Education Institute
This paper (by Palmen et al., 2019) provides an overview of the theoretical assumptions, methods, and key results from the Evaluation Framework for Promoting Gender Equality in Research and Innovation (EFFORTI) Project, which was funded by the European Commission. The purpose of EFFORTI was to analyse the impact of interventions to promote gender equality in research and innovation (R&I) and to establish criteria for more responsible and responsive research and innovation (RRI) systems in Europe. This paper provides an overview of the project’s main results and the lessons learned from the empirical analysis of R&I systems in several European countries and a comparison of 19 gender equality intervention measures (case studies). The article summarises the lessons learned and the recommendations for measuring gender equality.
Part of interest: The entire document.
Target groups: RPOs, Innovation Business, Higher Education institutions
The document (Stages Project, Deliverable D8.3, 2015) contains a description of action plans for promoting institutional change towards equal opportunity in science and a list of 20 useful recommendations for implementing structural change action plans, articulated in the following areas: collecting data and monitoring gender equality; engaging leadership; policy-making and institutionalisation; networking and empowering women to take action; integrating gender in education and research; communication and visibility.
Part of interest: See in particular recommendations #4 and #5.
Target groups: RPOs, Innovation Business, Higher Education Institution
The article (by Henderson et al., 2020) describes the survey carried out online to assess and monitor significant progress in gender equity (GE) to be eligible to apply for funding in Biomedical Research Centres (BRC). This is the first survey tool designed to rank and identify new GE markers specific to the NIHR BRCs.
Part of interest: The entire document.
Target groups: RPOs, Innovation Business, Higher Education institutions
The resource has been made in the context of ACT Project and represents the deliverable D2.1. The Gender Equality Audit and Monitoring (GEAM) tool provides an integrated environment for carrying out survey-based gender equality audit in organisations (e.g., university or research performing organisation) or organisational units (faculty, departments). The GEAM tool is based upon the Athena Survey of Science, Engineering, and Technology (ASSET) and on existing measurement scales in the scientific literature. The GEAM tool provides an integrated environment for carrying out survey-based gender equality audits and monitoring that involves a pre-defined set of recommended questions (GEAM Core) and a database for managing and sharing newly developed or adapted questionnaires.
Part of interest: The entire document.
Target groups: RPOs, Higher Education institutions
The present report, prepared within the framework of H2020 GENDERACTION Project, explores the possible intersections between gender and Open Science/Open Innovation, and should be considered as a starting point for stakeholders to reflect on how the two ERA priorities may create reinforcing synergies. The report also underscores the need for further studies and analyses. The report contains data on the situation of Open Science and Open innovation from the gender point of view. The document contains also recommendations. The third and fourth recommendations are related to gender equality assessment in RPOs and RFOs.
Part of interest: See in particular the part on recommendations.
Target groups: Policymakers, RPOs, Innovation Business, Intermediaries, Payers
The Gender Equality (GE) Academy Project is developing and implementing a high-quality capacity-building programme on gender equality in research, innovation, and higher education for researchers, managers, administrators, HR managers, and academics staff. The project promotes different kinds of training and capacity-building initiatives (composed of tailor-made training materials) aimed at increasing the skills or deepening the expertise of people involved in implementing measures towards gender equality in their institutions. Among the issues of the training, there are: the definition and implementation of a Gender equality plan; gender bias; recruitment and promotion of women leadership; gender in research content; intersectionality; the role of men; Gender equality plan and RRI; sustainability of Gender equality plan, etc. Among the resources, there is the Deliverable “D2.1 Inventory of key resources” including Area 8 specifically focused on “Setting indicators, monitoring, and evaluation” (Pg. 148-170). Training initiatives, as well as the available resources, can accompany and support the institutional change process and contribute to its sustainability.
Part of interest: The entire website, and in particular the pages Repository, Deliverables, Past training, and Future offer.
Target groups: Policymakers, CSOs, RPOs, Innovation business, intermediaries, Higher education institutions
E3
Impact assessment and sustainabilityThis final subcategory of the Toolbox is focused on the complex issues of the impact assessment of research and innovation initiatives: projects and interventions; policies; institutional change processes, etc. Impact assessment plays an important role also to guarantee the sustainability of the promoted changes or interventions or innovation proposed.
As shown by some of the resources collected here, the issue of sustainability requires to be taken into account since the beginning of the research and innovation activities.
On the basis of the characteristics of CHERRIS approach, this subcategory presents methods of assessment to investigate diverse domains of impacts.
The first area of resources is focused on sustainability. The 2030 Agenda for Sustainable Development, adopted by all United Nations Member States in 2015, “provides a shared blueprint for peace and prosperity for people and the planet, now and into the future”. At its heart, there are the 17 Sustainable Development Goals (SDGs), which are an urgent call for action by all countries for a global partnership. This first group of resources contains tools (also self-reflexive tools) for assessing the impact with the respect to the 17 United Nations SDGs. Most of the EU projects are requested to contribute to the SDGs. Most of the resources of this group are specifically addressed to industries and SMEs.
The second group of resources describes methods and approaches for assessing the implementation of territorial RRI initiatives. One should note that the promotion of projects and initiatives aimed at embedding the RRI approach in the governance of territory to manage territorial challenges and risks is a fairly new development. These resources provide some first examples of methods (a different one from the other) for assessing RRI impacts (see the experiences of SeeRRI, TeRRItoria, Transform, etc.). These projects are also tied with the implementation of the UE Smart Specialisation Strategy (S3) in each UE region. Two resources are also focused on the assessment of the implementation of S3.
The last group of resources contains reflections and methods on how to assess and measure the impact of research activities (in general and with a focus on health) on the wellbeing of people and communities. The literature on research impact assessment (RIA) is rather abundant, so the resources collected here have been chosen for their innovativeness and because they have been taken into account in the definition of CHERRIES evaluation approach.
The SDG Impact Assessment Tool is a free online learning tool that visualizes the results from a self-assessment of how an activity, organisation, or innovation affects the SDGs. It aims to stimulate the user to get a better understanding of the complexity of sustainable development and the different aspects of the SDGs and to support the user in prioritizing actions.
Part of interest: The entire tool.
Target groups: Business Innovation, RPOs, CSOs, Policy Makers
This is a Tool on Impact Management for the Sustainable Development Goals for Business organisations. Access to the Tool requires free registration.
Part of interest: The entire tool.
Target groups: RPOs, Innovation business, Intermediaries
This is a tool a company can use to measure its impact on its workers, community, environment, and customers. The Tool provides a certification. The use of the Tool foresees three steps: 1 step, assess the impact of the organisation with regard to the following impact areas: governance, workers, community, and environment); step 2: share the results with teams and compare the impact with that of other organisations; step 3: improve the impact by designing a road map, taking into account the answer about the impact of the organisation. The website contains examples and case studies.
Part of interest: The entire tool.
Target groups: RPOs, Innovation business, Intermediaries
This is a platform set up by the Global Reporting Initiative Standards, aimed at creating a common language for organisations – large or small, private or public – to report on their sustainability impacts in a consistent and credible way. In addition to reporting companies, the standards contained in the platform are highly relevant to many other groups, including investors, policymakers, capital markets, and civil society.
Part of interest: The entire report and in particular the standard.
Target groups: RPOs, Innovation Business, CSO, Payers, Policymakers
This is a RRI self-assessment tool set up in the framework of KARIM project. The Karim assessment Matrix helps entrepreneurs and innovators to gain insight into where they are concerning the social, environmental, and economic impact of their companies and how they have organised their processes. Applying it to the research and innovation process(es) of an enterprise, it will allow to know the strengths. Moreover, the areas in which it can make progress are pointed out. This will help innovators to decide the area of improvement. The Matrix foresees four steps: selection of relevant topics; description of the situation; possible improvements; identification of possible obstacles; needed resources. Filling the matrix requires around 4 hours.
Part of interest: The entire tool.
Target groups: RPOs, Innovation Business, Start-ups
The report presents a set of preliminary conceptual and practical considerations on the evaluation of the Smart Specialisation policy. It opens a discussion that aims to set the scene for more articulated and detailed reflections. This is one of the official Guides to evaluate RIS3 policies.
Part of interest: The entire document.
Target groups: Policymakers, RPOs, Higher Education Institutions, Innovation Business, CSOs
The Wheel is a tool of the RIS3 Guide. It presents a system for the synthetic representation of the progress made in drafting/designing a RIS3 that allows condensing a huge amount of information in one visual modality.
Part of interest: The entire document.
Target groups: Policymakers, RPOs, Higher Education Institutions, Innovation Business, CSOs
This document (SeeRRI project, Deliverable D6.1, 2020) provides information on the design methods for evaluating the SeeRRI project activities and the framework for self-sustaining ecosystems in terms of: Outcomes of the activities initiated in the three territories affiliated to the SeeRRI Project; Societal, democratic, environmental, economic and scientific impacts of activities in the territories; Recommendations on policy and governance structures to facilitate the creation and maintenance of self-sustaining RRI ecosystems.
Part of interest: The entire document.
Target groups: Policymakers, RPOs, Innovation business, CSOs
This is the Deliverable D7.1 of the Transform Project. It provides a first set of instruction and advice on the monitoring and assessment of embedding the principles and practices of RRI – Responsible Research and Innovation – into institutions, policies, and practices of innovation at the regional scale. The project involves itself in processes of innovation, policy-making, and practice in three European regions: Lombardy, Catalonia, and the Brussels-Capital Region.
Part of interest: The entire document.
Target groups: Policymakers, RPOs, Innovation business, Intermediaries, Higher education institutions, Payers
This is the first evaluation scheme set up by the TeRRItoria project (Deliverable D6.1.). This scheme is being up-dated and tailored with respect to the content of the five on-going TeRRItoria experiments (TE) of implementation of RRI approach at the territorial level. By bringing together a wide range of stakeholders in the development and implementation of the five experiments, the project intends to address and mitigate territorial R&I challenges through the advancement of “Territorial RRI”. Overall, the internal evaluation of activities is designed to assess the implementation of TE actions/initiatives and their impacts produced throughout the project and to use this knowledge to ensure the long-term sustainability of the experimental activities commenced.
Part of interest: The entire document.
Target groups: Policymakers, RPOs, Innovation business, Intermediaries, CSOs, Payers
This is a paper (by Jolyet al., 2015) describing the ASIRPA approach for assessing the socio-economic impact of RPOs through case studies. This approach has been set up and used in the framework of ASIRPA (Socio-Economic Analysis of the Impacts of Public Agricultural Research) project. The cases are theory-based, selected to characterize the diversity of the broader impacts, and standardized to allow the scaling-up of the analysis of the impact to the level of the organisation.
Part of interest: The entire document.
Target groups: RPOs, Innovation business, Policymakers, Payers, CSOs, Higher Education Institute, Providers
This is an article (by Patton, 1994) describing, also with examples, the approach of the developmental evaluation, applied to policies, projects, interventions. This is the definition of Developmental evaluation: Evaluation processes and activities that support program, project, product, personnel and/ or organisational development (usually the latter). The evaluator is part of a team whose members collaborate to conceptualize, design, and test new approaches in a long-term, on-going process of continuous improvement, adaptation, and intentional change. The evaluator’s primary function in the team is to elucidate team discussions with evaluative data and logic and to facilitate data-based decision-making in the developmental process.
Part of interest: The entire document.
Target groups: Policymakers, RPOs, Higher Education Institutions, Innovation Business, CSOs
This is a paper (by Kok, and Schuit, 2012) describing a new method for assessing and enhancing the impact of health research. Contribution Mapping is inspired by a perspective from social studies of science on how research and knowledge utilisation processes evolve. For each research project that is assessed, a three-phase process map is developed that includes the main actors, activities, and alignment efforts during research formulation, production, and knowledge extension. The approach focuses on the actors involved in, or interacting with, a research project (the linked actors) and the most likely influential users, who are referred to as potential key users.
Part of interest: The entire document.
Target groups: RPOs, Innovation business, Providers, Professionals, CSOs, Policymakers