Summary

The evaluation standards published by EVAL-INNO aim to contribute to the improved implementation and exploitation of Research, Technological Development and Innovation (RTDI) measures by promoting meaningful evaluation procedures to foster strategic intelligence building and evidence-based decision-making in the field of science, technology and innovation (STI) policy. They address:

1. authorities commissioning RTDI evaluations (often ministries in charge of research, technological development and innovation and their respective measures, programmes and policies);
2. evaluators carrying out RTDI evaluation studies;
3. organizations and stakeholders subjected to evaluations, such as funding agencies, public research organizations, universities or intermediary organizations (e.g. technology transfer offices, technology and science parks, impulse and innovation centres etc.).

The evaluation standards provide information about the purposes and characteristics of evaluations in the field of STI. They introduce an internationally acknowledged terminology and evaluation theory framework, guide users in practical issues concerning governance, conduct and use of RTDI evaluations and provide many practical hints on how to plan and implement evaluations, including the writing of Terms of References (ToR) to procure external RTDI evaluations and the structuring of meaningful evaluation reports, to mention just a few.

The publishing of RTDI evaluation standards is motivated by the complexity and heterogeneity of research and innovation systems, which requires ERDF and IPA countries to possess strategic intelligence in order to design, implement and follow-up RTDI measures at different spatial levels (local, national, regional and European) by addressing issues of relevance, efficiency, efficacy, impact and sustainability. Evaluations are one of the most essential tools for evidence-based decision-making. This is especially true in the South East European (SEE) region, which is characterised by an emergence of new RTDI policies, programmes, and (support) organizations and a funding transformation towards competitive schemes. Coevally, however, a lack of methodological and procedural know-how both on the side of evaluators and awarding authorities concerning purpose, design and use of evaluations is obvious[1]and the potential of evaluations to build strategic intelligence is not being fully exploited yet.

Especially under tight financial regimes, public spending for RTDI has to be justified through the identification of correct rationales and mechanisms for performance-based RTDI funding from the start. To secure the optimum use of taxpayers’ money, the principles of good governance have to be respected. Adequately conducted RTDI evaluations are a proper tool for ensuring transparency and accountability and contribute to an efficient new public management. The evaluation standards guide provides support to conduct proper and meaningful tenders to procure RTDI evaluations as well as to implement them in a way that secures strategic intelligence building and evidence-based decision-making.

The standards have been drawn up in an interactive process involving experts[2] from six countries through discussion of and reflection on existing RTDI experience and framework conditions in the South East European region. It is the first attempt of this kind at the SEE level. The authors have not been working from scratch and `re-inventing the wheel`, but rather were using the existing practice of evaluation standards from EU countries (especially the Austrian Evaluation Standards in Research and Technology Policy[3]) and the USA[4]as successful examples of good practice and trying to adapt them as comprehensibly concisely and usefully as possible to the particular situation and needs of the region.

Towards the end of the project, these RTDI Evaluation Standards will be revised and an updated version will be issued, enriched through the regionally focused experience that the consortium will gain through joint work under the umbrella of the EVAL-INNO project.

Background

The major objective of EVAL-INNO is to promote the role of RTDI evaluations as an important element of strategic policy intelligence for the improvement of national research and innovation systems.

In particular, the project aims to:

• promote the role of RTDI evaluation as a crucial precondition for a reflexive learning research and innovation system;
• develop needed capacities and competencies for comprehensive RTDI evaluations;
• provide procedural and methodological know-how and tool kits both on the side of evaluators and on the side of awarding authorities to facilitate the use of RTDI evaluations.

The activities aimed at building up RTDI evaluation intelligence in the South East Europe region include: training activities, workshops and conferences, the establishment of exchange platforms and databases, the implementation of pilot evaluations and — last but not least — the publication of evaluation standards.