The recent FP7-REGPOT call for proposals targeting the Western Balkan countries aimed to support the strategic development of a couple of outstanding research organisations in South East Europe. A mandatory requirement for the application is the inclusion of a completed SWOT analysis (Strengths, Weaknesses, Opportunities and Threats) in the proposal. SWOT is a tool, which is regularly used in business, but has since long diffused to many other socio-cultural spheres for strategy-making. One of them is the field of science and technology. However, the quality of SWOT analyses differs considerably. A well-designed SWOT approach is doubtlessly more than brainstorming. A critical factor for a successful SWOT analysis is the definition of categories relevant for strategy development and the formulation of the right questions. This calls for a comprehensive insight into the topic and needs a coherent research design. Moreover, data requirements can be extensive. A committed management to support the SWOT and to initiate necessary change pro-cesses (e.g. in format of an action plan) based on the results of the SWOT-based evaluation is another must.

Klaus Schuch from the Centre for Social Innovation and Stane Pejovnik from the University of Ljubljana have prepared a SWOT-based design for benchmarking research organisations under the previously funded REGPOT project “SWOTChemistry-Food: Evaluation of the research capacity and development of a strategy for further growth in chemistry in general and in food science in particular”, coordinated by the Institute of Chemistry of the UKIM in Skopje. In the research design the following three dimensions of comprehensive research processes, which characterise a modern research organisation, have been assessed:
1. Knowledge generation
2. Knowledge utilisation
3. Knowledge diffusion

The research design was deducted from the RECORD benchmarking manual developed under FP5, which became a standard for the benchmarking of innovative research organisations in European Accession Countries.
Data requirements centred on

• Input data (human resources and material and non-material investments; intellectual, social and relational capital)
• Through- and output data (publications, patents, projects, innovations etc.)
• Information about organisational framework conditions and systemic RTDI conditions.

In a next step data have been assigned to certain key factors:

Internal factors, which are under direct control of the organisation under scrutiny

• e.g. mission and value system
• critical mass (differentiated by scientific specialisations)
• progressive management
• human resource management
• creativity and innovation (“outputs”)

External factors, which are outside the control of the organisation

• e.g. mobility and international networking
• user involvement
• public relations
• financial position

Negotiated factors, over which the organisation has just limited “control”; these factors have to be negotiated with at least one external party to become effective,

• e.g. RTDI policy of the country/region under scrutiny (as framework condition)
• the “capital market” for financing RTDI
• (in)dependency of political parties
• relevant industrial sectors in transition

In order to assess the performance of a research organisation, three potential benchmarks for comparison are distinguished:
1. Internal benchmarking across time
2. External benchmarking with comparable organisations
3. Functional benchmarking (analysis of functions and practices vis-a-vis some accepted standards).

Although the emphasis of this SWOT-based benchmarking was mostly on the learning dimension (“why and how to improve”), the methodology can be used for accountability purposes, too (“have the goals been achieved or not”). It provides, for instance, a decision making basis for whether or not an institute qualifies as national, regional or even international Centre of Excellence. Another possibility is to use the methodology for institutional evaluation under existing performance contracting schemes. Even the adequacy of the allocation of institutional funding (“block or basic funding”) can be assessed with such an approach. There are of course limitations and too mechanistic benchmarking approaches should be avoided. For more reading on this I recommend the contributions of Attila Havas, Balázs Borsi and Gábor Papanek, Klaus Schuch and Peter Stanovnik and others in the book “Supporting Record Centres of Excellence: Conclusions for Policy”, edited by Balázs Borsi and Gábor Papanek.
 

Author: Klaus Schuch