Pursuant to the Law on Scientific Research, the Government of Kosovo (under UNSCR 1244) requested the National Research Council (NRC) to carry out an evaluation of public research institutions. According to the Law, public research institutions are the Kosovo Academy of Sciences and Arts, the Albanological Institute, the Institute of History and Universities.

The National and University Library

The NRC decided to develop a methodology which is suitable for a range of different institutions, although the current plan is to evaluate the Albanological Institute and the Institute of History. NRC based its work on the experience of the RECORD Thematic Network project supported by the European Commission STRATA policy initiative. The main objective of this Network was to assist in learning the practice of benchmarking RTDI (research, technological development and innovation) organisations. In 2004, the Network published “The RECORD Manual – Benchmarking Innovative Research Organisations in European Accession Countries”.

The Academy of Science and Arts

In June 2011, the NRC adopted the document “Guidelines for evaluation of research institutions in Kosovo”. Given the level of development of RTD activities in Kosovo (under UNSCR 1244), the NRC considers that research  institutions in Kosovo cannot be benchmarked based on parameters generally accepted in Europe, but the evaluation process should rather raise awareness of all parties on the actual situation and come up with recommendations that would improve the position of respective institutions in society.

Specific objectives of the evaluation are:

• to establish an inventory of scientific contributions of each institution, and analyse the factors which have contributed to its successes and failures,
• to help improve the performance of research institutions,
• to inform decision makers on fund allocation,
• to assist targeted institutions in building a culture of continuous self-evaluation.

Benchmark Groups

Based on the Law and best practices from European countries, the following benchmark groups were established:

1. General benchmarks (vision, mission, values, etc.)
2. Critical mass (researchers, infrastructure, investments, etc.)
3. Progressive management (strategy, management, ICT infrastructure, etc.)
4. Human resource management (staff development, development plans, etc.)
5. Innovative and creative teams (projects, innovations, international and domestic publications, supervising PhD candidates, etc.)
6. Mobility of researchers (mobility of domestic researchers, hosting foreign researchers, etc.)
7. Links to users (consultancies, commercial services, etc.)
8. Financial issues

For each of the eight benchmark groups, a certain number of benchmarks was defined to guide the evaluation process. For example, benchmark group 5 – Innovative and Creative Teams – consists of four subgroups including the following:

Projects:

• Research projects coordinated by the institution and their financial value,
• Research projects in which the institution participates as a partner and their financial value,
• National projects and their financial value,
• International projects and their financial value.

Innovations:

• Innovations in the last five years,
• Number of patents in the last five years,
• Plans for innovation.

National and international publications:

• National publications of the institution,
• International publications of the institution,
• Publications of researchers in domestic journals,
• Publications of researchers in international journals,
• Number of citations.

The evaluation procedure envisions the following major steps:

1. NRC initiates the evaluation process;
2. An evaluation team of 3 to 5 NRC members and international evaluators is established;
3. The Institution concerned produces a self-evaluation report based on an agreed format;
4. The Evaluation Team analyses the self-evaluation report and develops a plan for the site visit;
5. A 2 to 3 day site visit is conducted;
6. The draft evaluation report is shared with the institution;
7. The Evaluation Team produces the Evaluation Report and submits it to the NRC,
8. 8. NRC reviews the Evaluation Report and establishes its final form.