The need for policy makers and the wider public to obtain insight
into the scholarly quality of research activities in universities is
legitimate, but scholarly research quality is not as
straightforwardly measured and ranked as performance in many other
societal domains.
Rankings are in a sense one-dimensional: entities are ordered by
descending score on one particular statistic. They disregard
relationships among entities, particularly how the performance of
one entity depends upon that of other entities.
Differences may exist among countries, and even among institutions
within a particular country, with respect to the criteria that are
applied in assigning the name ‘university’ to an
institution of higher education.
A research university is defined as a university of which the size
of research activity exceeds a certain level. However, it is
extremely difficult to define a precise threshold for a university’s
number of published papers above which it is to be considered as a
research university.
It is useful to distinguish between general and specialised
universities, even though it is difficult to draw a sharp borderline
between the two. A disciplinary specialisation index is proposed to
measure the degree of disciplinary specialisation in a university’s
research activities, applying a classification of published articles
into 15 disciplines.
In
order to define a university’s article output, all papers were
selected in which its name or that its major departments is
mentioned explicitly in the address, taking into account all
sorts of name variants. Moreover, papers were added from hospitals
affiliated to a university, published by authors who revealed strong
collaboration links with that university. Accuracy levels are
between 90 and 95 per cent.
Indicators
were calculated of the size of a university’s article output
and its citation impact per published article, compared to a world
average and corrected for differences in citation practices among
disciplines. Data were extracted from the Web of Science
(WoS), created by Thomson Scientific.
WoS
coverage is excellent or good in most disciplines, except in parts
of social sciences and humanities. Therefore, the indicators
calculated in this study may not properly reflect a university’s
position in the latter domains of scholarship, especially when it is
located in a non-English speaking country.
US universities are highly overrepresented in the top of the world
ranking based on published article output, and particularly in that
based on citation impact. But it needs emphasising that an European
‘top’ university tends to be among the best 25 per cent
in the world in at least one discipline, although the number of
disciplines in which it is world leader is on average substantially
lower than that calculated for a US top university.
A distinction is made between two models for distributing ‘top’
research among universities: a concentration model in which a
limited number of big research universities carries out research at
a top level in a wide range of disciplines, and a distributed model,
in which top research is more evenly distributed among universities,
and a strong link between teaching and research is maintained.
It is proposed to conduct more empirical studies on the structure of
(supra-)national academic systems, analysing the extent to which
they are structured according to one of the models described above,
and to address the question which model provides the most optimal
conditions for ‘top’ research in the various countries.
Results for an individual university can only be interpreted
properly when one takes into account the structure of the national
academic system in which it is embedded, and the particular role of
the university therein.
Historical, political and cultural factors – including
national or regional rivalry, different religious traditions or
different concepts of academic education – may account for
structural differences across national academic systems.
Bigger universities in terms of numbers of published articles tend
to generate per paper a higher normalised citation impact, and tend
to be more general than smaller ones. Universities with a higher
share of their papers published jointly with institutions from the
private sector tend to publish more papers and to generate a higher
citation impact per paper than universities that have less
co-publications with the private sector.
In Europe there is no tendency that national academic systems
showing more concentration of research activities among its
universities generate – as a whole – a higher citation
impact per paper than national systems in which the article output
is more evenly distributed among academic institutions.
No linear correlation was found between a university's degree of
disciplinary specialisation and its overall citation impact per
paper. General and specialised universities show similar citation
impacts per paper.
The
claim that universities specializing in a discipline tend to perform
in their areas of specialisation better than general universities do
in the same areas, was found to be valid in 4 disciplines:
biological sciences primarily related to humans, clinical
medicine, molecular biology and biochemistry, and in
physics.
In all other disciplines, no significant correlation was found
between a university's degree of activity in a discipline and the
average citation impact of its papers in that discipline. Perhaps
these outcomes indicate that the concept of 'critical mass' in
research activity is more relevant in 'big science' than it is in
other domains of scholarship.
Maps
based on a series of bibliometric and network
indicators are useful tools to analyse the structure of national
academic systems and the position of a university in its
international, national and regional environment.
The
publication data per university analysed in this paper were not
verified by representatives of the institutions involved, except in
a few cases. A main future task will be to find ways to enable them
to verify the data.
Combining ‘output’ with ‘input’ data,
applying compatible classification systems, further contributes to
the creation of a public information system on world research
universities, that is not only useful for the general public, but
also constitutes a database for further research on research
performance and its determinants.