Centre for Science and Technology Studies Centre for Science and Technology Studies 2333AL Leiden Zuid Holland 31715273909
  • Seminars
  • Announcements
  • Meetings that make a difference: Making grant review panels work – online and onsite

Meetings that make a difference: Making grant review panels work – online and onsite

Kristin Oxley


Fri 9 Jun 2023 | 15:00 - 16:15 (CEST)
Online
Kristin Oxley

Join Webinar Online

Download the material for this Seminar

Peer review panels assessing applications for research grants constitute a key redistributive and quality negotiation mechanism in the research system, but their added value is contested. Building on extensive observations of grant review panels and interviews with panellists in a broad variety of funding organisations, this study explores the value of grant panel review in terms of its scope for improving information sharing and processing, and whether this is affected by moving deliberations online. Adopting a systems view of the assessment process, we find that individual panellists’ characteristics and the organizational set-up of the funding competition as well as how these factors are mediated by the group dynamics of panel discussions, determine whether panel review adds value to proposal assessment. While panel deliberation has potential intrinsic value by integrating diverse expertise, improving error detection in individual assessments and contributing to a more uniform understanding of review criteria and scoring scales, deliberation also has potential extrinsic value, affecting how individual assessments are carried out. The prospect of panel deliberation constitutes a potent accountability mechanism which leads panellists, motivated by reputational concerns, to assess proposals more systematically and thoroughly. While moving deliberation online is associated with certain limitations to information sharing and processing, these limitations can largely be overcome through organisational measures. However, evaluative complexity is challenging in an online environment, and unconventional research is thus best assessed in onsite or standing rather than ad-hoc online panels.

presenter

Kristin Oxley (TIK Centre for Technology, Innovation and Culture, University of Oslo)

Message from the presenter
 

Dear all,

The paper that I will present in the next Friday webinar is very much work in progress. It initially started out as an attempt to discuss the impact of moving grant review panels' deliberations online - prompted by Covid. However, I found that I was not able to discuss this properly without exploring the broader question of if/how deliberation adds value to grant proposal assessment. This means the paper is currently too long and I am also worried if the story I am telling is sufficiently integrated or if it comes off as me trying to cram two papers into one. Your feedback on this, and any other elements that grabs your attention is most welcome.

Best,
Kristin
 

Build on Applepie CMS by Waltman Development