Experts from several disciplines have raised concerns about the use of seemingly precise models to guide policies in matters as complex as the current Covid-19 pandemic. A group of scholars, including Ismael Rafols (senior researcher at CWTS), wrote down five principles on how to responsibly use scientific models. This comment was published in Nature on 24 June 2020 – available here.
They argue that models that disregard uncertainties undermine society’s trust in science and convey a false sense of predictability. This leads to potentially harmful consequences to society if the model-based conclusions are used to guide policies in the real, uncertain world.
Once a number takes centre-stage with a prevailing narrative, other possible estimates can disappear from view. This may lead to problematic consequences as other options are marginalized. This dynamics of can be observed in many issues from risk assessment to research evaluation.
The call for ‘responsible modelling’ resonates and builds on CWTS ongoing work on ‘responsible evaluation’ (see special CWTS theme hub) following our contributions on the Leiden Manifesto and the UK Metric Tide.
The authors of the comment have proposed a set of principles to increase the transparency and suitability of models to provide useful insights for policy-making. These guidelines combine statistical approaches with social norms that should be observed throughout the process that goes from model design to the effective implementation of model-based policies.
The group argues that models should make their assumptions explicit, systematically account for uncertainties, tame model hubris and acknowledge model limitations. Multiple views on the issue of interest should be integrated, and keep stakeholders involved, to do justice to the complexity of the situation at hand. They also ask modelers to acknowledge ignorance, a much needed virtue to prevent end-users of the model from making excessively adventurous claims on issues that might not have a clear cut fix.