Craig2024FAIRMEPR

Title

From Open Review to Reproducible Review: FAIR Metrics Analysis of Open Peer Reviews for Brain Informatics Literature
View PDF | Save PDF

Authors

Adam Craig, Carl Taswell

Affiliations

Brain Health Alliance Virtual Institute, Ladera Ranch, CA 92694 USA

Abstract

Because the goal of brain informatics is to help researchers discover and derive new insights from existing data and metadata, documentation of new methods, platforms, and data sources in scholarly reports is especially important in this field. Expert evaluation of new scholarly works through peer review is essential to maintaining the integrity of these reports, but how best to evaluate the quality of the peer reviews remains an open question. We previously proposed the paradigm of reproducible peer review, in which a second reviewer should be able to draw on the same factual claims as the first in order to reach the same conclusion. We then introduced the Fair Attribution to Indexed Reports (FAIR) Metrics for peer reviews as metrics of how well reviewers attributed the claims substantiating their recommendations to sources. However, we previously only demonstrated these new FAIR Metrics on five example reviews. We here report the results of FAIR Metrics analyses of published peer reviews of 14 brain informatics articles. These analyses demonstrate the value of the FAIR Metrics by highlighting key ways the brain informatics community can improve the reproducibility of the peer review process, particularly by calling for references to or quotes from the specific passages of the work under review, indication of which standards of the publication venue the work meets or fails to meet, and citation of the literature when drawing on prior knowledge of the problem domain.

Keyphrases

Brain informatics, peer review, reproducibility, FAIR Metrics, PDP-DREAM Ontology, NPDS Cyberinfrastructure, PORTAL-DOORS Project.

Citation

Brainiacs Journal 2024 Volume 5 Issue 2 Edoc QA6A795A3
DOI: 10.48085/QA6A795A3
PDP: /Nexus/Brainiacs/Craig2024FAIRMEPR
URL: BrainiacsJournal.org/arc/pub/Craig2024FAIRMEPR

Dates

Created 2024-10-07, presented 2024-10-09, updated 2024-12-23, published 2024-12-23.