Open Code and Peer Review

In this episode, we are talking about “open code” or “open source” and the benefits of making your code available in a peer review process and having it checked.

Our guest is Dr. Stephen Eglen from the department of Applied Mathematics and Theoretical Physics at the University of Cambridge.

Together with Dr. Daniel Nüst, from the University of Münster, he has created Code Check – an open-science- initiative to facilitate the sharing of computer programs and results presented in scientific publications.

Checking the code

What CODE CHECK actually do is to run the code to make sure that the workflow can be reproduced. As Eglen explains in the podcast episode, they don’t evaluate the result or pass judgment on scientific merit.

That means that it can be fairly easy to cheat their system. However, and that is the beauty of open science, the source code is open. That means that at some point someone will probably discover any attempts to cheat the CODE CHECK-system.

They also put their badge on the article that shows that the code works, and also points to where the checked files are.

Benefits to Open Code

According to Eglen, there are several benefits to having code from research checked. But the whole idea of CODE CHECK is to create benefits for the entire publication cycle.

Eglen was recently at the Munin Conference in Tromsø (Norway), where he held a talk about the project.

Here are the benefits that Eglen showed during the Conference in 2019:

1. The author gets an early check that the “code works”: and gets a snapshot of code archived and increased trust in the stability of results.
2. The codechecker gets insight into the latest research and methods, credit from the community, and a citable object.
3. The publisher gets a citable certificate with code/data bundle to share and increase reputation of published articles.
4. Peer reviewers can see certificate rather than check the code themselves
5. the reader can check certificate and build upon work immediatley.

You can find the abstract here:
https://doi.org/10.7557/5.4910

You can also watch the talk here.

Code Check is funded by the UK Software Sustainability Institute, a Mozilla Open Science Mini grant and German Research Foundation (DFG) under project number PE 1632/17-1.

If you found this interesting, you should also check out these episodes:

Episode 6 – The Problem with Peer Review

Episode 18 – Preregistration in Science

Episode 3 – Senior Scientists & Valuable Data