Collateral benefit
20 Dec 2017As the preparation of a development roadmap for a software application takes a lot of organization that takes part behind the scenes and the forthcoming end of the year predisposes you to take a broader look around you, I thought this is a good time to take a step back and recognize the achievements in terms of open approaches and standards that became visible around me.
For the neuroscience community in general the piling of signs pointing at research being not reproducible seemed to act as a substantial wake-up call. I already tapped into the reasons for why an inability to replicate results from a study that showed statistically significant results could occur in my last blog post and mentioned for example the selection bias. A completely different source of problems arising for the replicability of a study can stem from closed processing scripts. Generally imaging data undergoes a large number of processing steps of which we mostly only see the final outcome that ultimately appears in a publication. Through all of the different steps of course errors can occur that are not verifiable for a reviewer without the scripts containing the code used to produce the final outcomes. In this case reviewers have to rely on the methods section of a paper alone that often enough does not describe the processing pipeline in sufficient detail or can of course also contain errors. Neuroscience has its fair share of processing packages which generally enables researchers to choose a package that serves their needs best, but can also lead to false assumptions regarding how a specific package handles a processing step, especially if the processing pipeline is merged with custom code. So particularly in cases where an analysis relies on custom code, the disclosure of these scripts is important. This can be vastly facilitated by using open source tools that are already out there like GitHub which primarily offers a versioning and hosting solution for code and on which this blog is also hosted, or Figshare which gives you the opportunity to release citable datasets, posters or figures along with the code by also associating them with a digital object identifier or DOI. Instead of working on making your code readable only if it is required by a journal for publication, there is a lot of benefit to gain in checking in your code regularly into a git repository and making it readable pretty much from the start as you will be able to easily share the code for a publication and your lab mates or other researchers will be able to adopt your code for own analyses and forks which improves the longevity of your code; and we all know how much time it takes to write decent code!
For all of you interested in this topic, a recent NeuroView article in Neuron by Gleeson et al. (2017, doi:10.1016/j.neuron.2017.10.013) gives a concise overview on this and is even associated with an open letter on Open Source for Neuroscience which asks researcher to commit to making important parts of their research output open source.
Fortunately I already work in a lab that values open science approaches which can be seen on our brand-new lab website which directly states in the mission statement our dedication to open research practices. Even in this already fostering environment regarding open science, I notice that being part of the Open Science Fellows Program makes me consider open approaches more often and be creative when it comes to which parts of our research can be opened up.
For example I was happy to see my colleagues talk about and adopt the making of a roadmap for a research project that details milestones of the project and aims that should be achieved as well as clarifies which steps have to be taken in order to achieve them. An example of such a roadmap is attached at the end of this post which is the roadmap for this blog’s project (in german). Good feeling to now also have this out in the open!
Another point I also liked a lot was a schematic on how the work and possible publications are distributed among us in one of our bigger research pojects on the impact of transcutaneous vagus nerve stimulation on effort-based decision-making. Completing a cumulative PhD requires a set of publications that demonstrate your ability to use scientific methods to back up your claims. Especially in bigger projects it is often unclear until the end which publications come out of it and publication authorship is a common topic of debate. So planning ahead is a huge benefit for every person involved in a project and we even discussed to make this schematic openly available which perhaps encourages other groups to also adopt such a strategy.
With these thoughts on the benefits of thinking about open science approaches more often, I now leave you to your well-earned holidays and wish you all a merry christmas!
Roadmap: Crowdsourcing the mosaic of the mind
