Electronic lab notebook

I was interested to read C. Titus Brown‘s recent post, “Is version control an electronic lab notebook?

I think version control is really important, and I think all computational scientists should have something equivalent to a lab notebook. But I think of version control as serving needs orthogonal to those served by a lab notebook.

As Titus points out, a traditional lab notebook serves two purposes: provenance and protocol. Version control could be useful for provenance, but I don’t really care about provenance. And for protocol, version control doesn’t really matter.

Version control

I really like git with github. (See my tutorial.) But for me, the basic need served by version control is that embodied in the question, “This shit worked before; why isn’t it working now?”

You don’t want to edit working code in place and so possibly break a working system. Version control lets you try things out, and to try something out in any version of your software, from any point in time.

The other basic use of version control is for managing projects with multiple contributors. If there are multiple programmers working on a software project, or multiple authors working on a manuscript, version control is the best way to manage things, particularly for merging everyone’s efforts.

These are really useful things, but version control is more about merging and history and not so much reproducible research.

Make is the thing

To me, the basic tool to make research reproducible is GNU make (see my minimal tutorial). You create a Makefile that documents all analysis steps in a project. (For example, “Use this script to turn these raw data files into that combined file, and use this script to create figure 1 and that script to create figure 2, then combine them with this LaTeX file to make the manuscript PDF.”)

With GNU make (see also rake), you both document and automate these processes. With well-documented/commented scripts and an all-encompassing Makefile, the research is reproducible.

Add knitr, and you’ve got a notebook

The other ingredient to create the computational scientist’s equivalent of a lab notebook is knitr, which allows one to combine text (e.g., in Markdown or asciidoc) and code (e.g., in R) to make documents (e.g., in html or PDF) that both do the work and explain the work. Write such documents to describe what you did and what you learned, and you’ve got an electronic lab notebook.

You could even get rid of your Makefile by having an over-arching knitr-based document that does it all. But I still like make.

But it’s so much work!

Going into a file and deleting a data point is a lot easier than writing a script that does it (and also documents why). But I don’t think you should be going in and changing the data like that, even if it is being tracked by version control. (And that is the main complaint potential users have about version control: “Too time consuming!”)

I think you have to expect that writing well-documented scripts and knitr-based reports that capture the totality of a data analysis project will take a lot of work: perhaps double (or more!) the effort. But it will save a ton of time later (if others care about what you did).

I don’t really want to take this time in the midst of a bout of exploratory data analysis. I find it too inhibiting. So I tend to do a bunch of analyses, capturing the main ideas in a draft R script (or reconstructed later from the .Rhistory file), and then go back later to make a clean knitr-based document that explains what I was doing and why.

It can be hard to force myself to do the clean-up. I wish there were an easier way. But I expect that well-organized lab scientists devote a lot of time to constructing their lab notebooks, too.

Advertisements

Tags: , ,

7 Responses to “Electronic lab notebook”

  1. Jarle Pahr Says:

    The github tutorial link is dead.

  2. Me Says:

    Karl, very nice post. While I think that the idea of using make is intriguing, I’m not sure I understand how this would be better than a following workflow using knitr like:

    1. write an article in R markdown and place it on github
    2. scripts that are long are placed outside of the main document and then sourced from within the main document (inside an R chunk, usually cached)
    3. handle bibliography through bibtex, directly or indirectly (meaning to using knitcitations and cito and convert to bibtex once you need to submit)
    4. tables, graphics, inline results all generated through r chunks, inline or not, with packages such as xtable or simply subsetting model objects
    5. use pandoc to convert from rmd to md to latex and then pdf or whatever might be needed for submission

    notice that I’m not criticizing the use of make, just trying to understand what the advantages would be over a “pure knitr” workflow like the one above

    • Karl Broman Says:

      I agree that you can do most everything with knitr, especially as you can have code chunks in basically any language, including shell commands.

      But I still think make has several advantages.

      1. It’s easier to remember the command make than Rscript -e 'library(knitr);knit("myfile.Rnw")'.

      2. Your Makefile can specify the relevant commands to create the main document, as well as that using pandoc to convert your Rmd file to latex.

      3. The Makefile is a bit more explicit about the dependencies among files: which are the original data sources and code files, and how each processed file is created from each other one.

      4. With make, you only compile the bits that need to be compiled, given the changes. I suppose you could do that within a knitr document, but you’d probably have to code those relationships yourself.

      Some of my projects have a lot of different things going on (e.g., this paper, with this crazy Makefile), and I find make to be a quite natural way to organize all the different tasks.

  3. Electronic Lab Notebook: Take 1 | Daniel J. Hocking Says:

    […] knitr, make, and GitHub for a lab notebook […]

  4. ebw Says:

    What about
    https://pythonhosted.org/Sumatra/
    or
    http://bbcf.epfl.ch/bein/index.html#
    as computationnal lab notebook. It use version control to keep track of the version used to run a computation and keep also the parameter file.

  5. Reproducible data analysis and mind-blowing dynamic lab notebook entries using knitr and markdown in R – David Schmidt Says:

    […] Electronic lab notebook (..) […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s