Bench Press

The Crossroads of Science and Tech

Moving Online

with 23 comments

Neil Saunders recently wrote about the the difficulty some people find in keeping an electronic notebook; over the past year, during my rotations, I’ve used both paper and electronic notebooks, and I’ve come to one conclusion: whatever you can make electronic, make it so. Trust me, it’s worth the effort.

First, a bit of context. I’m an experimental biologist, not a computational biologist (at least, not primarily computational), so I have to deal with the fact that not everything in my research can be “electronified.” I’ve got samples in the freezer, vials of things in the fridge, cultures going in the incubators, and so on. Thus, I’m not looking for a notebook where I can keep every single bit of my research; maybe when the Matrix Google finally digitizes all of reality, I can finally just plug into my computer and never leave my desk.

On the other hand, though I’m not a computational biologist, I am pretty comfortable with computers. I program, I know HTML, and I can use Photoshop pretty quickly. I’m fine with cobbling together my own electronic lab notebook of sorts from the tools I can find on the internet (such as making a wiki); others (such as some in my lab) might find even formatting a wiki post to be an intimidating prospect. So what I do might not work for you.

Right now, my lab has an internal wiki using the MediaWiki engine, and I’ve been using my wiki space as my lab notebook. The wiki is backed up regularly, is password-protected, lets me view and edit the wiki from any internet-connected computer, has full-text search built-in, and has an editing format lets me put in cross-references to other entries; this is all I generally need for a lab notebook.

How do I manage the offline stuff, like gel pictures, data sheets, and so on? Well, I can put a surprising number of things on the wiki (for example, the lab next door lets me use their UV transilluminator, which has a CCD camera that I use to save TIFF images of my gels onto my network account), but basically I keep everything in a giant binder, numbered in order by date, and then put a reference to it in the online notebook. That way, I can easily find a result; just search the wiki for the entry I’m looking for, look up the reference number, and then find it in my binder of results. I do something similar with my experimental samples; my initials, followed by an experiment number, and then a vial number (such as “EJS-109-10″).

Unlike some people, my note-taking philosophy is that the lab notebook shouldn’t necessarily be a dirty log of absolutely everything that I do or think about while I’m in the lab; I don’t put in routine calculations or procedures, such as cell culture maintenance, making common reagents, and so on. The whole point of a lab notebook is so that I can keep track of what I did, so if I, or someone else, needs to look up what I did, or what’s in a vial in the freezer, I don’t have to spend hours trying to remember what I meant with “RfMQ2-3a-4-5-07″ on a tiny tube cap.

Sure, it’s sometimes nice to have everything in one lab notebook, gels pasted in and so on, but frankly, I find that even just the organizational benefits of being able to read my own handwriting and being able to search and cross-reference my posts to be worth giving up the all-in-one solution. When I kept a paper notebook, I spent so much time flipping back and forth between pages trying to remember where I’d written the concentration for the vial in my hand. Now, I just search for the vial number and voila!

Not only that, but I like to organize my lab notebook by project, rather than chronologically, because I generally have more than one thing going on at the same time. Organizing that way is doable on paper if you use a binder and loose-leaf paper, but still quite a hassle, especially since some experiments don’t always fit cleanly in one project or another, making it hard to find later (“Did I file it under this project or the other one?”). I prefer the electronic notebook, which lets me organize by both time and project, and lets me put experiments under more than one lab notebook by simply putting a link to it from both project pages. Not only that, but if I’m repeating an experiment with slightly different conditions, I can simply copy-and-paste a previous experiment and change just a few things.

Paper just doesn’t cut it for me anymore. If I were keeping a paper notebook, I’d basically be doing all the same things as I do on my electronic notebook: a loose-leaf notebook organized by projects, numbered experiments, separate binder of raw data, cross-referencing based on page numbers or experiment numbers, and so on. My online notebook does all this and adds remote access, readability, copy-and-paste, and searching to boot.

Not only that, but because my lab notebook is online, if I’m writing a paper or making slides for a presentation, I can be lazy and work from home. Win for the web!

Written by Eric

September 4th, 2008 at 4:58 am

  • http://usefulchem.blogspot.com Jean-Claude Bradley

    We’ve also been using a wiki as lab notebook for 2.5 years and it has stood the test of time. Since our aim is to make our data public we make extensive use of GoogleDocs, Flickr, YouTube, etc. A blog linking back to individual lab notebook pages on the wiki makes it easy to discuss higher level concepts while providing links to the fine details should anyone care to probe further.

  • http://www.benchside.com Eric

    Yeah; my workflow, I think, is significantly easier because I’m not writing the notebook for public consumption; it makes dealing with non-text data much easier by letting me keep most of that offline.

  • http://psiqueii.blogspot.com laura

    first of all, only four posts in and already a cool blog – impressive :)

    i’m starting my PhD and supervisor told me to keep a log of meetings i attend etc. which got me thinking i could as well open up another blog in which to post these things, instead of keeping a word file, eg in order to be able to incorporate links to stuff i looked up after a discussion. not sure how supervisor will feel about me writing about experiments or thoughts re projects, but might do it anyway, too.

    anyway, sorry about the rant.

    point is: do you think that keeping a blog as notebook is feasible? what would be the advantages of a wiki?

  • http://hublog.hubmed.org/ alf

    What do you find are the advantages of using a wiki over something more blog-like such as WordPress?

  • http://humans.scienceboard.net Alethea

    I’ve been having the same thoughts. What you write about even wiki markup being problematic for some is true. Jean-Claude, I gave wikispaces a look and a definite advantage is the visual editor and ease of image upload. On the other hand, a definite advantage of the OWW platform is automatic entry creation by clicking on a date. What I haven’t found in either is a way of retrieving chronologically ordered entries by tag (where I would then specify a “project” in the way JCB does by Experiment), but I know it could be possible. That would be pretty damn useful as we have a lot of different things going on, some of which crosses into two ostensibly separate projects.

    GoogleDocs is perfect for what we need, JCB; thanks for putting me onto that. I knew about it but not that it was so easy to link out.

    I’m still mulling over my list and priority order of desiderata but overall, I agree with your whole assessment, Eric.

    I wanted to ask both of you (if JCB comes back for more) how you back up a wiki? I was wondering if I shouldn’t just print out my entries once a month for my administration, who is quite reticent to have us move off paper as they’ve spent large amounts of money on serially numbered formal lab notebooks with room for countersignatures and the like. But that would stink wrt all the links, and the source is illegible.

  • http://www.benchside.com Eric

    It comes down to personal preference, I think. Each has there own advantages. A good blogging system can do almost everything a wiki can do, especially with the static pages that most platforms seem to support (such as WordPress).

    A blog is probably better for organizing entries by tags, categories, or chronology. With a wiki, you have to organize the chronology by hand; it’s more flexible, but it’s also not automatic. If you have a lot of entries, though, a blog probably becomes pretty unwieldy to browse through unless you have a really good search. I prefer organizing everything by hand, so that I can make it exactly the way I want it. Tags would be nice, but it’s not a deal-breaker for me.

    A blog would also be better for those who find editing wikis to be intimidating, since many of them have WYSIWYG interfaces; as Alethea mentioned, though, wikispaces is quite nice for those who have a hard time with the syntax, because the buttons and such are right there.

    I like wikis, because I make a lot of really short entries over the course of a day, which would be a little bit more painful for a blog; it probably depends on your blog’s visual layout, whether that would work for you. Not only that, but a wiki is helpful for working together in a lab, since the wiki can become a one-stop location for all the information in the lab. That would be a little harder to do with a blog.

    Wikis are also better for keeping track of revisions and changes. A few blogs have that sort of ability, including WordPress, but it’s not as easy as with a wiki.

    Really, I think it depends on the style and function you want out of a lab notebook.

    As for backing up a wiki, it’s the same as with backing up a blog. I usually make a dump of the backend database onto a hard drive; that’s only for those who are comfortable with a command line, I’m afraid. In addition, the lab’s wiki is on a university server, and the IT department has set up automatic backup of the entire server for every 2 hours.

    I don’t know if I’d print out wiki pages; if you’re like me and you go back and add incremental notes to different pages, it would take forever (and lots of paper!) to print out all the pages with changes.

  • http://blog.benchside.com/ Ben

    Laura, make it snarky, and let “anonymous visitors” comment :-)

    Eric took the words right out of my mouth. Although it seems that blogs and Wiki’s are increasingly converging in terms of functionality, I think it boils down to blogs being more single-person endeavors (in the sense that each entry hypothetically “belongs” to an author) while a Wiki is a more multi-person endeavor (in the sense that each entry is expected to be viewed and edited by multiple people).

    And there is one other difference, although it’s easily changeable, I detest how standard Wiki interfaces look. Search is a must-have feature, but someone really needs to teach people how to make better looking Wiki’s…

  • http://blog.openwetware.org/scienceintheopen Cameron Neylon

    We use a blog as a lab notebook and Ben’s comment is interesting as we definitely go for the ‘one person – one blog’ approach. We use a pretty fine grained approach to the organisation which I think is useful and have some automated stuff built in. We also tend to use OpenWetWare for actually sharing protocols and finalised things within the group.

    Overall I think we get a bit hung up about blogs vs wikis. They are both fundamentally PHP running over SQL (or something similar). The differences are ones of expectation rather than underlying technology. I think its pretty clear the best system would take some of the strengths of both and put them together in a useful way. I would really like to look at the same underlying database in a ‘wiki-view’ and ‘blog-view’ to be honest – with each view bringing different things out.

  • http://www.benchside.com Eric

    I don’t think wikis and blogs are the same technology. The fact that they both generate web pages from SQL data is what makes them possible choices for lab notebooks at all; that’s kind of the bare minimum criterion for an online lab notebook in the first place, and thinking on that reductionistic level is just too coarse-grained to make useful judgment calls on what technology to use.

    The interfaces of wikis and blogs are significantly different, and each is suited for a different style of content management. A wiki makes it easy to make a deeply-linked network of pages, while a blog makes it easy to create a set of chronologically ordered entries.

    I do agree, though, that something that’s sort of a hybrid would be nice…

  • Pingback: Odds of losing ur Lab notebook vs ur Enotebook « My Weblog on Bioinformatics, Consed, Phrap, Genome science.

  • http://humans.scienceboard.net Alethea

    “I usually make a dump of the backend database onto a hard drive; that’s only for those who are comfortable with a command line, I’m afraid.”

    Could you point me in the direction of a tutorial? Even if my stuff is on OpenWetWare, I’d like to have a local copy in case they suddenly disappear (eg. if fascists get elected in November and decide to do completely away with free speech in Boston).

    Seriously, it’s a bit of a problem for me, as my blog is hosted by scienceboard.net which gives me WYSIWYG in exchange for my not having access to the actual database of entries, not being able to modify the look of the blog, not entering tracking codes or anything like that. With some WordPress functionalities nonetheless. Sigh. But Nature Network who kindly seduced me to write a blog with them, gives you about the same flexibility without WordPress, so not even WYSIWYG.

    /rant

  • http://www.benchside.com Eric

    Here’s a tutorial, but it requires access to the actual MySQL database server, which I’m not sure OWW would give you. The best thing is probably to talk to Bill Flanagan; he might be able to work something out with you.

  • http://www.axiope.com Rory Macneil

    We at Axiope will next week launch private beta testing of e-CAT, an OLN which has been specifically designed to enable non technical scientists to carry out experiments and manage data online. When our updated website goes live there will be a ‘Links’ section on the homepage where we link to interesting current discussions about online notebooks and related topics. I think this Moving Online thread is a good presentation of what is currently possible using wikis and blogs, and the issues with the two approaches. Would you have any objection if we linked to the thread on our homepage?

    Thanks.

    Rory

  • http://www.axiope.com Rory Macneil

    Sorry about that previous comment – number 14. It was posted in error. I would be grateful if you could remove it.

    Thanks.

  • http://www.benchside.com Eric

    Sure, link it up, no problem.

  • http://www.axiope.com Rory Macneil

    Thanks, Eric, that’s great. I will let you know when we go live.

  • http://usefulchem.blogspot.com Jean-Claude Bradley

    As for back-ups, Wikispaces gives you the option of dumping the whole site into a zip file either as HTML or Wikitext at the click of a button.

    The choice of blog or wiki depends a lot of what your requirements are. The ability to track versions is critical for us since an experiment can take some time to completely report and interpret.

    Just like in a paper notebook, a table of contents is a small price to pay for quickly accessing experiments and easily done in a wiki.

    For a quick look at some screenshots of what I am talking about see slides from my talk in Manchester yesterday:
    http://www.slideshare.net/jcbradley/manchester-open-notebook-science-talk-presentation/

  • http://humans.scienceboard.net Alethea

    Thanks, JCB!

  • http://www.axiope.com Rory Macneil

    The e-CAT beta went live this morning. There is a link to Moving Online on the revamped axiope.com homepage, where you can also watch the two introductory videos, and read about e-CAT’s features, as well as the inaugural blog post to see a bit of the history and our aims for e-CAT. Feedback would be much appreciated!

  • Pingback: Corpus callosum: 1st edition of open science round-up « I was lost but now I live here

  • Pingback: Happy birthday Foucault’s Pendulum at Bench Press

  • Pingback: blog.twidox.com » Blog Archive » 1st edition of an open science round-up

  • drgunn

    Just wanted to drop a late comment in here about BioKM. It's a simple web-based online lab notebook started by my friend Johnathan Gross.

    I've blogged protocols before and had an account on open wetware that I've rarely used. Openness is a real advantage of a wiki, but sometimes a purpose-built app just fits better out-of-the-box.