Error notifications from R

4 Sep 2014

I’m enthusiastic about having R notify me when my script is done.

But among my early uses of this, my script threw an error, and I never got a text or pushbullet about that. And really, I’m even more interested in being notified about such errors than anything else.

It’s relatively easy to get notified of errors. At the top of your script, include code like options(error = function() { } )

Fill in the function with your notification code. If there’s an error, the error message will be printed and then that function will be called. (And then the script will halt.)

You can use geterrmessage() to grab the error message to include in your notification.

For example, if you want to use RPushbullet for the notification, you could put, at the top of your script, something like this:

options(error = function() { 
                    library(RPushbullet)
                    pbPost("note", "Error", geterrmessage())
                })

Then if the script gives an error, you’ll get a note with title “Error” and with the error message as the body of the note.

Update: I knew I’d heard about this sort of thing somewhere, but I couldn’t remember where. Duh; Rasmus mentioned it on twitter just a couple of days ago! Fortunately, he reminded me of that in the comments below.

Notifications from R

3 Sep 2014

You just sent a long R job running. How to know when it’s done? Have it notify you by beeping, sending you a text, or sending you a notification via pushbullet.

Read the rest of this entry »

The mustache photo

28 Aug 2014

A certain photo of me has been following me around for some time.

Karl with a mustache, 15 Nov 2002

The thing is sitting on my website, so I suppose I have only myself to blame. I actually quite like the photo. I look happy. I was happy. I’m not always happy.

Read the rest of this entry »

Yet another R package primer

28 Aug 2014

Hadley Wickham is writing what will surely be a great book about the basics of R packages. And Hilary Parker wrote a very influential post on how to write an R package. So it seems like that topic is well covered.

Nevertheless, I’d been thinking for some time that I should write another minimal tutorial with an alliterative name, on how to turn R code into a package. And it does seem valuable to have a diversity of resources on such an important topic. (R packages are the best way to distribute R code, or just to keep track of your own personal R code, as part of a reproducible research process.)

So I’m going ahead with it, even though it doesn’t seem necessary: the R package primer.

It’s not completely done, but the basic stuff is there.

If I could do it over again, I’d self-publish

12 Aug 2014

In 2009, Ĺšaunak Sen and I wrote a book about QTL mapping and the R/qtl software. We started working on it in the fall of 2006, and it was a heck of a lot of work.

We’d talked to several publishers, and ended up publishing with Springer. John Kimmel was the editor we worked with; I like John, and I felt that Springer (or John) did a good job of keeping prices reasonable. We were able to publish in full color with a list price of $99, so that on Amazon it was about $65. (In April, 2013, there was a brief period where it was just $42 at Amazon!)

Springer did arrange several rounds of reviews; they typically pay reviewers $100 or a few books. But the copy editing was terrible (at the very least, you want a copy editor to read the book, and it was pretty clear that our copy editor hadn’t), and the actual type-setting and construction of the index was left to us, the authors.

It feels nice to have written a proper book, but I don’t think it makes that big of a difference, for me or for readers.

And John Kimmel has since left Springer to go to Chapman & Hall/CRC, and Springer has raised the price of our book to $169, so it’s now selling for $130 at Amazon. I think that’s obnoxious. It’s not like they’ve gone back and printed extra copies, so it’s hard to see how their costs could have gone up. But in the publishing agreement we signed, we gave Springer full rights to set the price of the book.

I have a hard time recommending the book at that price; I’m tempted to help people find pirated PDFs online. (And seriously, if you can’t find a pirated copy, you should work on your internet skills.)

I corresponded with an editor at Springer, on why our book has become so expensive and whether there’s anything we can do about it. They responded

  • If we do a new edition, it could be listed as $129.
  • If the book is adopted by university classes, “the pricing grid it is based on would have lower prices.”
  • Our book is available electronically, for purchase by chapter as well.

Purchase by chapter? Yeah, for $30 per chapter!

Springer has published books and allowed the authors to post a PDF, but only for really big sellers, and ours is definitely not in that category.

I’m both disgusted and embarrassed by this situation. If I could do it all over again, I’d self-publish: post everything on the web, and arrange some way for folks to have it printed cheaply.

Testing an R package’s interactive graphs

1 Aug 2014

I’ve been working on an R package, R/qtlcharts, with D3-based interactive graphs for quantitative trait locus mapping experiments.

Testing the interactive charts it produces is a bit of a pain. It seems like I pretty much have to just open a series of examples in a web browser and tab through them manually, checking that they look okay, that the interactions seem to work, and that they’re not giving any sort of errors.

But if I want to post the package to CRAN, it seems (from the CRAN policy) that the examples in the .Rd files shouldn’t be opening a web browser. Thus, I need to surround the example code with \dontrun{}.

But I was using those examples, and R CMD check, to open the series of examples for manual checking.

So, what I’ve decided to do:

  • Include examples opening a browser, but within \dontrun{} so the browser isn’t opened in R CMD check.
  • Also include examples that don’t open the browser, within \dontshow{}, so that R CMD check will at least check the basics.
  • Write a ruby script that pulls out all of the examples from the .Rd files, stripping off the \dontrun{} and \dontshow{} and pasting it all into a .R file.
  • Periodically run R CMD BATCH on that set of examples, to do the manual checking of the interactive graphs.

This will always be a bit of a pain, but with this approach I can do my manual testing in a straightforward way and still fulfill the CRAN policies.

Update: Hadley Wickham pointed me to \donttest{}, added in R ver 2.7 (in 2008). (More value from blog + twitter!)

So I replaced my \dontrun{} bits with \donttest{}. And I can use devtools::run_examples() to run all of the examples, for my manual checks.

UseR 2014, days 3-4

21 Jul 2014

Three weeks ago, I’d commented on the first two days of the UseR 2014 conference. I’m finally back to talk about the second half.

Dirk Eddelbuettel on Rcpp

Dirk Eddelbuettel gave a keynote on Rcpp [slides]. The goal of Rcpp is to have “the speed of C++ with the ease and clarity of R.” He gave a series of examples that left me (who still uses .C() to access C code) thinking, “Holy crap this is so much easier than what I do!”

Take a look at the Rcpp Gallery, and the slides from Dirk’s Rcpp tutorial.

Dirk ended with a detailed discussion of Docker: a system for virtual machines as portable containers. I didn’t fully appreciate this part, but according to Dirk, Docker “changes how we build and test R….It’s like pushing to GitHub.”

Sponsors Talk

After Dirk’s talk was the Sponsor’s Talk. But if I’m going to skip a session (and I strongly recommend that you skip at least some sessions at any conference), anything called ”Sponsor’s Talk“ is going to be high on my list to skip.

Lunch at Venice Beach

Karthik Ram and I met up with Hilary Parker and Sandy Griffith for lunch at Venice Beach.

It took us a bit longer to get back than we’d anticipated. But I did get a chance to meet up with Leonid Kruglyak at his office at UCLA.

R and reproducibility

David Smith from Revolution Analytics and JJ Allaire from RStudio each spoke about package management systems to enhance reproducibility with R.

For your R-based project to be reproducible, the many packages that you’ve used need to be available. And future versions of those packages may not work the same way, so ideally you should keep copies of the particular versions that you used.

David Smith spoke about the R reproducibility toolkit (RRT). The focus was more on business analytics, and the need to maintain a group of versioned packages that are known to work together. CRAN runs checks on all packages so that they’re all known to work together. As I understand it, RRT manages snapshots of sets of packages from CRAN.

JJ Alaire spoke about RStudio‘s solution: packrat. This seems more suitable for scientific work. It basically creates privates sets of packages, specific to a project.

I’ve not thought much about this issue. packrat seems the best fit for my sort of work. I should start using it.

Poster session

The second poster session was in a different location with more space. It was still a bit cramped, being in a hallway, but it was way better than the first day. There were a number of interesting posters, including Hilary’s on testdat, for testing CSV files; Sandy’s on using Shiny apps for teaching; and Mine Çetinkaya-Rundel and Andrew Bray’s poster on “Teaching data analysis in R through the lens of reproducibility“ [pdf].

Met more folks

The main purpose of conferences is to meet people. I was glad to be able to chat with Dirk Eddelbuettel, Ramnath Vaidyanathan, and also Tim Triche. Also karaoke with Sandy, Karthik, Hilary, Rasmus, and Romain.

Wish I’d seen

I had a bit of a late night on Wednesday night, and then I was in a hurry to get down (via public transit!) to the Science Center to meet up with my family. So I’m sorry that I didn’t get to see Yihui Xie‘s talk on Knitr Ninja.

Looking back through the program, there are a number of other talks I wish I’d seen:

Summary

UseR 2014 was a great meeting. In addition to the packages mentioned with Days 1-2, I need to pick up Rcpp and packrat.

Slides for many of the talks and tutorials are on the UseR 2014 web site. If you know of others, you can add them via the site’s GitHub repository and make a pull request.

Why hadn’t I written a function for that?

16 Jul 2014

I’m often typing the same bits of code over and over. Those bits of code really should be made into functions.

For example, I’m still using base graphics. (ggplot2 is on my “to do” list, really!) Often some things will be drawn with a slight overlap of the border of the plotting region. And in heatmaps with image, the border is often obscured. I want a nice black rectangle around the outside.

So I’ll write the following:

u <- par("usr")
rect(u[1], u[3], u[2], u[4])

I don’t know how many times I’ve typed that! Today I realized that I should put those two lines in a function add_border(). And then I added add_border() to my R/broman package.

It was a bit more work adding the Roxygen2 comments for the documentation, but now I’ve got a proper function that is easier to use and much more clear.

Update: @tpoi pointed out that box() does the same thing as my add_border(). My general point still stands, and this raises the additional point: twitter + blog → education.

I want to add, “I’m an idiot” but I think I’ll just say that there’s always more that I can learn about R. And I’ll remove add_border from R/broman and just use box().

2014 UseR conference, days 1-2

2 Jul 2014

I’m at UCLA for the UseR Conference. I attended once before, and I really enjoyed it. And I’m really enjoying this one. I’m learning a ton, and I find the talks very inspiring.

In my comments below, I give short shrift to some speakers (largely by not having attended their talks), and I’m critical in some places about the conference organization. Having co-organized a small conference last year, I appreciate the difficulties. I think the organizers of this meeting have done a great job, but there are some ways it which it might have been better (e.g., no tiny rooms, a better time slot for the posters, and more space for the posters).

Read the rest of this entry »

hipsteR: re-educating people who learned R before it was cool

15 May 2014

This morning, I started a tutorial for folks whose knowledge of R is (like mine) stuck in 2001.

Yesterday I started reading the Rcpp book, and on page 4 there’s an example using the R function replicate, which (a) I’d never heard before, and (b) is super useful.

I mean, I often write code like this, for a bootstrap:

x <- rnorm(2500)
sapply(1:1000, function(a) quantile(sample(x, replace=TRUE), c(0.025, 0.975)))

But I could just be writing

x <- rnorm(100)
replicate(1000, quantile(sample(x, replace=TRUE), c(0.025, 0.975)))

“Oh, replicate must be some new function.” Yeah, new in R version 1.8, in 2003!

I’m in serious need of some re-education (e.g., I should be using more of Hadley Wickham’s packages). Hence the beginnings of a tutorial.


Note: the title was suggested by Thomas Lumley. No connection to @hspter; it’s not really so hip. I probably should have written “geeksteR.”


Follow

Get every new post delivered to your Inbox.

Join 89 other followers