Using RMarkdown documents in Pelican

by Paul Agapow
in programming

tagged programming, web-development, r, markdown

One of the big sells of Pelican is being able to use multiple markdup formats, being able to drop in notebooks and documents written in other contexts. One such format is RMarkdown (.rmd) documents, such as are used in knitr for reproducible analyses, allowing you to do an analysis and then just render it directly on your website.

However, a number of things can go wrong along the way and Pelican's error messages aren't very helpful. Here's what I had to do to get it working.

Install the reader

An obvious error, but you have to install the rmd_reader to allow RMarkdown documents to be converted. If it isn't installed or Pelican can't find it, you'll get an odd error like:

ERROR: Could not process programming/foo.Rmd

or:

'Pelican does not know how to parse %s', '/content/programming/foo.Rmd'

Edit your pelicanconf.py:

PLUGINS = [
   'foo',
   'rmd_reader',
   'bar',
]

Adding the fortmat to the markup directive may also be necessary:

MARKUP = ('md', 'ipynb', 'rmd')

Problems with R

The reader calls R to convert Rmarkdown to plain markdown. But just because RStudio understands RMarkdown doesn't mean that your native R can. Find out which R you are calling and install the knitr and Rmarkdown packages:

r> install.packages ('knitr')
r> install.packages ('rmarkdown')

Let me guess - R is bugging you about choosing a repository? Just because you've selected one in RStudio doesn't mean that R (under the hood) knows about it:

--- Please select a CRAN mirror for use in this session ---

And there seems be an odd error that sometimes causes this choice to hang. So you can define it in the install call:

install.packages ('knitr', repos='http://cran.us.r-project.org')

Longer term, put this in ~/.Rprofile:

## Default repo
local({r <- getOption("repos")
   r["CRAN"] <- "http://cran.r-project.org"
   options(repos=r)
})

Problems with Python

At this point, you might have Pelican complaining that it can't find metadata (specifically, the title) in the document. How rmd_reader works is that it uses R to convert RMarkdown to Markdown and then uses Python to convert that to HTML. However, more recent version of RMarkdown use a YAML block to detail metadata:

---
title: "Hadley’s vocabulary: part 1 - the basics"
output: html_document
---

and older versions of the Python Markdown library don't understand this, giving complaints like:

ERROR: Skipping programming/r/vocabulary.Rmd: could not find information about 'title'

You can test the version of your Python library in Python:

python> import markdown
python> markdown.version

Versions 2.5+ seem to work fine. If you have something older, update it:

% pip install --upgrade markdown

Various other issues

  • While messing around with various inputs and configurations, Pelican's caching (and the dates of modification) can get in the way, preventing you from seeing changes. It might help to touch the appropriate files and clear the cache (rm -rf cache).
  • The formatting of the metadata block in RMarkdown may be a little strict. It seems that it must bve the very first thing in the file and no blank lines are permitted.
  • Charmingly, the markdown parser in the Atom editor, insists on "correcting" the metadata blocks into a title.