What is shinytest and Why Should You Care?

At work, I am part of a team that develops and maintains in-house R packages and shiny apps for distributed energy modeling. The ability to easily develop and iterate the modeling code alongside the web app– full-stack analytics development, all in R– allows us to move fast to support our sales engineers as they help create the growing, distributed energy industry.

As our tools have become more business critical, we need to demonstrate that we’re building “production” software. This has always been a chip on the shoulder of every R developer, and is made worse by historic projects at my company, however the R community, strongly led by RStudio, is continuously pushing back on this image with tools designed for robustness. Production software is a series of best-practices and processes, and is not a specific feature of a language.

The shinytest package is a project in active development led by the excellent team at RStudio, with the heaviest contribution from Winston Chang and Gábor Csárdi. It provides a means for automating functional tests for a shiny app and running them with a headless browser (specifically PhantomJS). By adopting automated testing, a developer can feel more confident they are releasing a high-quality, defect-free application.

Initial Thoughts and Motivation for an Example

As with all their projects, the folks at RStudio have written excellent documentation on shinytest to help users get up and running quickly. This is what I read to get started, and I highly recommend you read that page first.

After reading the “getting started” page, I was able to record my first test and run it. However, our internal apps make heavy use of htmlwidgets as a means of embedding high-quality javascript elements into the shiny app. In particular, we display most of our results with the DataTables widget to make them interactive and capture selections for drilling-down on particular scenario configurations.

The RStudio “Testing in Depth” article notes that the htmlwidgets pose some challenges for capturing inputs, as these elements don’t have shiny input bindings, however the shinytest developers have done some great work to mitigate the impact to mainly the screenshots. However, looking at my expected output json from my first recorded test, I realized that the snapshot had captured the DataTable metadata but not the actual values I was hoping to test.

After some trial-and-error and review of some of the shiny documentation, I was able to explicitly export the reactive dataframe with my values that I wanted to test as part of my shiny app, such that I am able to view the data that is displayed in the DataTable. I have created a small, reproducible example here to explain how.

An Example App for Testing

I have created a trivial shiny app to demonstrate how I’m testing my DataTable values with shinytest. You can download the code here, or just look at the files I reference via GitHub.

The app displays a 10 row DataTable and provides a slider for the user to scale up or down the right column. My simple test will load the app, set the slider to a value of 7, and then test that the output in the DataTable matches my expected values.

DataTable and the Test Recorder

I created my first test using the shinytest test recorder. The test recorder runs your app within the recorder app as an iframe, and then captures the changes you make to the inputs. You then tell the test recorder when to take a snapshot of the outputs.

As I mentioned above, the test recorder worked well and I was able to create my first test, but the output generated by the DataTable widget did not include my data. You can see the first test I recorded in ./tests/recordedDTTest.R and the first expected output json file in ./tests/recordedDTTest-expected/001.json. As you can see, by comparing the json file to the screenshot shinytest takes with the snapshop, the output json captures the DataTable metadata but not the actual reactive dataframe data.

Using shiny::exportTestValues to Capture DataTable Data

I realized that I needed to explicitly control what was being captured in my expected output json, and so I leveraged the exported values mentioned in the “Testing in Depth” article.

The shiny package has a function, exportTestValues() that allows a user to specify named expressions to be exported when the app is in test. The shinytest package, uses the shiny test mode and allows you to explicitly capture these exported expressions as an element in the snapshot for comparison. The expressions can even include reactive values!

FYI, It’s possible to programmatically check when a shiny app is in test mode by checking the option isTRUE(getOption("shiny.testmode")). This is also an option shinytest must look for.

The code for my scaledData() reactive dataframe that I send to the DataTable to render, as well as the code to export the value for testing is shown below:

scaledData <- reactive({
  dummyData[, "y"] <- dummyData[, "y"] * input$scale
  return(dummyData)
})

# To actually test the DT values, we need to export them
exportTestValues(scaledData = scaledData())

By explicitly capturing my scaledData values, my test runs properly by adjusting the slider input and checking for the correct table values against the exported json. You can see the new test code in ./tests/customDTTest.R and the new expected values in the ./tests/customDTTest-expected/001.json file.

Moving Forward with Development

By leveraging this ability to capture the data outputs I care the most about, I could now add more complicated logic to my shiny app and re-run the tests to see if my output has been affected. This allows me to feel comfortable that my most important results are protected against unexpected regressions, and also forces me to explicitly acknowledge when I’ve changed the output of my app by forcing me to change the expected test results.

Hopefully this sample application of shinytest with shiny::exportTestValues() helps you to write production-quality apps with peace of mind.