Importing Report Data

Neil Harding

2023-09-13

On the NoviSci Web Platform the report system supports two types of reports:

1. Static HTML - output from RMarkdown using the NoviSci Hugo Template uploaded to the platform via nswpr package
2. Dynamic Report - built via a web based report creation tool with data uploaded via nswpr package.

This article describes importing the data for option 2.

Data Sets

A data set for dynamic reports is a small-ish rectangular block of data that is used to render one or more visualizations on a report page. This can be imported into the platform from an R dataframe.

At a high level, the columns of this data set will represent either subgroups (ie, race, age group, sex, etc.) or values to be plotted/displayed (count, mean, percent, etc).

For more details on the format and granularity of data sets see the Report Data Format article.

Import rectangular data into the platform (line, bar, table, and forest plot data)

When importing data into the platform you will need the following information:

• report_id - Unique identifier for the data set. When you want to update a data set in the future, pass this same key.
• data_set_key - Unique identifier for the data set. When you want to update a data set in the future, pass this same key.
• subgroups - A list of columns that represent the ways the data is divided up (stratified by or filtered by)
• data_set_label - Optional. User friendly name for this data set. This will be displayed to a user on the platform when building a report.
• chart_types - A list of chart types this dataset is intended to support - acceptable values: “line”, “bar”, “boxPlot”, “table”, “map”, “sankey”, and “sunburst”

sex <- c(NA, NA, NA, "M", "M", "M", "F", "F", "F")
x <- c("2010-01-01", "2011-01-01", "2012-01-01", "2010-01-01", "2011-01-01", "2012-01-01", "2010-01-01", "2011-01-01", "2012-01-01")
y <- sample(1:100, length(x))

df <- data.frame(sex, x, y)

nswpr::import_report_data_df(df,
                             report_id = "p0000",
                             data_set_key = "random_values",
                             subgroups = list("sex"),
                             data_set_label = "Random Values",
                             chart_types = list("line", "bar"))

Import nested / JSON data into the platform (sunburst & sankey data)

Sunburst and sankey visualizations have a nested structure that needs to be imported as a list of lists. All of the inputs to the import function are the same as the dataframe version, except the first input is a list of lists.

sunburst_data <- list(
  list( # This list can be repeated for each subgroup combination (ie, 2 more - one for male and one for female)
    sex = NA, # There can be more than one subgroup if we want to have multiple filters/strata options
    types = list( t1 = "Treatment One", t2 = "Treatment Two" ), # This is metadata about the sunburst (labels)
    data = list( # This is the sunburst data itself
      list(
        type = "t1",
        value = 23,
        children = list(
          list( type = "t1", value = 5 ),
          list( type = "t2", value = 7 )
        )
      ),
      list(
        type = "t2",
        value = 58,
        children = list(
          list( type = "t1", value = 3 ),
          list( type = "t2", value = 5 )
        )
      )
    )
  )
)

nswpr::import_report_data_nested(sunburst_data,
                                 report_id = "p0000",
                                 data_set_key = "my_sunburst_data",
                                 subgroups = list("sex"),
                                 data_set_label = "My Sunburst Data",
                                 chart_types = list("sunburst"))

Checking the status of the import

The call to import_report_data queues up the dataset to be processed asynchronously by the platform. Some larger datasets can take longer to process, to check the status of an import you can go to the “Manage Data” page of the report.