2016-10 A transformable markup document format

Many technologies now exist for writing a document in a format that can be transformed into various output formats for sharing. This report proposes that using markup, rather than one of the Markdown languages, is a good fit for writing flexible human- and machine-readable transformable documents. We propose a transformable markup document format written in XML. We also provide several examples of how the document format could be transformed.

Ashley Noel Hinton and Paul Murrell

Download

2016-08 OpenAPI version 0.6

This report describes changes to the OpenAPI architecture in version 0.6. Module host elements were implemented in OpenAPI version 0.5; version 0.6 has extended module hosts so it is possible to provide a module’s host information through a module input. The way a module author specifies a module’s language has also been changed in version 0.6, with the introduction of the language element. This element also allows the author to specify details of the version of the language required.

Ashley Noel Hinton and Paul Murrell

Download

2016-07 OpenAPI version 0.5

The OpenAPI architecture is designed to help people connect with data. The architecture specifies an XML schema for wrapping pieces of data analysis code in modules, and combining modules in pipelines. The architecture also specifies the requirements of OpenAPI glue systems—software which can interpret and execute modules and pipelines (Introducing OpenAPI). In this report we describe the changes to the OpenAPI architecture implemented in version 0.5.

Version 0.3 of OpenAPI introduced a host attribute to the module specification as a method for guaranteeing that module source script requirements would be met (OpenAPI version 0.3). This report describes how the module host attribute has been replaced with host elements in version 0.5. This implementation permits more control over the host types supported. This report describes how vagrant and docker hosts are supported in version 0.5. Version 0.5 of OpenAPI also introduces a persistent result format for saving and sharing the results of executing modules and pipelines.

Ashley Noel Hinton and Paul Murrell

Download

2016-06 An OpenAPI Pipeline for NZ Crime Data

In March 2016, the NZ Herald ran a week-long series on NZ crime data, including an interactive map, which shows the total number of crimes per region (for July 2015 to December 2015). Harkanwal Singh (Data Editor) made the crime data avaiable via github. This report describes the construction of OpenAPI pipelines to further explore the NZ Crime data.

Two previous reports have described the creation of one-off OpenAPI pipelines for CPI data and demonstrated that running an OpenAPI pipeline can be made a very simple task. This report demonstrates the creation of slightly more general-purpose OpenAPI modules and a demonstration that combining modules into a pipeline can be made a very simple task.

Paul Murrell

Download

2016-05 An Improved Pipeline for CPI Data

This report provides an update on a previous report, “An OpenAPI Pipeline for CPI Data”, which described the construction of an OpenAPI pipeline to produce a plot of CPI data. Two issues were identified with the original pipeline: the system requirements necessary to run the pipeline were significant; and the system requirements necessary just to run the OpenAPI glue system, the R package ‘conduit’, while much less, were still a potential obstacle to use of the pipeline by a lay audience. This report demonstrates solutions for both issues: a new “host” mechanism for OpenAPI modules to satisfy module system requirements; and an example of a user-friendly web front-end for running OpenAPI pipelines in ‘conduit’.

Paul Murrell

Download

2016-02 Debugging Display List Internals

This report documents the process of debugging a problem with the recording and replaying of R plots from one R session to another. The purpose of this report is to record the source of the problem, to record the solution to the problem, to explain some of the internal details of recorded R plots, and to demonstrate the ‘hexView’ package for exploring binary blobs.

Paul Murrell

Download

2016-01 OpenAPI version 0.3

The OpenAPI project aims to help people connect with data. OpenAPI is a software architecture for wrapping component pieces of data analysis – datasets, programming scripts, etc. – in reusable modules. These simple modules can be combined in pipelines which describe more complex data analysis workflows. Modules and pipelines are created in XML, and executed in software called a glue system. This report details the improvements made to the OpenAPI architecture between versions 0.1 and 0.3.

Ashley Noel Hinton and Paul Murrell

Download