Composing functions and sequencing operations are core programming concepts.
Some notable realizations of sequencing or pipelining operations include:
- CMS Pipelines.
F#‘s forward pipe operator
- Haskel’s Data.Function
The idea is: many important calculations can be considered as a sequence of transforms applied to a data set. Each step may be a function taking many arguments. It is often the case that only one of each function’s arguments is primary, and the rest are parameters. For data science applications this is particularly common, so having convenient pipeline notation can be a plus. An example of a non-trivial data processing pipeline can be found here.
[Related Article: Validating Type I and II Errors in A/B Tests in R]
The ideas are:
wraprdot arrow pipe includes a detailed
S4configurable interface (detailed in the RJournal here). These interfaces are able to treat objects as functions: i.e. they can pipe data into objects.
wrapr::UnaryFnclass supplies a convenient tool for the partial function application needed to work with pipelines.
Or: pipe notation assumes a world data transforms are single argument functions (with other parameters already bound in), and the
UnaryFn derived classes we discuss here help realize such a world.
This can be made clearer with examples.
Suppose we build a linear model of
log(y) as follows.
We can see our predictions in original
y-units by making the prediction and then exponentiation:
It is natural to want to apply a model later to new data. This can be done as follows.
wrapr package allows us to use a “piping into a function” notation as follows.
In the above example the
model contents are captured in the function closure. However, it is better practice to explicitly store data in objects.
wrapr supplies a method to do this, which we will now demonstrate.
model_o is an object (not a function). However we can pipe into
model_o as if it were a function.
This works because
model_o is derived from the
wrapr has definitions for
apply_left.UnaryFn, which integrate this class into the
wrapr dot-arrow pipe. The family of
UnaryFn classes single argument functions. This system happens to be implemented by
wrapr dot arrow extension mechanisms also allow users to build their own pipe-compatible systems. (
S4 extension details can be found in the RJournal here.
The pipe notation is not strictly required as the apply is done through the
The above methods can be used to wrap substantial functions such as
vtreat::prepare() to create very powerful data processing pipelines. A more involved example of trying this technique can be found here.
wrapr right-dispatch we are using is only triggered when the right-hand side of a pipeline is a symbol or name. This is consistent with pipelines such as “
5 %.>% sin” where we are not so much piping into the sin-function, but into a name that refers to the sin-function. However, piping into names covers most practical cases.
We can apply processing pipelines piece by piece.
We can also build such a pipeline by piping pieces into each other.
The pipe notation is not required (but is a nice notation). The apply a list of function objects effect can be achieved directly with
The idea is: the processing pipelines store an arbitrary number of function objects as a simple list. The list declares function-like behavior to both
ApplyTo and the
wrapr dot-arrow pipe through
S4 class declarations. Function objects do not capture environments as function closures do (though obviously any function in them does have its own closure). List of function objects can be easier to work with, store, and share than function closures or other pipeline structures.
We can look at the contents of a pipeline as follows.
In addition to the
PartialNamedFn class we suggest looking at the following additional adapters:
srcfn()which accepts the source code for an arbitrary expression (quoted either with quote-marks or with
wrapfn()class which directly accepts a function (including the closure).
The idea is
wrapr supplies many possible variations of notations: functions sequences as lists, function composition by pipe, function composition by call, function application by pipe, and function application by call. Then the user can pick what notation they prefer.
rquerypipelines are very restricted (they date
data.frames and pre-check a number of invariants).
UnaryFn pipelines are more free-form, they check very little before application.
[Related Article: Introduction to R Shiny]
The demonstrated design and functionality is inspired by partially applied functions, but a bit more circumspect in what is carried around. In Lisp “code is data”, in
R it is a bit more complicated- so a pure-data solution has some merits.
And these are the basics of
wrapr function objects. For a more substantial data processing example please see here.
Originally Posted Here