
A Future for R: Available Future Backends
Henrik Bengtsson
Source:vignettes/future-2b-backend.md.rsp
future-2b-backend.Rmd
The future package comes with built-in future backends that leverage the parallel package part of R itself. In addition to these backends, others exist in package extensions, e.g. future.callr, future.mirai, and future.batchtools. Below is an overview of the most common backends that you as an end-user can chose from.
Package / Backend | Features | How futures are evaluated |
---|---|---|
futuresequential
|
📶 ♻️ |
sequentially and in the current R process;
default Example: plan(sequential)
|
futuremultisession
|
📶 ♻️ |
parallelly via background R sessions on current
machine Examples: plan(multisession) and
plan(multisession, workers = 2)
|
futurecluster
|
📶 ♻️* |
parallelly in external R sessions on current, local,
and/or remote machines Examples: plan(cluster, workers = "raspberry-pi") ,
plan(cluster, workers = c("localhost", "n1", "n1", "pi.example.org"))
|
futuremulticore
|
📶 ♻️ |
(not recommended) parallelly via forked R processes on
current machine; not with GUIs like RStudio; not on Windows Examples: plan(multicore) and
plan(multicore, workers = 2)
|
future.callrcallr
|
📶(next) ♻️(next) |
parallelly via transient callr
background R sessions on current machine; all memory is returned when as
each future is resolved Examples: plan(callr) and
plan(callr, workers = 2)
|
future.miraimirai_multisession
|
📶(next) ♻️(next) |
parallelly via mirai
background R sessions on current machine; low latency Examples: plan(mirai_multisession) and
plan(mirai_multisession, workers = 2)
|
future.miraimirai_cluster
|
♻️(next) |
parallelly via mirai
daemons running locally or remotely Example: plan(mirai_cluster)
|
future.batchtoolsbatchtools_lsf batchtools_openlava batchtools_sge batchtools_slurm batchtools_torque
|
📶(soon) ♻️(soon) |
parallelly on HPC job schedulers (Load Sharing Facility [LSF], OpenLava, TORQUE/PBS, Son/Sun/Oracle/Univa Grid Engine [SGE], Slurm) via batchtools; for long-running tasks; high latency |
📶: futures relay progress updates in real-time, e.g. progressr
♻️:
futures are interruptible and restartable; * interrupts are disabled by
default
(next): next release; (soon): in a near-future release