High Performance Applications for R

Overview

zNetR is highly scalable, elastic, and fault tolerant parallel backend for applications written in the R programming language. The zNetR includes the doznet R package and the zEngineR application server. doznet offers the application developer a foreach construct with a parallel backend for distributing R functions to a cluster of computers running zEngineR servers. By utilizing the doznet package, R applications automatically obtain advanced distributed computing capabilities such as:

  • service discovery
  • fault-tolerance
  • real-time load balancing
zNetR supports variety of deployment options including:
  • private and public clouds
  • clusters
  • grid and super computers

Visit our Amazon EC2 store to access zNetR application servers.

Why zNetR?

  • Fastest time to market
    • Simple API - open source foreach and iterators packages to run multiple iterations in parallel via zNetR backend
    • Self-discoverable zEngineR computation engines
    • Drop-in installation of the computation engines
    • Operations support via web and cli GUI with open API
  • Proven Technology
    • Auto R Data file upload onto "commodity" out-of-the-box zEngineR computation engines
    • Data Cache for frequently used data, supports run-time refresh and synch across all nodes
    • Dynamic Load Balancing, optimally loads all available hardware to maximize ROI
    • Fastest execution, zNetR technology minimizes network overhead, passing arguments only, not function bodies
  • Near linear Performance improvement on commodity hardware
  • Cloud Enabled - e.g., Amazon EC2, GoGrid, and RackSpace

Target Audience

R programmers maintaining or developing software that requires high performance characteristics and which can benefit from parallelizing foreach constructs.

Features & Benefits

The following table compares zNetR with two other R-based parallelization engines:

Feature

zNetR

doSnow

doMPI

Use foreach () package

Fault tolerant

Grid Management

Support arbitrary complex "R" functions

Distributed R Data Cache

Supports elastic hardware

Load Balancer

Real time

limited

limited

Security API bindings

 

zNetR provides the following capabilities to accelerate and scale up R applications:

  • Pre-built R Application Server (zEngineR). The zEngineR executable is part of installation. It is capable of programmatically loading any custom R package, discovering and announcing its functions and executing them on request from the client application.
    • Self-Discoverable. No coding required for service location or specification. Just start the number of required instances at any time.
    • Safe and Predictable. If a server fails to load the R data file, zNetR will not send any invocation to it, thereby avoiding runtime errors.
    • Elastically Scalable. zEngineR can be started or stopped at any point during the client R application activity. It automatically loads the proper R file and is added to the distributed invocations.
  • Parallel backend for R Functions of arbitrary complexity. The zNetR backend uses the doznet foreach package to execute multiple R functions remotely on distributed zEngineR servers by routing functions parameters to the least loaded server. Correspondent R files are loaded onto distributed zEngineR servers programmatically, supporting execution of arbitrary complex functions.
    • Fail Safe. In case of hardware or network failure zNetR automatically re-routes failed requests to available zEngineR application servers, ensuring that client computations are completely processed.
    • High-Efficiency. When execution times of distributed R functions fluctuate, zNetR ensures best utilization of all available hardware by routing most work to the least loaded servers.
    • Distributed Cache. Function variables that are not changed between invocations are optimized by zNetR framework, delivered only once to each zEngineR server, and stored there for subsequent retrieval.

Technical Specs

  • Built-in High Performance Queues (HPQ) for inter-process communication
  • Binary protocols supporting all R data types
  • 40 micro sec avg. latency (5K messages)
  • Built-in programming patterns
    • Broker Messaging
    • Load Balancer with real-time feedback
    • Distributed cache (zCache)
  • R Application Server (zEngineR) in COTS package

Amazon EC2

zNetR is available on Amazon EC2 via two Linux-based Amazon Machine Instances (AMIs) supporting both 32 and 64 bit Linux platforms.

zNetR images are identified by their manifest names as

  • zNet-R-Linux_x86_64.manifest.xml (64-bit version)
  • zNet-R-Linux_x86_32.manifest.xml (32-bit version)
Review Key steps to launch zNetR cloud or download detailed step by step instructions (Download PDF)
  • Select Viewing Filter  to [“Public Images”] type [zNet-R]

 

 

  • Follow AWS Instance launch wizard

 

(!) Important Notes:

  • The very first instance is considered to be a root / startup instance
  • Startup instance will be automatically configured with zNetR controller services , pre-defined projects and configuration folders
  • zNetR Computation node(s) must be launched after startup instance is up and running and has internal established IP address
  • During zNetR Computation node launch make sure to specify startup internal IP address in User Data text box, see image below

 

zNetR cloud validation:
  • Change folder to /deployment/ZirconProjects/znetr/R_vector_creation
  • On successful execution, results will be written in Results.log file as shown below

 

 

 

Download detailed step by step instructions (PDF Download)