Running Matlab in Parallel on a Windows Cluster using compiled Matlab* Code and the Matlab Compiler Runtime (MCR)

  • [アーティクル]

 

General Overview:

The MATLAB Compiler uses the MATLAB Compiler Runtime (MCR), a standalone set of shared libraries that enables the execution of MATLAB files on computers without an installed version of MATLAB.  For many HPC users, this provides a means for Matlab to be executed across many nodes of a system for commonly used Matlab applications/codes that do not require interaction and are used repeatedly.

 The other, more traditional method of running parallel Matlab on a cluster, is to use the Mathworks “Parallel Computing Toolkit” which is part of the MATLAB Distributed Computing Server (MDCS) environment.  This type of architecture is more robust in particular for development and single/short time execution of a model(s). It allows the user to run their parallel MATLAB applications from within the MATLAB client.

  

USING MATLAB ON AN HPC CLUSTER VIA COMPILED MATLAB CODE AND THE
MATLAB Compiler Runtime (MCR):

 

For many users that want to run a matlab code repeatedly and/or at large scales on an HPC cluster, they choose to run compiled Matlab applications leveraging the MCR. 

 

The general work flow for this type of operation is as follows:

 

  1. A user will generate, test, and compile their Matlab code using the Matlab client on their desktop/workstation.
  2. To run on the HPC cluster, the user will copy (if not available to the cluster via some type of share storage) their Matlab executable and data they wish to execute to the HPC system.
  3. They will submit their application to the HPC cluster as a “Job” which will consist of one or more “tasks”  (note, for an Windows HPC cluster a “job” is comprised of one or ore “tasks” which are individual groups of work – usually an application/script that operates on a set of data).
  4. The HPC scheduler will execute the user’s “Job” which includes their Matlab executable as a task or a set of tasks (usually just the command line .exe of their Matlab code) on the compute resources the user requested as part of their job submission.
  5. The compiled Matlab executables are then initiated and run on each compute node, leveraging the locally installed MATLAB Compiler Runtime (MCR), typically enerating output files to local storage.
  6. Depending on configuration of the system, the output data will be collected/moved to a shared storage device (if not already writing to a shared device), the head
    node, and generate another form of output.
  7. User collects results, many times copying to their local machine for further analysis and data reduction.

General Setup:

  1. Install the MCR on each of the nodes (many ways to do this)
    1. Use ClusterCopy to copy installer to each node and then run the installer silently basic on the Mathworks instructions,
    2. RDP directly to each node, copy and install,
    3. Ceate an HPC job with a script to copy and install the MCR silently,
    4. Use a fileshare that is accessible by each node.
  2. Copy input data and compiled matlab application to each node (or have them available to each node via a fileshare).
  3. Execute via clusrun utility ( as shown in the link https://technet.microsoft.com/en-us/library/cc947685(WS.10).aspx) and/or submit job/task to the scheduler via
    gui, command line, or powershell (as shown in the link https://technet.microsoft.com/en-us/library/ee783562(WS.10).aspx).

Comments

  • Anonymous
    May 08, 2014
    Pingback from matlab together guess on several nodes | Search R
  • Anonymous
    May 12, 2014
    Pingback from matlab together guess on several nodes | Wonda