Understanding Affinity
MPI process placement and affinity can result in significant performance improvements for MPI applications. The mpiexec command takes a number of arguments which allow job owners to control MPI process placement, affinity, and other run time parameters. For more details on these parameters, see the Windows HPC Command Line Reference. The affinity parameter allows job owners to specify that each rank of their MPI application should be locked to a single core (which can significantly improve performance for certain applications).
If the job scheduler sets affinity, the affinity setting that is specified in the mpiexec command is ignored and applications might experience performance degradation. Configure the job scheduler Affinity Type setting according to your cluster workload.
Job scheduler Affinity Type
The affinity type determines how the job scheduler interacts with your applications to assign processor cores to tasks in a job. If the job scheduler sets affinity for a task, it overrides the affinity that is set by the application. Affinity Type has the following settings:
All jobs: The job scheduler sets affinity for any tasks to which a partial node is allocated (even if the job has exclusive use of the node). This ensures that no two tasks use the same core. This setting is recommended for applications that run many instances per node, such as parametric sweeps and SOA jobs.
Non-Exclusive jobs (recommended): The job scheduler sets affinity for any tasks to which a partial node is allocated, unless the task is in a job that is marked as Exclusive. This setting is recommended for MPI jobs and applications that run exclusively on a node and manage their own affinity, and for non-exclusive parametric sweeps and SOA jobs.
Note
You can use Job Templates to specify the types of jobs that run exclusively on nodes.
No jobs: The job scheduler does not set affinity for any task. Tasks run only on the cores that are assigned to them by your applications. Use this setting if MPI jobs encounter problems when the job scheduler sets affinity, or if your applications manage their own affinity but do not always run exclusively on a node.