Example 5-7 – HP XC System 3.x Software User Manual
Page 51
Example 5-7 Submitting an MPI Job
$ bsub -n4 -I mpirun -srun ./hello_world
Job <24> is submitted to default queue
<
<
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 3 of 4 on host2
Hello world! I'm 0 of 4 on host1
Hello world! I'm 2 of 4 on host2
You can use the LSF-SLURM External Scheduler option to add capabilities at the job level and queue level
by including several SLURM options in the command line. For example, you can use it to submit a job to
run one task per node, or to submit a job to run on specific nodes.
“LSF-SLURM External Scheduler”
discusses this option.
shows an MPI job that uses the LSF-SLURM External Scheduler option to run the same hello
world
program on each of 4 compute nodes.
Example 5-8 Submitting an MPI Job with the LSF-SLURM External Scheduler Option
$ bsub -n4 -ext "SLURM[nodes=4]" -I mpirun -srun ./hello_world
Job <27> is submitted to default queue
<
<
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 2 of 4 on host2
Hello world! I'm 3 of 4 on host3
Hello world! I'm 4 of 4 on host4
Some preprocessing may need to be done:
•
If the
job requires the use of an appfile
•
If there is another reason that prohibits the use of the srun command as the task launcher
This preprocessing should determine the node host names to which mpirun's standard task launcher
should launch the tasks. In such scenarios, you need to write a batch script; there are several methods
available for determining the nodes in an allocation. One method is to use the SLURM_JOBID environment
variable with the squeue command to query the nodes. Another method is to use the LSF-HPC environment
variables such as LSB_HOSTS and LSB_MCPU_HOSTS, which are prepared by the HP XC job starter script.
5.3.3 Submitting a Parallel Job Using the SLURM External Scheduler
The LSF-SLURM external scheduler option provides additional capabilities at the job level and queue level
by allowing the inclusion of several SLURM options in the LSF-HPC command line. With LSF-HPC
integrated with SLURM, you can use the LSF-SLURM External Scheduler to specify SLURM options that
specify the minimum number of nodes required for the job, specific nodes for the job, and so on.
Note:
The SLURM external scheduler is a plug-in developed by Platform Computing Corporation for LSF-HPC;
it is not actually part of SLURM. This plug-in communicates with SLURM to gather resource information
and request allocations of nodes, but it is integrated with the LSF-HPC scheduler.
The format of this option is shown here:
-ext "SLURM[slurm-arguments]"
The bsub command format to submit a parallel job to an LSF-HPC allocation of compute nodes using the
external scheduler option is as follows:
5.3 Submitting a Parallel Job
51