HP XC System 4.x Software - Manual (Page 53)

variable that was set by LSF; this environment variable is equivalent to the number provided by
the -n option of the bsub command.
Any additional SLURM srun options are job specific, not allocation-specific.
The mpi-jobname is the executable file to be run. The mpi-jobname must be compiled with
the appropriate HP-MPI compilation utility. For more information, see the section titled Compiling
applications in the HP-MPI User's Guide.
Example 5-7 shows an MPI job that runs a hello world program on 4 cores on 2 compute
nodes.
Example 5-7 Submitting an MPI Job
$ bsub -n4 -I mpirun -srun ./hello_world
Job <24> is submitted to default queue .
<>
<>
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 3 of 4 on host2
Hello world! I'm 0 of 4 on host1
Hello world! I'm 2 of 4 on host2
You can use the LSF-SLURM External Scheduler option to add capabilities at the job level and
queue level by including several SLURM options in the command line. For example, you can
use it to submit a job to run one task per node, or to submit a job to run on specific nodes.
“LSF-SLURM External Scheduler” discusses this option.
Example 5-8 shows an MPI job that uses the LSF-SLURM External Scheduler option to run the
same hello world program on each of 4 compute nodes.
Example 5-8 Submitting an MPI Job with the LSF-SLURM External Scheduler Option
$ bsub -n4 -ext "SLURM[nodes=4]" -I mpirun -srun ./hello_world
Job <27> is submitted to default queue .
<>
<>
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 2 of 4 on host2
Hello world! I'm 3 of 4 on host3
Hello world! I'm 4 of 4 on host4
Some preprocessing may need to be done:
• If the MPI job requires the use of an appfile
• If there is another reason that prohibits the use of the srun command as the task launcher
This preprocessing should determine the node host names to which mpirun's standard task
launcher should launch the tasks. In such scenarios, you need to write a batch script; there are
several methods available for determining the nodes in an allocation. One method is to use the
SLURM_JOBID
environment variable with the squeue command to query the nodes. Another
method is to use the LSF environment variables such as LSB_HOSTS and LSB_MCPU_HOSTS,
which are prepared by the HP XC job starter script.
5.3.3 Submitting a Parallel Job Using the SLURM External Scheduler
The LSF-SLURM external scheduler option provides additional capabilities at the job level and
queue level by allowing the inclusion of several SLURM options in the LSF command line. With
LSF integrated with SLURM, you can use the LSF-SLURM External Scheduler to specify SLURM
5.3 Submitting a Parallel Job 53