HP XC System 4.x Software - Manual (Page 86)

The LSF environment is set up automatically for the user on login; LSF commands and their
manpages are readily accessible:
• The bhosts command is useful for viewing LSF batch host information.
• The lshosts command provides static resource information.
• The lsload command provides dynamic resource information.
• The bsub command is used to submit jobs to LSF.
• The bjobs command provides information on batch jobs.
10.2 Overview of LSF Integrated with SLURM
LSF was integrated with SLURM for the HP XC system to merge the scalable and efficient resource
management of SLURM with the extensive scheduling capabilities of LSF. In this integration:
• SLURM manages the compute resources.
• LSF performs the job management.
SLURM extends the parallel capabilities of LSF with its own fast parallel launcher (which is
integrated with HP-MPI), full parallel I/O and signal support, and parallel job accounting
capabilities.
Managing the compute resources of the HP XC system with SLURM means that the LSF daemons
run only on one HP XC node and can present the HP XC system as a single LSF host. As a result:
• All the nodes are configured as LSF Client Hosts; every node is able to access LSF. You can
submit jobs from any node in the HP XC system.
• The lshosts and bhosts commands only list one host that represents all the resources of
the HP XC system.
LSF integrated with SLURM obtains resource information about the HP XC system. This
information is consolidated and key information such as the total number of cores and the
maximum memory available on all nodes becomes the characteristics of the single HP XC “LSF
Execution Host”. Additional resource information from SLURM, such as pre-configured node
“features”, are noted and processed during scheduling through the external SLURM scheduler
for LSF.
Integrating LSF with SLURM on HP XC systems provides you with a parallel launch command
to distribute and manage parallel tasks efficiently. The SLURM srun command offers much
flexibility for requesting requirements across an HP XC system; for example, you can request
• Request contiguous nodes
• Execute only one task per node
• Request nodes with specific features
This flexibility is preserved in LSF through the external SLURM scheduler; this is discussed in
more detail in the section titled
“LSF-SLURM External Scheduler” (page 92)
A SLURM partition named lsf is used to manage LSF jobs. Thus:
• You can view information about this partition with the sinfo command.
• The total number of cores listed by the lshosts and bhosts commands for that host should
be equal to the total number of cores assigned to the SLURM lsf partition.
When a job is submitted and the resources are available, LSF creates a properly-sized SLURM
allocation and adds several standard LSF environment variables to the environment in which
the job is to be run. The following two environment variables are also added:
SLURM_JOBID This environment variable is created so that subsequent srun commands
make use of the SLURM allocation created by LSF for the job. This variable
can be used by a job script to query information about the SLURM allocation,
as shown here:
$ squeue --jobs $SLURM_JOBID
86 Using LSF