HP XC System 3.x Software - Manual (Page 49)

to the number provided by the -n option of the bsub command. Any additional SLURM srun options are
job specific, not allocation-specific.
The mpi-jobname is the executable file to be run. The mpi-jobname must be compiled with the appropriate
HP-MPI compilation utility. Refer to the section titled
Compiling applications in the HP-MPI User's Guide for
more information.
Example 5-7 shows an MPI job that runs a hello world program on 4 cores on 2 compute nodes.
Example 5-7 Submitting an MPI Job
$ bsub -n4 -I mpirun -srun ./hello_world
Job <24> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on lsfhost.localdomain>>
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 3 of 4 on host2
Hello world! I'm 0 of 4 on host1
Hello world! I'm 2 of 4 on host2
You can use the LSF-SLURM External Scheduler option to add capabilities at the job level and queue level
by including several SLURM options in the command line. For example, you can use it to submit a job to run
one task per node, or to submit a job to run on specific nodes.
"LSF-SLURM External Scheduler" discusses
this option.
Example 5-8 shows an MPI job that uses the LSF-SLURM External Scheduler option to run the same hello
world program on each of 4 compute nodes.
Example 5-8 Submitting an MPI Job with the LSF-SLURM External Scheduler Option
$ bsub -n4 -ext "SLURM[nodes=4]" -I mpirun -srun ./hello_world
Job <27> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on lsfhost.localdomain>>
Hello world!
Hello world! I'm 1 of 4 on host1
Hello world! I'm 2 of 4 on host2
Hello world! I'm 3 of 4 on host3
Hello world! I'm 4 of 4 on host4
Some preprocessing may need to be done:
•
If the MPI job requires the use of an appfile
•
If there is another reason that prohibits the use of the srun command as the task launcher
This preprocessing should determine the node hostnames to which mpirun's standard task launcher should
launch the tasks. In such scenarios, you need to write a batch script; there are several methods available
for determining the nodes in an allocation. One method is to use the SLURM_JOBID environment variable
with the squeue command to query the nodes. Another method is to use the LSF-HPC environment variables
such as LSB_HOSTS and LSB_MCPU_HOSTS, which are prepared by the HP XC job starter script.
Submitting a Batch Job or Job Script
Use the following bsub command format to submit a batch job or job script:
bsub -n num-procs [bsub-options] script-name
The -n num-procs parameter, which is required for parallel jobs, specifies the number of cores the job
requests.
The script-name argument is the name of the batch job or script. The script can contain one or more
srun
or mpirun commands.
The script will execute once on the first allocated node, and the srun or mpirun commands within the script
will be run on the allocated compute nodes.
Submitting a Batch Job or Job Script 49