Using the gnu parallel make capability – HP XC System 3.x Software User Manual

Page 93

Advertising
background image

Determine the address of your monitor's display server, as shown at the beginning of

"Running an X Terminal

Session from a Remote Node"

. You can start an X terminal session using this address information in a bsub

command with the appropriate options. For example:

$ bsub -n4 -Ip srun -n1 xterm -display 14.26.206.134:0.0

Job <159> is submitted to default queue <normal>.

<<Waiting for dispatch ...>>

<<Starting on lsfhost.localdomain>>

The options used in this command are:

-n4

allocate 4 cores

-Ip

interact with the X terminal session

srun -n1

run the job on 1 core

xterm

the job is an X terminal session

-display <address>

monitor's display server address

A remote X terminal session appears on your monitor. The X terminal session job is launched from node
n47

, which is the LSF execution host node. You can view this job using LSF-HPC and SLURM commands.

For example:

$ sinfo

PARTITION AVAIL TIMELIMIT NODES STATE NODELIST

lsf up infinite 2 alloc n[46,48]

$ squeue

JOBID PARTITION NAME USER ST TIME NODES NODELIST

117 lsf hptclsf@ username R 0:25 2 n[46,48]

$ bjobs

JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME

119 lsfadmi RUN norma n48 4*n47 *8.136:0.0 date and time

You can now run some jobs from the X terminal session that you started and make use of the full allocation
within the LSF-HPC node allocation. For example:

$ srun -n4 hostname

n46

n48

n46

n48

$ srun -n2 hostname

n46

n48

Exiting from the X terminal session ends the LSF-HPC job.

Using the GNU Parallel Make Capability

By default, the make command invokes the GNU make program. GNU make has the ability to make
independent targets concurrently. For example, if building a program requires compiling 10 source files,
and the compilations can be done independently, make can manage multiple compilations at once — the
number of jobs is user selectable. More precisely, each target's rules are run normally (sequentially within
the rule). Typically the rules for an object file target is a single compilation line, so it is common to talk about
concurrent compilations, though GNU make is more general.

On non-cluster platforms or command nodes, matching concurrency to the number of cores often works well.
It also often works well to specify a few more jobs than cores so that one job can proceed while another is
waiting for I/O. On an HP XC system, there is the potential to use compute nodes to do compilations, and
there are a variety of ways to make this happen.

One way is to prefix the actual compilation line in the rule with an srun command. So, instead of executing
cc foo.c -o foo.o

it would execute srun cc foo.c -o foo.o. With concurrency, multiple command

nodes would have multiple srun commands instead of multiple cc commands. For projects that recursively
run make on subdirectories, the recursive make can be run on the compute nodes. For example:

$ cd subdir; srun $(MAKE)...

Using the GNU Parallel Make Capability

93

Advertising