dbcast [OPTION] SRC DEST
Parallel MPI application to recursively broadcast a single file from a global file system to node-local storage, like ramdisk or an SSD.
The file is logically sliced into chunks and collectively copied from a global file system to node-local storage. The source file SRC must be readable by all MPI processes. The destination file DEST should be the full path of the file in node-local storage. If needed, parent directories for the destination file will be created as part of the broadcast.
In the current implementation, dbcast requires at least two MPI processes per compute node, and all compute nodes must run an equal number of MPI processes.
-s
,
--size
SIZE
¶The chunk size in bytes used to segment files during the broadcast. Units like "MB" and "GB" should be immediately follow the number without spaces (ex. 2MB). The default size is 1MB. It is recommended to use the stripe size of a file if this is known.
-h
,
--help
¶Print the command usage, and the list of options available.
mpirun -np 128 dbcast /global/path/to/filenane /ssd/filename
mpirun -np 128 dbcast -s 10MB /global/path/to/filenane /ssd/filename
lfs getstripe /global/path/to/filename
The mpiFileUtils source code and all documentation may be downloaded from <https://github.com/hpc/mpifileutils>