Most users do not need to worry about file systems. However, there are two issues: using NFS (the Network File System) with MPI-IO and using NFS with some automounters. These issues are covered in the next two sections.
To use MPI-IO multihost on NFS file systems, NFS should be version 3, and the
shared NFS directory must be mounted with the ``no attribute caching'' (noac)
option set (the directory cannot be automounted). If NFS is not mounted
in this manner, the following error could occur:
MPI_Barrier: Internal MPI error: No such file or directory
File locking messages
In order to reconfigure NFS to handle MPI-IO properly, the following sequence
of steps are needed (root permission required):
rpcinfo -p `hostname` | grep nfs
for example, there should be a '3' in the second column
fire >rpcinfo -p fire | grep nfs
100003 3 udp 2049 nfs
root >grep epm1 /etc/fstab
gershwin:/epm1 /rmt/gershwin/epm1 nfs bg,intr,noac 0 0
If the ``noac'' option is not present, add it and then remount this
directory on each of the machines that will be used to share MPI-IO files.
root >umount /rmt/gershwin/epm1
root >mount /rmt/gershwin/epm1
root >grep gershwin /etc/mnttab
gershwin:/epm1 /rmt/gershwin/epm1 nfs
noac,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0 0 0 899911504
Automounters are programs that dynamically make file systems available when needed. While this is very convenient, many automounters are unable to recognize the file system names that the automounter itself generates.* For example, if a user accesses a file /home/me, the automounter may discover that it needs to mount this file system, and does so in /tmp_mnt/home/me. Unfortunately, if the automounter on a different system is presented with /tmp_mnt/home/me instead of /home/me, it may not be able to find the file system. This would not be such a problem if commands like pwd returned /home/me instead of /tmp_mnt/home/me; unfortunately, it is all too easy to get a path that the automounter should, but does not, recognize.
To deal with this problem, configure allows you to specify a filter
program when you configure with the option -automountfix=PROGRAM, where
PROGRAM is a filter that reads a file path from standard input, makes
any changes necessary, and writes the output to standard output.
mpirun uses this program to help it find necessary files.
By default, the value of PROGRAM is
sed -e s@/tmp_mnt/@/@gThis uses the sed command to strip the string /tmp_mnt from the file name. Simple sed scripts like this may be used as long as they do not involve quotes (single or double) or use % (these will interfere with the shell commands in configure that do the replacements). If you need more complex processing, use a separate shell script or program.
As another example, some systems will generate paths like
/a/thishost/root/home/username/....which are valid only on the machine thishost, but also have paths of the form
/u/home/username/....that are valid everywhere. For this case, the configure option
-automountfix='sed -e s@/a/.\*/home@/u/home@g'will make sure that mpirun gets the proper filename.