Building on CentOS6

This page is currently under development. Please excuse inconsistencies.

This build was done on a VirtualBox virtual machine with a clean install of CentOS 6.9. The compiler and library packages available for CentOS 6.9 are quite old, so it is necessary to build GridPACK prerequisites from source and newer versions need to be avoided. This build was done with the a stock compiler (GNU 4.4), but, it in at least one other case, GridPACK was built using Devtoolset-4 (GNU 5.3).

In this description, each of the prerequisite software libraries is stored in its own directory. This is the approach taken in the documentation on installing on a Linux cluster. An alternative is to have all libraries and include files in the same directories. This approach is used in the documentation for building on a workstation using Redhat Linux. The instructions below can be adapted to use this second approach with relatively little effort. Directories or architecture variables that should be set to reflect local user environments are colored red.

We recommend that users put the configuration commands described below into a script instead of typing them in on the command prompt. This minimizes the chance of errors and makes it easier to fix problems if a mistake is made. Some information on how to set up scripts can be found here.

Requisite System Software

Some needed software can be installed with system packages:

sudo yum install environment-modules openmpi openmpi-devel gcc-c++ cmake bzip2-devel git

Before continuing, it will be necessary to either open another terminal and proceed there, or log out and log in. This makes the module command available for use.

OpenMPI is installed as a "module". In order to use the compiler wrappers the OpenMPI "module" needs to be "loaded":

module load openmpi-x86_64

If this does not work, then OpenMPI can be built by hand. To do this download an OpenMPI tarfile from the download site and unzip it into a directory in your home file area. Some additional information on how to do this can be found here. In this example, we store the MPI libraries in a local user directory, but if you have administrative privileges, you can install it in system directories.

After cd'ing into the OpenMPI directory, configure MPI using the script

  ./configure --enable-shared=no --enable-static=yes \
      --prefix="/home/admin/Desktop/software/openmpi-2.1.2/install" CC=gcc CXX=g++ FC=gfortran
  make -j 4
  make install

You will then need to add the MPI compiler wrappers, libraries and man pages to your environment. This can be done in the c-shell using the commands

  setenv PATH /home/admin/Desktop/software/openmpi-2.1.2/install/bin:${PATH}
  setenv MANPATH /home/admin/Desktop/software/openmpi-2.1.2/install/share/man:${MANPATH}
  setenv LD_LIBRARY_PATH /home/admin/Desktop/software/openmpi-2.1.2/install/lib:${LD_LIBRARY_PATH}

Alternatively, if you are using the Bourne shell, the corresponding commands are

  export PATH=/home/admin/Desktop/software/openmpi-2.1.2/install/bin:${PATH}
  export MANPATH=/home/admin/Desktop/software/openmpi-2.1.2/install/share/man:${MANPATH}
  export LD_LIBRARY_PATH=/home/admin/Desktop/software/openmpi-2.1.2/install/lib:${LD_LIBRARY_PATH}

Boost, version 1.54.0

Modern versions of Boost will probably not work here (especially Serialization and MPI). An older version is needed and we recommend 1.54.0. If you are using the OpenMPI module, make sure to load it before running the following script.

   rm -f ~/user-config.jam
   rm -f ./tools/build/v2/user-config.jam
   echo "using mpi : mpicxx ;" >> ./tools/build/v2/user-config.jam
   sh ./bootstrap.sh \
       --prefix="/home/admin/Desktop/software/boost_1_54_0" \
       --without-icu \
       --with-toolset=gcc \
       --without-libraries=python,log
   ./b2 -a -d+2 link=static stage
   ./b2 -a -d+2 link=static install

If you have any problems building Boost, it is generally a good idea to fix your build script, remove the current boost directory, and obtain a new directory from the Boost tar file. Trying to restart a failed build of Boost does not usually work well.

Global Arrays, version 5.6.5

Modern versions of Global Arrays work fine. In this example, we use GA 5.6.5, but others will work as well. This script should be run in the top-level GA directory

   ./configure \
       --enable-cxx \
       --disable-f77 \
       --enable-i4 \
       --with-mpi \
       --with-mpi-ts \
       --enable-autodetect=yes \
       --prefix="/home/admin/Desktop/software/ga-5.6.5" \
       --without-blas \
       --without-lapack \
       --without-scalapack \
       --enable-shared=no \
       --enable-static=yes \
       MPICC=mpicc MPICXX=mpicxx MPIF77=mpif90 \
       MPIEXEC=mpiexec MPIRUN=mpirun CFLAGS="-pthread" FCFLAGS="-pthread" CXXFLAGS="-pthread"
   make
   make install

PETSc, version 3.6.4

For some reason, more recent PETSc versions were not recognized by the GridPACK CMake configuration, possibly because of the old compiler. This version appears to work.

prefix="$HOME/gridpack"
module load openmpi-1.10-x86_64
PETSC_DIR="$prefix/petsc-3.6.4"
export PETSC_DIR
python ./config/configure.py \
python ./config/configure.py \
    PETSC_ARCH=linux-gnu44-real-opt \
    --with-prefix="$prefix/" \
    --with-mpi=1 \
    --with-cc="mpicc" \
    --with-fc="mpif90" \
    --with-cxx="mpicxx" \
    --with-c++-support=1 \
    --with-c-support=0 \
    --with-fortran=1 \
    --with-scalar-type=real \
    --with-precision=double \
    --with-clanguage=c++ \
    --with-fortran-kernels=generic \
    --with-valgrind=0 \
    --download-superlu_dist \
    --download-parmetis \
    --download-metis \
    --download-f2cblaslapack=1 \
    --download-mumps=0 \
    --download-scalapack=0 \
    --with-shared-libraries=0 \
    --with-x=0 \
    --with-mpirun=mpiexec \
    --with-mpiexec=mpiexec \
    --with-debugging=0
make PETSC_DIR=/files0/perksoft/petsc-3.6.4 PETSC_ARCH=linux-gnu48-real-opt all
make PETSC_DIR=/files0/perksoft/petsc-3.6.4 PETSC_ARCH=linux-gnu48-real-opt test

GridPACK

prefix="$HOME/gridpack"
module load openmpi-1.10-x86_64
cmake -Wdev --debug-trycompile \
      -D GA_DIR:PATH="${prefix}" \
      -D BOOST_ROOT:PATH="${prefix}" \
      -D USE_PROGRESS_RANKS:BOOL=OFF \
      -D PETSC_DIR:PATH="${prefix}/petsc-3.6.4" \
      -D PETSC_ARCH:STRING="linux-gnu44-real-opt" \
      -D MPI_CXX_COMPILER:STRING="mpicxx" \
      -D MPI_C_COMPILER:STRING="mpicc" \
      -D MPIEXEC:STRING="mpiexec" \
      -D USE_GLPK:BOOL=OFF \
      -D MPIEXEC_MAX_NUMPROCS:STRING="4" \
      -D GRIDPACK_TEST_TIMEOUT:STRING=10 \
      -D BUILD_SHARED_LIBS:BOOL=OFF \
      -D CMAKE_BUILD_TYPE:STRING=Debug \
      -D CMAKE_VERBOSE_MAKEFILE:BOOL=TRUE \
      -D CMAKE_INSTALL_PREFIX:PATH="${prefix}" \
      ..
make 
make test