Difference between revisions of "How to Build GridPACK"

Revision as of 17:00, 17 October 2018

Overview

This section will provide a brief overview of how to configure and build GridPACK and its associated libraries. More detail can be found by looking at builds of GridPACK on specific systems. These can be found by following the links below. These examples provide complete instructions for building GridPACK and its associated libraries. We strongly recommend that you find an example that is similar to the system you are planning on building on and examine the commands for that system before attempting to build GridPACK. Users will need to make sure that they have a compiler on their system (we have used GNU and Intel compilers to build GridPACK), an MPI library and CMake (version 2.8.8 or greater).

More general information on configuring the GridPACK build can be found in the sections below and additional information on libraries used by GridPACK can be found here.

• Mac OS X (High Sierra) with MacPorts
• CentOS or RHEL 7
• CentOS or RHEL 6
• Ubuntu Linux
• Debian Linux
• PNNL PIC Cluster (Linux cluster with Infiniband)

If you run into problems, feel free to contact us for further help. You can also look at our troubleshooting page.

Prerequisite Software

Currently, GridPACK builds on Linux/UNIX sytems. Other operating systems are not supported at this time.

Building GridPACK can be complicated, primarily because it depends on several third-party software packages. These need to be built and installed prior to building GridPACK. Refer to the list of required software for what is needed. Detailed information on building these packages on different platforms is available on the links listed above for building GridPACK on different platforms. More information on the individual libraries can be found on the software required to build GridPACK page.

GridPACK requires the MPI, Global Arrays, Boost, PETSc and Parmetis libraries to build. The Parmetis libraries can usually be downloaded and built when configuring and building PETSc and this is the preferred way of obtaining Parmetis. If necessary, Parmetis can be downloaded and built separately.

Some version of MPI is already available on most clusters, but users may need to build their own copy of MPI if running on a workstation. Several versions of MPI, including OpenMPI, MPICH and MVAPICH are available for free. There are also commercial implementations, such as Intel MPI, that can be used.

In general, it is a good idea to build Boost, Global Arrays and PETSc yourself. This guarantees that all libraries are using the same compiler and version of MPI. Compilers that have been used to build GridPACK include the GNU and Intel compilers. The instructions for building these libraries vary from one platform to the next. Users should consult the list of builds above and find one that most resembles the platform you wish to build on. The scripts in these builds can then be adapted for your system. For Linux workstations and clusters, the builds for a Redhat workstation or an Infiniband cluster are a good place to start. Builds for CentOS, Debian and Ubuntu Linux are also available. For the Apple computers, we have a MacPorts build that has been demonstrated on the Yosemite OS. These builds may require modifications if your system differs from those listed, but modifications should be small. If you have problems, feel free to contact us for additional help.

Configuring GridPACK

Configuration is the most complicated part of the process of actually building GridPACK once all the libraries listed above are available. CMake is used to configure GridPACK for building. CMake is usually available on most Linux systems but older versions of Linux may have a version of Cmake that is too old to build GridPACK. GridPACK requires version 2.8.8 or newer. If the installed version of CMake is too old, users will need to download CMake and build it themselves. The version of CMake can be found by typing

 cmake -version

The configure process insures that required software is available and usable. CMake expects to configure GridPACK in a directory other than the one containing the source code. Typically, one makes an empty directory, usually called something like build, cd's into this directory, and executes the command

 cmake [options] gridpack/source/directory

where options are used to locate required software and set compiler options. The shell script example_configuration.sh shows some examples of configuration options for a few systems. If you don't get the configure right the first time, then you should make sure that you get rid of all the files that CMake created when you tried configuring previously. This can be done by typing

 rm -rf CMake*

in your build directory. This will remove all CMake-related configuration files from your build directory so that the new build is not corrupted by the previous build. If you are using a script to configure GridPACK, then you should include this line at the start of the script.

To guarantee that CMake finds the correct C and C++ compilers, you should define the environment variables CC and CXX (if you are trying to build the Fortran interface, you should define FC and F77 as well). In addition, it may also be necessary to define CFLAGS = "-pthread" etc. depending on how some of the other libraries were built. Using C-shell, the environment variables are

 setenv CC gcc
 setenv CFLAGS "-pthread"
 setenv CXX g++
 setenv CXXFLAGS "-pthread"
 setenv FC gfortran
 setenv FCFLAGS "-pthread"
 setenv F77 gfortran
 setenv F77FLAGS "-pthread"

A complete configuration line for GridPACK is

 cmake -Wdev \
     -D BOOST_ROOT:STRING='$HOME/software_new/boost_1_55_0' \
     -D PETSC_DIR:STRING='$HOME/software_new/petsc-3.6.0' \
     -D PETSC_ARCH:STRING='linux-openmpi-gnu-cxx' \
     -D PARMETIS_DIR:STRING= \
       '$HOME/software_new/petsc-3.6.0/linux-openmpi-gnu-cxx/lib' \
     -D GA_DIR:STRING='$HOME/software_new/ga-5-4-ib' \
     -D USE_PROGRESS_RANKS:BOOL=FALSE \
     -D GA_EXTRA_LIBS='-lrt -libverbs' \
     -D MPI_CXX_COMPILER:STRING='mpicxx' \
     -D MPI_C_COMPILER:STRING='mpicc' \
     -D MPIEXEC:STRING='mpiexec' \
     -D CMAKE_INSTALL_PREFIX:PATH='$GRIDPACK/src/build/install' \
     -D CMAKE_BUILD_TYPE:STRING='RELWITHDEBINFO' \
     -D MPIEXEC_MAX_NUMPROCS:STRING="2" \
     -D CMAKE_VERBOSE_MAKEFILE:STRING=TRUE \
     ..

This example assumes that the build directory is immediately below the GRIDPACK/src directory so that the .. at the end of this command are pointing to GRIDPACK/src. The directories that are arguments to options like BOOST_ROOT and PETSC_DIR should be replaced with locations of these libraries on your system. More information on the CMake options is available on the required software page as well as the GridPACK overview document (see the section on configuring and building GridPACK). Additional examples can also be found in the pages describing builds on different platforms.

The Boost install directory is specified with the BOOST_ROOT option, the PETSc install directory and PETSc architecture are specified with the PETSC_DIR and PETSC_ARCH options. If ParMETIS has been downloaded and built as part of PETSc, it is unnecessary to specify the location of the ParMETIS installation and the PARMETIS_DIR option does not need to be included. If it ParMETIS has been installed separately, then the location of the ParMETIS and METIS libraries is specified by PARMETIS_DIR.

The installation directory for Global Arrays can be specified using the GA_DIR option. Extra libraries and compiler flags that may be needed by GA can be specified using the GA_EXTRA_LIBS option. For MPI-based runtimes, this option usually is not needed. If GA was built with the progress ranks runtime (--with-mpi-pr), then the USE_PROGRESS_RANKS option must be included and set to true. Otherwise, it can be left out or set to false.

The MPI compilers and execute commands are specified in the cmake command with the options

   -D MPI_CXX_COMPILER:STRING='mpicxx' \
   -D MPI_C_COMPILER:STRING='mpicc' \
   -D MPIEXEC:STRING='mpiexec' \

These options set the MPI wrappers for both the C and C++ compilers.

Some GridPACK-specific options are available for controlling the configuration:

Building

Once configured, GridPACK is built with

 make

which will take some time. (The make command should be available by default on most Linux platforms.) If building on a multi-core system, building can go faster if multiple cores are use, e.g.,

 make -j 8

will use 8 simultaneous processes to build GridPACK. It is possible that you may run into problems building with multiple cores since some dependencies may get out of order. If the build stops, then try continuing using only 1 core.

Running Tests

After a successful build, GridPACK unit tests can be run with

 make test

or

 ctest

at the top of the GridPACK build directory. This will produce a list of tests and whether they passed or failed. While every effort is taken to make sure these tests pass, several things can lead to test failure, not just errors in GridPACK code, including OS, compilers, and requisite software versions. We cannot test all permutations. Typical output from ctest might look like this (abridged example):

   Running tests...
   /usr/bin/ctest3 --force-new-ctest-process 
   Test project /home/d3g096/Projects/GridPakLDRD/GridPACK.github/master/src/build
         Start  1: greetings_serial
    1/87 Test  #1: greetings_serial .....................   Passed    0.27 sec
         Start  2: greetings_parallel
    2/87 Test  #2: greetings_parallel ...................   Passed    0.32 sec
   [...]
         Start 68: state_estimation_serial
   68/87 Test #68: state_estimation_serial ..............   Passed    0.30 sec
         Start 69: state_estimation_parallel
   69/87 Test #69: state_estimation_parallel ............***Failed    0.33 sec
         Start 70: kalman_ds_serial
   70/87 Test #70: kalman_ds_serial .....................   Passed    2.88 sec
         Start 71: kalman_ds_parallel
   71/87 Test #71: kalman_ds_parallel ...................***Failed    0.45 sec
         Start 72: hello_world_serial
   72/87 Test #72: hello_world_serial ...................   Passed    0.29 sec
         Start 73: hello_world_parallel
   73/87 Test #73: hello_world_parallel .................   Passed    0.34 sec
   [...]
   
   98% tests passed, 2 tests failed out of 87
   
   Total Test time (real) =  67.29 sec
   
   The following tests FAILED:
            69 - state_estimation_parallel (Failed)
            71 - kalman_ds_parallel (Failed)
   Errors while running CTest

In most cases, there is a serial and parallel versions of each test. Note that, in this example, two tests were reported as failed and the reason was very helpfully reported as Failed. If the test is marked as Failed without any further information, it could mean the test crashed and did not run to completion or it did run to completion and the output indicated it failed.. Very often tests are reported to fail because of a Timeout, which means the test ran, but longer than was allowed. If this happens, the test timeout needs to be increased. When configuring GridPACK, add an option to cmake like

   -D GRIDPACK_TEST_TIMEOUT:STRING=30

which indicates any unit is only allowed to run for 30 seconds before is killed. Another common failure is reported by ctest as Required regular expression not found. This means that the test ran to completion, but the output was not recognized as a correct. Most of the time this indicates a real error, but sometimes, particularly with parallel tests,

If lots of tests fail, there is probably some problem with the GridPACK configuration or some other system setup (like MPI peculiarities). If only a few tests fail, they can probably be ignored, unless the test failure indicates some functionality of importance to you. If a test fails, it possible that a bug report has been made. Check current issues to see.

Failed tests can be diagnosed by running them individually and looking at the output. For example, to run test 2 only, run ctest at the top of the GridPACK build directory like this

   ctest -VV -I 2,2

This produces something like

   r/src/build/DartConfiguration.tcl
   UpdateCTestConfiguration  from :/home/d3g096/Projects/GridPakLDRD/GridPACK.github/master/src/build/DartConfiguration.tcl
   Test project /home/d3g096/Projects/GridPakLDRD/GridPACK.github/master/src/build
   Constructing a list of tests
   Done constructing a list of tests
   Checking test dependency graph...
   Checking test dependency graph end
   test 2
       Start 2: greetings_parallel
   
   2: Test command: /usr/lib64/openmpi/bin/mpiexec "-n" "4" "greetings"
   2: Test timeout computed to be: 9.99988e+06
   2: I am process 0 of 4.
   2: 
   2: Creating communicators using split:
   2: 
   2: I am process 1 of 4.
   2: I am process 2 of 4.
   2: I am process 3 of 4.
   2: I am process 0 (original process is 0) of 2 on communicator 0.
   2: I am process 1 (original process is 1) of 2 on communicator 0.
   2: I am process 0 (original process is 2) of 2 on communicator 1.
   2: I am process 1 (original process is 3) of 2 on communicator 1.
   2: 
   2: Creating communicators using divide:
   2: 
   2: I am process 0 (original process is 2) of 2.
   2: (assignment) I am process 0 (original process is 2) of 2.
   2: (copy) I am process 0 (original process is 2) of 2.
   2: I am process 1 (original process is 3) of 2.
   2: (assignment) I am process 1 (original process is 3) of 2.
   2: (copy) I am process 1 (original process is 3) of 2.
   2: I am process 1 (original process is 1) of 2.
   2: (assignment) I am process 1 (original process is 1) of 2.
   2: (copy) I am process 1 (original process is 1) of 2.
   2: I am process 0 (original process is 0) of 2.
   2: 
   2: Testing assignment operator for communicators:
   2: 
   2: (assignment) I am process 0 (original process is 0) of 2.
   2: 
   2: Testing copy constructor for communicators:
   2: 
   2: (copy) I am process 0 (original process is 0) of 2.
   2: 
   2: Testing summation operator for communicators
   2: 
   2: 
   2: Summation test for communicators passed
   2: 
   1/1 Test #2: greetings_parallel ...............   Passed    0.31 sec
   
   100% tests passed, 0 tests failed out of 1
   
   Total Test time (real) =   0.32 sec

If you wish to report a bug about a failed unit test, run the test as described and post it with the bug report.

Running the Powerflow Example(s)

If desired, the powerflow example application can be run on a number of different networks. Starting from the top GridPACK build directory, change into the powerflow application build directory:

   cd applications/powerflow

Run the IEEE 14 bus problem serially using

   ./pf.x input_14.xml

or in parallel using

   mpiexec -np 2 ./pf.x input_14.xml

(but don't use too many processors). Input for other networks are available:

• input_118.xml: a (IEEE?) 118 bus case
• input_polish.xml: Polish network model with 3120 buses and 3684 branches
• input_european.xml: European network model with 13659 buses and 18625 branches

for which more processors can be deployed.

Obsolete Build Cases

The GridPACK build descriptions listed here are considered obsolete or less applicable than those described above. They still, however, may include useful information to those trying to build GridPACK and have conditions that don't exactly match the descriptions above.

• Mac OS X (Yosemite) with MacPorts ()
• Red Hat Enterprise Linux Workstation