Difference between revisions of "Building on Mac Yosemite"

From GridPACK
Jump to: navigation, search
 
(19 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This system is a relatively new Mac Pro with Mac OS X 10.10 (Yosemite).  The [[Building_on_MacOSX| previous instructions]] for Mac OS X are really not helpful anymore due to changes in the OS.   
+
These instructions have been superseded by those [[Building_on_Mac_High_Sierra| here]].  There still may be useful information, however, so they remain online.  <span style="color:red">Read and follow and your own risk!</span>
 +
 
 +
This system is a relatively new Mac Pro with Mac OS X 10.10 (Yosemite). <span style="color:red">'''You will need super user or sudo privileges for this installation'''</span>.  The [[Building_on_MacOSX| previous instructions]] for Mac OS X are really not helpful anymore due to changes in the OS.   
  
 
== Preparation ==
 
== Preparation ==
  
Create a suitable directory where GridPACK and related packages can be built and resulting binaries installed.  Herein, this directory is referred to as <code>$prefix</code>. In my case it was <code>~/Projects/GridPACK</code>. Adjust this to your personal set up.  
+
Create a suitable directory where GridPACK and related packages can be built and resulting binaries installed.  This directory will subsequently be referred to as <code>$prefix</code>. This example uses <code>~/Projects/GridPACK</code>. Before starting, type:
 
+
  set prefix = ~/Projects/GridPACK
 +
Adjust this to your personal set up. Also, the C-shell is used in this example; some commands may need adjustment if another shell such as the Bourne-shell is used.
  
 
== Compiler Set ==
 
== Compiler Set ==
  
Mac system do not have a compiler out of the box. To get the stock Apple compiler set ([https://en.wikipedia.org/wiki/LLVM LLVM]), XTools must be installed. [https://guide.macports.org/#installing.xcode Here] is a good guide.  
+
Mac systems do not have a compiler out of the box. To get the stock Apple compiler ([https://en.wikipedia.org/wiki/LLVM LLVM]), XTools must be installed. [https://guide.macports.org/#installing.xcode Here] is a good guide.  
  
Care must be taken to use a consistent compiler set when building GridPACK.  GridPACK is a C++ code.  Mac OS X provides two standard C++ libraries on modern systems: <code>libc++</code> (the one Apple wants to support), and <code>libstdc++</code> (that will be phased out).  The stock LLVM C++ compiler will use <code>libc++</code>. GNU compilers, if installed, will use <code>libstdc++</code>. If C++ objects from GNU compilers are mixed with the system compiler, they will not link together, because of conflicting and/or missing symbols from the standard library.  This gets complicated. The best thing to do is just use the same compilers to build GridPACK and any prerequisites.  
+
Care must be taken to use a consistent set of compilers when building GridPACK.  GridPACK is a C++ code.  Mac OS X provides two standard C++ libraries on modern systems: <code>libc++</code> (the one Apple wants to support), and <code>libstdc++</code> (that will be phased out).  The stock LLVM C++ compiler will use <code>libc++</code>. GNU compilers, if installed, will use <code>libstdc++</code>. If C++ objects from GNU compilers are mixed with the system compiler, they will not link together, because of conflicting and/or missing symbols from the standard library.  This gets complicated. By far the best thing to do is use the same compilers to build GridPACK and any associated libraries such as Boost and PETSc.  
  
In addition, there were some other problems had while getting this to work:
+
In addition, some other problems were encountered while getting this to work:
* Boost.MPI compiled with the stock LLVM compiler seemed to have trouble with serialization needed in the GridPACK code.  This may or may not be a real issue. After the trouble was encountered, the  CLang compilers were chosen and used there after.  (NOTE: Boost   does not report Boost.MPI unit test results for LLVM or CLang   compilers on any platform).   
+
* Compiling Boost.MPI with the stock LLVM compiler seemed to create problems with serialization needed in the GridPACK code.  This may or may not be a real issue. After the trouble was encountered, the  CLang compilers were chosen and used there after.  (NOTE: Boost does not report Boost.MPI unit test results for LLVM or CLang compilers on any platform).   
* Global Arrays 5.5 does not appear to work with LLVM or CLang, but the SVN   trunk version does.  Neither version worked as shared libraries, so GA must be built static.   
+
* Global Arrays 5.5 does not appear to work with LLVM or CLang, but the SVN trunk version does.  Neither version worked as shared libraries, so GA must be built static.   
  
After a couple of aborted starts with the LLVM compiler, CLang 3.8 (installed with MacPorts) was chosen, and, after some effort, it built GridPACK.  GNU C++ should be avoid.
+
After a couple of aborted starts with the LLVM compiler, CLang 3.8 (installed with MacPorts) was chosen, and, after some effort, it built GridPACK.  GNU C++ should be avoided.
  
 
== MacPorts ==
 
== MacPorts ==
Line 35: Line 38:
 
   graphviz @2.40.1_1+pangocairo+x11 (active)
 
   graphviz @2.40.1_1+pangocairo+x11 (active)
  
Doxygen and Graphviz are optional.
+
OpenBLAS was chosen in an attempt to be compatible with [http://julialang.org/|Julia]. Doxygen and Graphviz are optional.
  
 
== Building PETSc ==
 
== Building PETSc ==
  
[http://www.mcs.anl.gov/petsc/index.html PETSc] version 3.7.4 was configured and built as follows:
+
[http://www.mcs.anl.gov/petsc/index.html PETSc] 3.7.4 source was downloaded and unpacked in <code>$prefix</code>.  It was then configured and built as follows:
     setenv PETSC_DIR /Users/d3g096/Projects/GridPACK/petsc-3.7.4
+
     setenv PETSC_DIR "$prefix/petsc-3.7.4"
 
     unsetenv PETSC_ARCH
 
     unsetenv PETSC_ARCH
     ./configure \
+
     python ./configure \
 
         PETSC_ARCH=arch-macosx-clang-real-opt \
 
         PETSC_ARCH=arch-macosx-clang-real-opt \
 
         COPTFLAGS="-O3" \
 
         COPTFLAGS="-O3" \
 
         CXXOPTFLAGS="-O3" \
 
         CXXOPTFLAGS="-O3" \
 
         FOPTFLAGS="-O3" \
 
         FOPTFLAGS="-O3" \
         --with-prefix="/Users/d3g096/Projects/GridPACK" \
+
         --with-prefix="$prefix" \
 
         --with-mpi=1 \
 
         --with-mpi=1 \
 
         --with-cc=/opt/local/bin/mpicc \
 
         --with-cc=/opt/local/bin/mpicc \
Line 73: Line 76:
 
         --with-mpiexec=mpiexec \
 
         --with-mpiexec=mpiexec \
 
         --with-debugging=0
 
         --with-debugging=0
     make PETSC_DIR=/Users/d3g096/Projects/GridPACK/petsc-3.7.4 PETSC_ARCH=arch-macosx-clang-real-opt all
+
     make PETSC_DIR="$prefix/petsc-3.7.4 PETSC_ARCH=arch-macosx-clang-real-opt all
 +
 
 +
Here, OpenBLAS, installed with MacPorts, was used for BLAS and LAPACK. Other packages (ParMETIS, SuperLU_DIST, etc.) were downloaded and built by the PETSc configuration. PETSc can be built with complex support, if desired, by changing the <code>--with-scalar-type=real</code> to <code>--with-scalar-type=complex</code> and changing the value of <code>PETSC_ARCH</code>. 
 +
 
 +
It is possible to use the MacPorts PETSc package for GridPACK. I does not include SuperLU_DIST, however.  This is left as an exercise for the reader.
  
 
== Building Global Arrays ==
 
== Building Global Arrays ==
 +
 +
The trunk SVN version of [http://hpc.pnl.gov/globalarrays/index.shtml|Global Arrays] is required here.  Get the code using
 +
  svn checkout https://svn.pnl.gov/svn/hpctools/trunk/ga $prefix/ga-svn
 +
 +
 +
Configure, build, and install:
 +
  cd $prefix/ga-svn
 +
  ./configure \
 +
    --enable-cxx \
 +
    --disable-f77 \
 +
    --enable-i4 \
 +
    --with-mpi \
 +
    --with-mpi-ts \
 +
    --enable-autodetect=yes \
 +
    --prefix="$prefix" \
 +
    --without-blas \
 +
    --without-lapack \
 +
    --without-scalapack \
 +
    --enable-shared=no \
 +
    --enable-static=yes \
 +
    MPICC=mpicc MPICXX=mpicxx MPIF77=mpif90 \
 +
    MPIEXEC=mpiexec MPIRUN=mpirun NPROCS=4 \
 +
    LDFLAGS="-L/opt/local/lib"
 +
  make
 +
  make install
 +
 +
I had trouble using shared GA libraries, so only static libraries should be built.  Version 5.5 did not work (some problem with process groups).
  
 
== Building and Testing GridPACK ==
 
== Building and Testing GridPACK ==
  
[http://www.cmake.org/ CMake] projects are designed to be built outside of the source code location.  If the entire GridPACK {{{trunk}}} repository is checked out, the top of the the [http://www.cmake.org/ CMake] project is {{{.../src}}}.  Make a subdirectory there called {{{build}}} or something. Configure and build GridPACK in that directory.
+
At the time of writing, the GridPACK code needed is from a [https://github.com/wperkins/GridPACK fork] other than the main repository.  Some minor changes in that fork will eventually make their way to the [https://github.com/GridOPTICS/GridPACK main repository].  Get the GridPACK code using
 +
  git clone https://github.com/wperkins/GridPACK.git $prefix/src/gridpack
 +
[http://www.cmake.org/ CMake] projects are designed to be built outside of the source code location.  The top of the the [http://www.cmake.org/ CMake] project is <code>$prefix/src/gridpack/src</code>.  Make a subdirectory there called <code>build</code> or something. Configure and build GridPACK in that directory as follows:
 +
  cd $prefix/src/gridpack/src
 +
  mkdir build
 +
  cd build
 +
  setenv CC /opt/local/bin/clang
 +
  setenv CXX /opt/local/bin/clang++
 +
  cmake \
 +
        -D GA_DIR:STRING="$prefix" \
 +
        -D BOOST_ROOT:STRING="/opt/local" \
 +
        -D PETSC_DIR:STRING="$prefix/petsc-3.7.4" \
 +
        -D PETSC_ARCH:STRING="arch-macosx-clang-real-opt" \
 +
        -D MPI_CXX_COMPILER:STRING='/opt/local/bin/mpicxx' \
 +
        -D MPI_C_COMPILER:STRING='/opt/local/bin/mpicc' \
 +
        -D MPIEXEC:STRING='/opt/local/bin/mpiexec' \
 +
        -D MPIEXEC_MAX_NUMPROCS:STRING="4" \
 +
        -D GRIDPACK_TEST_TIMEOUT:STRING=10 \
 +
        -D USE_CPLEX:BOOL=OFF \
 +
        -D USE_GLPK:BOOL=ON \
 +
        -D GLPK_ROOT_DIR:PATH="/opt/local" \
 +
        -D CMAKE_INSTALL_PREFIX:PATH="$prefix" \
 +
        -D BUILD_SHARED_LIBS:BOOL=OFF \
 +
        -D CMAKE_BUILD_TYPE:STRING=Release \
 +
        -D CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE \
 +
        ..
 +
  make
 +
  make test
 +
  make install
 +
 
 +
Note that this configuration is in [https://github.com/wperkins/GridPACK/blob/master/src/example_configuration.sh <code>src/example_configuration.sh</code>] for host <code>WE32673</code>.

Latest revision as of 17:21, 17 October 2018

These instructions have been superseded by those here. There still may be useful information, however, so they remain online. Read and follow and your own risk!

This system is a relatively new Mac Pro with Mac OS X 10.10 (Yosemite). You will need super user or sudo privileges for this installation. The previous instructions for Mac OS X are really not helpful anymore due to changes in the OS.

Preparation

Create a suitable directory where GridPACK and related packages can be built and resulting binaries installed. This directory will subsequently be referred to as $prefix. This example uses ~/Projects/GridPACK. Before starting, type:

 set prefix = ~/Projects/GridPACK

Adjust this to your personal set up. Also, the C-shell is used in this example; some commands may need adjustment if another shell such as the Bourne-shell is used.

Compiler Set

Mac systems do not have a compiler out of the box. To get the stock Apple compiler (LLVM), XTools must be installed. Here is a good guide.

Care must be taken to use a consistent set of compilers when building GridPACK. GridPACK is a C++ code. Mac OS X provides two standard C++ libraries on modern systems: libc++ (the one Apple wants to support), and libstdc++ (that will be phased out). The stock LLVM C++ compiler will use libc++. GNU compilers, if installed, will use libstdc++. If C++ objects from GNU compilers are mixed with the system compiler, they will not link together, because of conflicting and/or missing symbols from the standard library. This gets complicated. By far the best thing to do is use the same compilers to build GridPACK and any associated libraries such as Boost and PETSc.

In addition, some other problems were encountered while getting this to work:

  • Compiling Boost.MPI with the stock LLVM compiler seemed to create problems with serialization needed in the GridPACK code. This may or may not be a real issue. After the trouble was encountered, the CLang compilers were chosen and used there after. (NOTE: Boost does not report Boost.MPI unit test results for LLVM or CLang compilers on any platform).
  • Global Arrays 5.5 does not appear to work with LLVM or CLang, but the SVN trunk version does. Neither version worked as shared libraries, so GA must be built static.

After a couple of aborted starts with the LLVM compiler, CLang 3.8 (installed with MacPorts) was chosen, and, after some effort, it built GridPACK. GNU C++ should be avoided.

MacPorts

The CLang compiler set and some of the prerequisite software was installed on this system using MacPorts. Several necessary packages were installed by

   sudo port install boost boost @1.59.0_2+clang38+no_single-no_static+openmpi+python27

Note that the openmpi Boost variant was installed. This will install the CLang compiler set and OpenMPI. It's a good idea to install the select packages for CLang and MPI and make sure the right versions are "selected":

   sudo port install mpi_select clang_select
   sudo port select --set mpi openmpi-clang38-fortran
   sudo port select --set clang mp-clang-3.8

The following ports were installed:

 cmake @3.7.2_0+docs+gui+python27+qt4 (active)
 boost @1.59.0_2+clang38+no_single+openmpi+python27 (active)
 OpenBLAS @0.2.19_0+clang+lapack (active)
 doxygen @1.8.13_2+qt4+wizard (active)
 graphviz @2.40.1_1+pangocairo+x11 (active)

OpenBLAS was chosen in an attempt to be compatible with [1]. Doxygen and Graphviz are optional.

Building PETSc

PETSc 3.7.4 source was downloaded and unpacked in $prefix. It was then configured and built as follows:

   setenv PETSC_DIR "$prefix/petsc-3.7.4"
   unsetenv PETSC_ARCH
   python ./configure \
       PETSC_ARCH=arch-macosx-clang-real-opt \
       COPTFLAGS="-O3" \
       CXXOPTFLAGS="-O3" \
       FOPTFLAGS="-O3" \
       --with-prefix="$prefix" \
       --with-mpi=1 \
       --with-cc=/opt/local/bin/mpicc \
       --with-fc=/opt/local/bin/mpif90 \
       --with-cxx=/opt/local/bin/mpicxx \
       --with-clanguage=c++ \
       --with-c++-support=1 \
       --with-c-support=0 \
       --with-fortran=0 \
       --with-scalar-type=real \
       --with-precision=double \
       --with-fortran-kernels=generic \
       --with-valgrind=0 \
       --with-blas-lapack-lib="/opt/local/lib/libopenblas.a" \
       --download-suitesparse=1 \
       --download-superlu_dist=1 \
       --download-parmetis=1 \
       --download-metis=1 \
       --download-f2cblaslapack=0 \
       --download-mumps=0 \
       --download-scalapack=0 \
       --with-shared-libraries=0 \
       --with-x=0 \
       --with-mpirun=mpiexec \
       --with-mpiexec=mpiexec \
       --with-debugging=0
   make PETSC_DIR="$prefix/petsc-3.7.4 PETSC_ARCH=arch-macosx-clang-real-opt all

Here, OpenBLAS, installed with MacPorts, was used for BLAS and LAPACK. Other packages (ParMETIS, SuperLU_DIST, etc.) were downloaded and built by the PETSc configuration. PETSc can be built with complex support, if desired, by changing the --with-scalar-type=real to --with-scalar-type=complex and changing the value of PETSC_ARCH.

It is possible to use the MacPorts PETSc package for GridPACK. I does not include SuperLU_DIST, however. This is left as an exercise for the reader.

Building Global Arrays

The trunk SVN version of Arrays is required here. Get the code using

 svn checkout https://svn.pnl.gov/svn/hpctools/trunk/ga $prefix/ga-svn


Configure, build, and install:

 cd $prefix/ga-svn
 ./configure \
   --enable-cxx \
   --disable-f77 \
   --enable-i4 \
   --with-mpi \
   --with-mpi-ts \
   --enable-autodetect=yes \
   --prefix="$prefix" \
   --without-blas \
   --without-lapack \
   --without-scalapack \
   --enable-shared=no \
   --enable-static=yes \
   MPICC=mpicc MPICXX=mpicxx MPIF77=mpif90 \
   MPIEXEC=mpiexec MPIRUN=mpirun NPROCS=4 \
   LDFLAGS="-L/opt/local/lib"
 make
 make install

I had trouble using shared GA libraries, so only static libraries should be built. Version 5.5 did not work (some problem with process groups).

Building and Testing GridPACK

At the time of writing, the GridPACK code needed is from a fork other than the main repository. Some minor changes in that fork will eventually make their way to the main repository. Get the GridPACK code using

 git clone https://github.com/wperkins/GridPACK.git $prefix/src/gridpack

CMake projects are designed to be built outside of the source code location. The top of the the CMake project is $prefix/src/gridpack/src. Make a subdirectory there called build or something. Configure and build GridPACK in that directory as follows:

 cd $prefix/src/gridpack/src
 mkdir build
 cd build
 setenv CC /opt/local/bin/clang
 setenv CXX /opt/local/bin/clang++
 cmake \
       -D GA_DIR:STRING="$prefix" \
       -D BOOST_ROOT:STRING="/opt/local" \
       -D PETSC_DIR:STRING="$prefix/petsc-3.7.4" \
       -D PETSC_ARCH:STRING="arch-macosx-clang-real-opt" \
       -D MPI_CXX_COMPILER:STRING='/opt/local/bin/mpicxx' \
       -D MPI_C_COMPILER:STRING='/opt/local/bin/mpicc' \
       -D MPIEXEC:STRING='/opt/local/bin/mpiexec' \
       -D MPIEXEC_MAX_NUMPROCS:STRING="4" \
       -D GRIDPACK_TEST_TIMEOUT:STRING=10 \
       -D USE_CPLEX:BOOL=OFF \
       -D USE_GLPK:BOOL=ON \
       -D GLPK_ROOT_DIR:PATH="/opt/local" \
       -D CMAKE_INSTALL_PREFIX:PATH="$prefix" \
       -D BUILD_SHARED_LIBS:BOOL=OFF \
       -D CMAKE_BUILD_TYPE:STRING=Release \
       -D CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE \
       ..
 make
 make test
 make install

Note that this configuration is in src/example_configuration.sh for host WE32673.