Difference between revisions of "Building on Windows"

From GridPACK
Jump to: navigation, search
(GridPACK)
m
 
(13 intermediate revisions by 2 users not shown)
Line 7: Line 7:
 
* [https://www.visualstudio.com Visual C++] in some form.
 
* [https://www.visualstudio.com Visual C++] in some form.
 
** The "free" ones, Visual Studio Express and Community, should work.
 
** The "free" ones, Visual Studio Express and Community, should work.
 +
** To get the 64-bit compilers for Visual Studio 2015, you need to run the x64 Native Tools Command Prompt (you may need to go into the Visual Studios 2015 folder and then into the Visual Studio Tools subfolder and then into the Windows Desktop Command Prompts folder under that to find it). Once you have opened an x64 Native Command Prompt, cd to C:"\Program Files (x86)\Microsoft Visual Studio 14.0\VC" and type "vcvarsall amd64". More information about this can be found [https://msdn.microsoft.com/en-us/library/x4d2c09s.aspx here].
 
* [https://www.microsoft.com/en-us/download/details.aspx?id=8279 Windows SDK] <br/> Usually, this is just installed with Visual Studio
 
* [https://www.microsoft.com/en-us/download/details.aspx?id=8279 Windows SDK] <br/> Usually, this is just installed with Visual Studio
* [https://cmake.org/download/ CMake] is required
+
* [https://cmake.org/download/ CMake] is required. During the installation process, the dialog will ask if you want CMake added to PATH variable. For some reason, the default is not to include CMake. You should change the selection to add CMake to the PATH for all users.
* A minimal [https://cygwin.com/ Cygwin] installation is necessary. <br/> Cygwin is required to build, and test for, [https://www.mcs.anl.gov/petsc/ PETSc]. If the system has a Cygwin installation that is being used, it would probably be best to make another installation for GridPACK installation exclusively.   
+
* A minimal [https://cygwin.com/ Cygwin] installation is necessary. <br/> Cygwin is required to build, and test for, [https://www.mcs.anl.gov/petsc/ PETSc]. If the system has a Cygwin installation that is being used, it would probably be best to make another installation for GridPACK installation exclusively.  If <code>gcc</code> and <code>cmake</code> are already installed in Cygwin, you can rerun the installer and deselect these packages (they can be found under the developer category).
 
** Minimum required packages:
 
** Minimum required packages:
 
*** Base
 
*** Base
 
*** Python
 
*** Python
 +
*** Make
 
** Rename <tt>/usr/bin/link.exe</tt> so it does not interfere with Windows <tt>LINK.EXE</tt>  
 
** Rename <tt>/usr/bin/link.exe</tt> so it does not interfere with Windows <tt>LINK.EXE</tt>  
 
** If you want to use a Cygwin shell to build and/or debug GridPACK applications:
 
** If you want to use a Cygwin shell to build and/or debug GridPACK applications:
 
*** Do not install a compiler set.
 
*** Do not install a compiler set.
 
*** Do not install CMake.  
 
*** Do not install CMake.  
* [https://msdn.microsoft.com/en-us/library/windows/desktop/bb524831%28v=vs.85%29.aspx Microsoft MPI] <br/> This appears to be the only modern implementation available for Windows.  In the past, the OpenMPI and MPICH implementation were available for Windows, but no more.
+
* [https://msdn.microsoft.com/en-us/library/windows/desktop/bb524831%28v=vs.85%29.aspx Microsoft MPI] <br/> This appears to be the only modern implementation available for Windows.  In the past, the OpenMPI and MPICH implementations were available for Windows, but no more. You will need to download both the <code>.msi</code> and <code>.exe</code> files.
 
* Some software to unpack <tt>.zip</tt>, <tt>.gz</tt>, and <tt>tar</tt> archives.  Commands to unpack all of these archives are available with Cygwin. Windows can handle  <tt>.zip</tt> archives natively.  
 
* Some software to unpack <tt>.zip</tt>, <tt>.gz</tt>, and <tt>tar</tt> archives.  Commands to unpack all of these archives are available with Cygwin. Windows can handle  <tt>.zip</tt> archives natively.  
 
* (optional) [https://www.microsoft.com/en-us/download/details.aspx?id=48215 MS-MPI Debugger Extension] is useful for debugging problem with parallel programs.
 
* (optional) [https://www.microsoft.com/en-us/download/details.aspx?id=48215 MS-MPI Debugger Extension] is useful for debugging problem with parallel programs.
Line 23: Line 25:
 
= Build Required Libraries =
 
= Build Required Libraries =
  
* In these instructions, everything is done from the Command Prompt.  Open a 64-bit Visual Studio command prompt, which should be available from the Start Menu. Type, or copy and paste, the commands below into that command prompt window.
+
* In these instructions, everything is done from the Command Prompt.  Open a 64-bit Visual Studio command prompt, which should be available from the Start Menu. If your version of Visual Studio has more than one type of Command Prompt, use the developer prompt. You should bring up Command Prompts as administrator, otherwise Visual Studio has trouble finding the common tools folder. To do this, mouse over the Command Prompt field and right-click. A menu should show up with "Run as Administrator" as one of the options. Type, or copy and paste, the commands below into that command prompt window.
 +
* The caret <tt>^</tt> character is the line continuation character for Command Prompt window.
 
* Choose a place to install libraries. <tt>C:\GridPACK</tt> is used in this case.  Avoid a path with spaces or special characters in it.  It's convenient to set an environment variable to hold this path.  <tt>GridPACKDir</tt> is used here.   
 
* Choose a place to install libraries. <tt>C:\GridPACK</tt> is used in this case.  Avoid a path with spaces or special characters in it.  It's convenient to set an environment variable to hold this path.  <tt>GridPACKDir</tt> is used here.   
* Make a folder there in which source archives can be unpacked and the builds can be performed.  <tt>C:\GridPACK\src</tt> is used here.  Again, avoid a path with spaces or special characters in it.
+
* Make a folder in the install directory in which source archives can be unpacked and the builds can be performed.  <tt>C:\GridPACK\src</tt> is used here.  Again, avoid a path with spaces or special characters in it.
* Prepare prepare VS/CMake to use MPI. Enter the command
+
* Prepare VS/CMake to use MPI. Enter the command
 
     set msmpi
 
     set msmpi
the response should be something like  
+
The response should be something like  
 
     MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\
 
     MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\
 
     MSMPI_INC=C:\Program Files (x86)\Microsoft SDKs\MPI\Include\
 
     MSMPI_INC=C:\Program Files (x86)\Microsoft SDKs\MPI\Include\
Line 40: Line 43:
 
* Bootstrap the boost.build, e.g.,
 
* Bootstrap the boost.build, e.g.,
 
     .\bootstrap.bat
 
     .\bootstrap.bat
* Add <tt>using mpi;</tt> to <tt>project-config.jam</tt>
+
* Add <tt>using mpi ;</tt> to <tt>project-config.jam</tt>
* As you might imagine, Boost is not built on Windows with MPI often. Consequently, it's a little out of date. The file <tt>mpi.jam</tt> needs to be modified. It's in a different place depending on the version. In Boost 1.61 it is <tt>./tools/build/src/tools/mpi.jam</tt>. Modify <tt>mpi.jam</tt> to make it work using the following patch:
+
* As you might imagine, Boost is not built on Windows with MPI often. Consequently, it's a little out of date. The file <tt>mpi.jam</tt> needs to be modified. It's in a different place depending on the version. In Boost 1.61 it is in <tt>./tools/build/src/tools/mpi.jam</tt>. Modify <tt>mpi.jam</tt> to make it work using the following patch:
 
     247a248,250
 
     247a248,250
 
     >    local win_ms_mpi_sdk = "C:\\Program Files (x86)\\Microsoft SDKs\\MPI" ;
 
     >    local win_ms_mpi_sdk = "C:\\Program Files (x86)\\Microsoft SDKs\\MPI" ;
Line 70: Line 73:
 
     ---
 
     ---
 
     >      .mpirun = "\"$(win_ms_mpi)\\Bin\\mpiexec.exe"\" ;
 
     >      .mpirun = "\"$(win_ms_mpi)\\Bin\\mpiexec.exe"\" ;
 +
 +
If you edit this file by hand, note that in addition to replacing the fields <tt>cluster_pack_path</tt> and <tt>cluster_pack_path_native</tt>, you also need to replace the values
 +
<tt>amd64</tt> and <tt>i386</tt>.
  
 
* Configure, build, and install. The <tt>python</tt> library is not needed; <tt>iostreams</tt> is skipped unless some additional libraries are installed; <tt>graph_parallel</tt> fails to build (see [https://svn.boost.org/trac/boost/ticket/11908 this ticket]).  There are two options for building:   
 
* Configure, build, and install. The <tt>python</tt> library is not needed; <tt>iostreams</tt> is skipped unless some additional libraries are installed; <tt>graph_parallel</tt> fails to build (see [https://svn.boost.org/trac/boost/ticket/11908 this ticket]).  There are two options for building:   
Line 99: Line 105:
 
* 1.58.0 works with VS 2013
 
* 1.58.0 works with VS 2013
 
* 1.61.0 works with VS 2010, but requires update 5 for VS 2013 (see [https://svn.boost.org/trac/boost/ticket/11885 this ticket])  
 
* 1.61.0 works with VS 2010, but requires update 5 for VS 2013 (see [https://svn.boost.org/trac/boost/ticket/11885 this ticket])  
* Boost decorates the library names with the compiler version, so explicitly specify the compiler and make sure that's the compiler you use for GridPACK.  
+
* Boost decorates the library names with the compiler version, so explicitly specify the compiler and make sure that's the compiler you use for GridPACK. If multiple compilers are available, it is possible to force the boost configuration to pick a specific one by adding an option like <tt>-toolchain=msvc-11.0</tt> to <tt>\.b2</tt>.
 
* Some references for these instructions:
 
* Some references for these instructions:
 
** A pretty complete set of instructions for [http://stackoverflow.com/questions/26147564/how-to-build-boost-mpi-for-ms-mpi-with-visual-studio-2012 building Boost on Windows with MPI]
 
** A pretty complete set of instructions for [http://stackoverflow.com/questions/26147564/how-to-build-boost-mpi-for-ms-mpi-with-visual-studio-2012 building Boost on Windows with MPI]
Line 105: Line 111:
 
** [http://www.boost.org/doc/libs/1_61_0/more/getting_started/windows.html Official Boost build instructions].
 
** [http://www.boost.org/doc/libs/1_61_0/more/getting_started/windows.html Official Boost build instructions].
 
** A pertinent Boost [https://svn.boost.org/trac/boost/ticket/11908 ticket].
 
** A pertinent Boost [https://svn.boost.org/trac/boost/ticket/11908 ticket].
 +
 +
== Algebra Libraries ==
 +
 +
Ordinarily, several packages used by GridPACK can be built by having PETSc download and build them as part of the PETSc build. Unfortunately, while this works quite well on the Linux OS, it does not work in many cases for Windows. Consequently, these packages must be downloaded and built separately. Note that the builds below all assume that you are calling the cmake command in a separate directory, below the main library directory.
  
 
== BLAS/LAPACK (CLAPACK) ==
 
== BLAS/LAPACK (CLAPACK) ==
  
 
Some implementation of BLAS/LAPACK is required for PETSc and some other libraries. The implementation described here was chosen because it does not require a Fortran compiler.  It is apparently ''really'' slow.  It will probably be necessary to install Intel compilers and MKL in order to get improvement in speed.  The Windows port of CLAPACK is described [http://icl.cs.utk.edu/lapack-for-windows/clapack/ here].
 
Some implementation of BLAS/LAPACK is required for PETSc and some other libraries. The implementation described here was chosen because it does not require a Fortran compiler.  It is apparently ''really'' slow.  It will probably be necessary to install Intel compilers and MKL in order to get improvement in speed.  The Windows port of CLAPACK is described [http://icl.cs.utk.edu/lapack-for-windows/clapack/ here].
* Get the source [http://icl.cs.utk.edu/lapack-for-windows/clapack/clapack-3.2.1-CMAKE.tgz here].
+
* Get the source [http://icl.cs.utk.edu/lapack-for-windows/clapack/clapack-3.2.1-CMAKE.tgz here]. This is a gzip'd tar archive.  This can be unpacked by [http://www.7-zip.org/download.html 7-zip], if available. Since Cygwin is required to build and use PETSc, this command
 +
  tar vxzf clapack-3.2.1-CMAKE.tgz
 +
can be executed in a Cygwin terminal to unpack the archive in the current directory
 
* Configure, build, and install
 
* Configure, build, and install
 +
    cd clapack-3.2.1-CMAKE
 +
    mkdir build
 +
    cd build
 
     cmake -Wdev ^
 
     cmake -Wdev ^
 
         -G "Visual Studio 10 2010 Win64" ^
 
         -G "Visual Studio 10 2010 Win64" ^
Line 117: Line 132:
 
     cmake --build . --config Release
 
     cmake --build . --config Release
 
     cmake --build . --target install --config Release
 
     cmake --build . --target install --config Release
 +
Depending on which version of Visual Studio you are using, the argument for -G will change. You can get a listing of options by typing <code>cmake -G</code>.
  
 
== SuiteSparse ==
 
== SuiteSparse ==
Line 122: Line 138:
 
* For use with PETSc
 
* For use with PETSc
 
* Get the source for the Windows port from [https://github.com/jlblancoc/suitesparse-metis-for-windows here].
 
* Get the source for the Windows port from [https://github.com/jlblancoc/suitesparse-metis-for-windows here].
* Needs BLAS/LAPACK that's consistent with PETSc, so try to just use the one PETSc likes.  
+
* Needs BLAS/LAPACK that's consistent with PETSc, so try to just use the one PETSc likes. Note that you can verify that both SuiteSparse and PETSc are using the same BLAS/LAPACK libraries by explicitly specifying them in both the SuiteSparse configuration (via the configuration variables <tt>SUITESPARSE_CUSTOM_BLAS_LIB</tt> and <tt>SUITESPARSE_CUSTOM_LAPACK_LIB</tt>) and the PETSc configuration (via the configuration option <tt>--with-blas-lapack-lib</tt>).
* Edit <tt>CMakeLists.txt</tt> to remove the library suffix at this point:
+
* Edit <tt>CMakeLists.txt</tt> in the root directory by commenting out the following lines to remove the library suffix:
 
   ## get POSTFIX for lib install dir
 
   ## get POSTFIX for lib install dir
 
   #if(CMAKE_SIZEOF_VOID_P MATCHES "8")
 
   #if(CMAKE_SIZEOF_VOID_P MATCHES "8")
Line 139: Line 155:
 
       -D SUITESPARSE_CUSTOM_BLAS_LIB:PATH=%prefix%\lib\blas.lib ^
 
       -D SUITESPARSE_CUSTOM_BLAS_LIB:PATH=%prefix%\lib\blas.lib ^
 
       -D SUITESPARSE_CUSTOM_LAPACK_LIB:PATH=%prefix%\lib\lapack.lib ^
 
       -D SUITESPARSE_CUSTOM_LAPACK_LIB:PATH=%prefix%\lib\lapack.lib ^
       -D CMAKE_INSTALL_PREFIX:PATH="%GridPACKDir" ^
+
       -D CMAKE_INSTALL_PREFIX:PATH="%GridPACKDir%" ^
 
       ..
 
       ..
 
   cmake --build . --config Release
 
   cmake --build . --config Release
Line 150: Line 166:
 
* The source distribution needs to modified to  
 
* The source distribution needs to modified to  
 
:# Have CMake look for MPI in the correct way
 
:# Have CMake look for MPI in the correct way
:# Edit <tt>metis.h</tt> so that <tt>REALTYPEWIDTH</tt> is 64.
+
:# Edit <tt>metis.h</tt> in the <tt>parmetis-4.0.3/metis/include</tt> directory so that <tt>REALTYPEWIDTH</tt> is 64.
 
:# Make sure <tt>metis.h</tt> is installed
 
:# Make sure <tt>metis.h</tt> is installed
: Here is the patch:
+
: Here is the patch for the <tt>CMakeLists.txt</tt> file located in the root diretory:
 
   diff -r -u parmetis-4.0.3/CMakeLists.txt parmetis-4.0.3.fixed/CMakeLists.txt
 
   diff -r -u parmetis-4.0.3/CMakeLists.txt parmetis-4.0.3.fixed/CMakeLists.txt
 
   --- parmetis-4.0.3/CMakeLists.txt 2013-03-30 09:24:50.000000000 -0700
 
   --- parmetis-4.0.3/CMakeLists.txt 2013-03-30 09:24:50.000000000 -0700
Line 193: Line 209:
 
   cmake --build . --target install --config Release
 
   cmake --build . --target install --config Release
 
   copy ..\metis\include\metis.h %prefix%\include
 
   copy ..\metis\include\metis.h %prefix%\include
 +
  copy libmetis\Release\metis.lib %prefix%\lib
 +
 +
:You still need to copy <tt>metis.h</tt> and <tt>metis.lib</tt> to the installation directory. For later versions of Visual Studio (e.g. 2015), you may get errors that look like
 +
 +
  C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\math.h(520):
 +
  error C2059: syntax error: '(' [C:\GridPACK\src\parmetis-4.0.3\build\libmetis\metis.vcxproj]
 +
 +
These can be fixed by going into the <tt>parmetis-4.0.3/metis/GKlib</tt> directory and commenting out the macro defining <tt>rint(x)</tt> at the bottom of <tt>gk_arch.h</tt>
  
: Still need to copy <tt>metis.h</tt> and <tt>metis.lib</tt> to the installation directory.
+
  #ifdef __MSC__
 +
  /* MSC does not have rint() function */
 +
  /*
 +
  #define rint(x) ((int)((x)+0.5))
 +
  */
 +
  /* MSC does not have INFINITY defined */
 +
  #ifndef INFINITY
 +
  #define INFINITY FLT_MAX
 +
  #endif
 +
  #endif
  
 
== PETSc ==
 
== PETSc ==
* These instructions mostly follow [https://github.com/INMOST-DEV/INMOST/wiki/0206-Compilation-PETSc-Windows these] and [https://github.com/jlblancoc/suitesparse-metis-for-windows those for Suitesparse for Windoze w/ CMake]
+
* These instructions mostly follow [https://github.com/INMOST-DEV/INMOST/wiki/0206-Compilation-PETSc-Windows the discussion here] as well as the instructions for building [https://github.com/jlblancoc/suitesparse-metis-for-windows Suitesparse on Windows with CMake]
 
* PETSc must be built within a Cygwin shell.  In this case, Cygwin was installed in <tt>C:\cygwin64</tt>
 
* PETSc must be built within a Cygwin shell.  In this case, Cygwin was installed in <tt>C:\cygwin64</tt>
* PETSc refuses to fetch and build many external packages when configured on Windows.  They need to be built individually.  See instructions above for the following:
+
* As mentioned above, PETSc refuses to fetch and build many external packages when configured on Windows.  They need to be built individually.  See instructions above for the following:
 
** BLAS/LAPACK
 
** BLAS/LAPACK
 
** ParMETIS
 
** ParMETIS
Line 208: Line 241:
 
* Within the VS Command Prompt, start a minimal Cygwin terminal and get it ready
 
* Within the VS Command Prompt, start a minimal Cygwin terminal and get it ready
 
   c:\cygwin64\bin\mintty.exe
 
   c:\cygwin64\bin\mintty.exe
* Within the Cygwin shell, review the contents of the <tt>PATH</tt> environment variable. Take anything out that's not directly related to the PETSc build.  Then, make sure Cygwin commands are available:  
+
* Within the Cygwin shell, review the contents of the <tt>PATH</tt> environment variable. Take out anything that is not directly related to the PETSc build.  Then, make sure Cygwin commands are available:  
 
   export PATH="/usr/bin:$PATH"
 
   export PATH="/usr/bin:$PATH"
 
* Configure and build for real values
 
* Configure and build for real values
Line 247: Line 280:
 
   make
 
   make
 
   make test
 
   make test
: The <tt>-EHsc</tt> flag is to enable C++ exceptions. Change <tt>--with-scalar-type=real</tt>  
+
: The <tt>-EHsc</tt> flag is to enable C++ exceptions. Change <tt>--with-scalar-type=real</tt> to <tt>--with-scalar-type=complex</tt> to build PETSc using complex numbers (you may also want to change the <tt>PETSC_ARCH</tt> variable from <tt>mswin-cxx-real-opt</tt> to <tt>mswin-cxx-complex-opt</tt>). The complex build appears to have a few problems. If the build stops, keep typing <tt>make</tt> until it completes.
 +
* You may have problems using DOS-type directory names as the arguments to options such as <tt>--with-mpi-include</tt>, particularly if they contain spaces. These can be converted to Linux-type directory names by using the cygpath command inside the Cygwin window. Type <tt>cygpath -d "C:\Program Files (x86)\Microsoft SDKs\MPI\Include"</tt> to get <tt>C:\PROGRA~2\MIA713~1\MPI\Include</tt> and use this name for any configure options that have a path listing.
 +
 
 +
* We have had difficulties building PETSc using Visual Studio 2015. For PETSc 3.6, we encountered too many compiler errors to get it to build but with PETSc 3.7 we were able to successfully build by modifying the file PETSC/src/mat/order/amd/amd.c. Change
 +
 
 +
  tval = (PetscBool)Control[AMD_AGGRESSIVE];
 +
 
 +
to
 +
  /* tval = (PetscBool)Control[AMD_AGGRESSIVE]; */
 +
  if (Control[AMD_AGGRESSIVE]) tval = PETSC_TRUE;
 +
  else tval = PETSC_FALSE;
  
 
* Verify installation by building the tests in GridPACK <tt>sandbox/petsc-cmake</tt>:
 
* Verify installation by building the tests in GridPACK <tt>sandbox/petsc-cmake</tt>:
Line 267: Line 310:
 
* Get the source from the trunk Subversion repository (see the [http://www.emsl.pnl.gov/docs/global home page]):
 
* Get the source from the trunk Subversion repository (see the [http://www.emsl.pnl.gov/docs/global home page]):
 
   svn checkout https://svn.pnl.gov/svn/hpctools/trunk/ga ga-svn
 
   svn checkout https://svn.pnl.gov/svn/hpctools/trunk/ga ga-svn
 +
* Note that you need to use the CMake build for GA. This has not been officially released but is available from the GA trunk. It is planned for release for GA version 5.6.
 
* Make a directory in which to build and change to that directory
 
* Make a directory in which to build and change to that directory
 
   mkdir ga-svn\build
 
   mkdir ga-svn\build
Line 286: Line 330:
 
* Check using Global Arrays tests
 
* Check using Global Arrays tests
  
* Check w/ GridPACK sandbox  
+
* Check w/ GridPACK sandbox. In the <tt>sanbox/ga</tt> directory, add a build directory and cd into it. Configure the GA test with
 
   cmake -Wdev --debug-trycompile ^
 
   cmake -Wdev --debug-trycompile ^
 
     -G "Visual Studio 10 2010 Win64" ^
 
     -G "Visual Studio 10 2010 Win64" ^
Line 292: Line 336:
 
     -D Boost_USE_STATIC_LIBS:BOOL=ON ^
 
     -D Boost_USE_STATIC_LIBS:BOOL=ON ^
 
     -D BOOST_INCLUDEDIR="%GridPACKDir%\include\boost-1_61" ^
 
     -D BOOST_INCLUDEDIR="%GridPACKDir%\include\boost-1_61" ^
     -D GA_DIR:PATH="%GridPACKDir\ga-svn" ^
+
     -D GA_DIR:PATH="%GridPACKDir" ^
    -D BOOST_ROOT:PATH="%GridPACKDir%" ^
 
 
     ..
 
     ..
  
 
= GridPACK =
 
= GridPACK =
  
* Check out the [https://github.com/wperkins/GridPACK/tree/windoze Windoze GridPACK fork]
+
* Check out the [https://github.com/wperkins/GridPACK/tree/windoze Windows GridPACK fork]
 
   git clone -b windoze https://github.com/wperkins/GridPACK.git gridpack
 
   git clone -b windoze https://github.com/wperkins/GridPACK.git gridpack
 
   cd gridpack
 
   cd gridpack
 
   git submodule init
 
   git submodule init
   git submoudle update
+
   git submodule update
 
* Make a build directory and change into it
 
* Make a build directory and change into it
 
   mkdir build
 
   mkdir build

Latest revision as of 19:26, 6 April 2018

Building GridPACK on Windows is not for the faint of heart.

System Preparation

The following are available as native Windows applications and can just be installed in the normal Windows way:

  • Visual C++ in some form.
    • The "free" ones, Visual Studio Express and Community, should work.
    • To get the 64-bit compilers for Visual Studio 2015, you need to run the x64 Native Tools Command Prompt (you may need to go into the Visual Studios 2015 folder and then into the Visual Studio Tools subfolder and then into the Windows Desktop Command Prompts folder under that to find it). Once you have opened an x64 Native Command Prompt, cd to C:"\Program Files (x86)\Microsoft Visual Studio 14.0\VC" and type "vcvarsall amd64". More information about this can be found here.
  • Windows SDK
    Usually, this is just installed with Visual Studio
  • CMake is required. During the installation process, the dialog will ask if you want CMake added to PATH variable. For some reason, the default is not to include CMake. You should change the selection to add CMake to the PATH for all users.
  • A minimal Cygwin installation is necessary.
    Cygwin is required to build, and test for, PETSc. If the system has a Cygwin installation that is being used, it would probably be best to make another installation for GridPACK installation exclusively. If gcc and cmake are already installed in Cygwin, you can rerun the installer and deselect these packages (they can be found under the developer category).
    • Minimum required packages:
      • Base
      • Python
      • Make
    • Rename /usr/bin/link.exe so it does not interfere with Windows LINK.EXE
    • If you want to use a Cygwin shell to build and/or debug GridPACK applications:
      • Do not install a compiler set.
      • Do not install CMake.
  • Microsoft MPI
    This appears to be the only modern implementation available for Windows. In the past, the OpenMPI and MPICH implementations were available for Windows, but no more. You will need to download both the .msi and .exe files.
  • Some software to unpack .zip, .gz, and tar archives. Commands to unpack all of these archives are available with Cygwin. Windows can handle .zip archives natively.
  • (optional) MS-MPI Debugger Extension is useful for debugging problem with parallel programs.

Build Required Libraries

  • In these instructions, everything is done from the Command Prompt. Open a 64-bit Visual Studio command prompt, which should be available from the Start Menu. If your version of Visual Studio has more than one type of Command Prompt, use the developer prompt. You should bring up Command Prompts as administrator, otherwise Visual Studio has trouble finding the common tools folder. To do this, mouse over the Command Prompt field and right-click. A menu should show up with "Run as Administrator" as one of the options. Type, or copy and paste, the commands below into that command prompt window.
  • The caret ^ character is the line continuation character for Command Prompt window.
  • Choose a place to install libraries. C:\GridPACK is used in this case. Avoid a path with spaces or special characters in it. It's convenient to set an environment variable to hold this path. GridPACKDir is used here.
  • Make a folder in the install directory in which source archives can be unpacked and the builds can be performed. C:\GridPACK\src is used here. Again, avoid a path with spaces or special characters in it.
  • Prepare VS/CMake to use MPI. Enter the command
   set msmpi

The response should be something like

   MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\
   MSMPI_INC=C:\Program Files (x86)\Microsoft SDKs\MPI\Include\
   MSMPI_LIB32=C:\Program Files (x86)\Microsoft SDKs\MPI\Lib\x86\
   MSMPI_LIB64=C:\Program Files (x86)\Microsoft SDKs\MPI\Lib\x64\

Boost

  • Download source from here and unpack in %GridPACKDir%\src
  • Remember to set msmpi as described above
  • Bootstrap the boost.build, e.g.,
   .\bootstrap.bat
  • Add using mpi ; to project-config.jam
  • As you might imagine, Boost is not built on Windows with MPI often. Consequently, it's a little out of date. The file mpi.jam needs to be modified. It's in a different place depending on the version. In Boost 1.61 it is in ./tools/build/src/tools/mpi.jam. Modify mpi.jam to make it work using the following patch:
    247a248,250
    >     local win_ms_mpi_sdk = "C:\\Program Files (x86)\\Microsoft SDKs\\MPI" ;
    >     local win_ms_mpi = "C:\\Program Files\\Microsoft MPI" ;
    >
    249,251c252,254
    <     local cluster_pack_path_native = "C:\\Program Files\\Microsoft Compute Cluster Pack" ;
    <     local cluster_pack_path = [ path.make $(cluster_pack_path_native) ] ;
    <     if [ GLOB $(cluster_pack_path_native)\\Include : mpi.h ]
    ---
    >     # local cluster_pack_path_native = "C:\\Program Files\\Microsoft Compute Cluster Pack" ;
    >     # local cluster_pack_path = [ path.make $(cluster_pack_path_native) ] ;
    >     if [ GLOB $(win_ms_mpi_sdk)\\Include : mpi.h ]
    255c258
    <         ECHO "Found Microsoft Compute Cluster Pack: $(cluster_pack_path_native)" ;
    ---
    >         ECHO "Found Microsoft Compute Cluster Pack: $(win_ms_mpi_sdk)" ;
    260,262c263,265
    <       options = <include>$(cluster_pack_path)/Include
    <                 <address-model>64:<library-path>$(cluster_pack_path)/Lib/amd64
    <                 <library-path>$(cluster_pack_path)/Lib/i386
    ---
    >       options = <include>$(win_ms_mpi_sdk)/Include
    >                 <address-model>64:<library-path>$(win_ms_mpi_sdk)/Lib/x64
    >                 <library-path>$(win_ms_mpi_sdk)/Lib/x86
    268c271
    <       .mpirun = "\"$(cluster_pack_path_native)\\Bin\\mpiexec.exe"\" ;
    ---
    >       .mpirun = "\"$(win_ms_mpi)\\Bin\\mpiexec.exe"\" ;

If you edit this file by hand, note that in addition to replacing the fields cluster_pack_path and cluster_pack_path_native, you also need to replace the values amd64 and i386.

  • Configure, build, and install. The python library is not needed; iostreams is skipped unless some additional libraries are installed; graph_parallel fails to build (see this ticket). There are two options for building:
  • Option 1: just build what GridPACK requires (header-only libraries are still installed)
 .\b2 ^
   --prefix=%GridPACKDir% ^
   --with-mpi ^
   --with-serialization ^
   --with-random ^
   --with-filesystem ^
   --with-system ^
   --build-type=complete ^
   threading=single ^
   address-model=64 ^
   link=static runtime-link=shared ^
   install
  • Option 2: build everything (except python -- I'm not sure what's required for that)
 .\b2 ^
   --prefix=%GridPACKDir% ^
   --without-python ^
   --build-type=complete ^
   threading=single ^
   address-model=64 ^
   link=static runtime-link=shared ^
   install

Notes:

  • 1.58.0 works with VS 2013
  • 1.61.0 works with VS 2010, but requires update 5 for VS 2013 (see this ticket)
  • Boost decorates the library names with the compiler version, so explicitly specify the compiler and make sure that's the compiler you use for GridPACK. If multiple compilers are available, it is possible to force the boost configuration to pick a specific one by adding an option like -toolchain=msvc-11.0 to \.b2.
  • Some references for these instructions:

Algebra Libraries

Ordinarily, several packages used by GridPACK can be built by having PETSc download and build them as part of the PETSc build. Unfortunately, while this works quite well on the Linux OS, it does not work in many cases for Windows. Consequently, these packages must be downloaded and built separately. Note that the builds below all assume that you are calling the cmake command in a separate directory, below the main library directory.

BLAS/LAPACK (CLAPACK)

Some implementation of BLAS/LAPACK is required for PETSc and some other libraries. The implementation described here was chosen because it does not require a Fortran compiler. It is apparently really slow. It will probably be necessary to install Intel compilers and MKL in order to get improvement in speed. The Windows port of CLAPACK is described here.

  • Get the source here. This is a gzip'd tar archive. This can be unpacked by 7-zip, if available. Since Cygwin is required to build and use PETSc, this command
  tar vxzf clapack-3.2.1-CMAKE.tgz

can be executed in a Cygwin terminal to unpack the archive in the current directory

  • Configure, build, and install
   cd clapack-3.2.1-CMAKE
   mkdir build
   cd build
   cmake -Wdev ^
       -G "Visual Studio 10 2010 Win64" ^
       -D CMAKE_INSTALL_PREFIX:PATH="%GridPACKDir%" ^
       ..
   cmake --build . --config Release
   cmake --build . --target install --config Release

Depending on which version of Visual Studio you are using, the argument for -G will change. You can get a listing of options by typing cmake -G.

SuiteSparse

  • For use with PETSc
  • Get the source for the Windows port from here.
  • Needs BLAS/LAPACK that's consistent with PETSc, so try to just use the one PETSc likes. Note that you can verify that both SuiteSparse and PETSc are using the same BLAS/LAPACK libraries by explicitly specifying them in both the SuiteSparse configuration (via the configuration variables SUITESPARSE_CUSTOM_BLAS_LIB and SUITESPARSE_CUSTOM_LAPACK_LIB) and the PETSc configuration (via the configuration option --with-blas-lapack-lib).
  • Edit CMakeLists.txt in the root directory by commenting out the following lines to remove the library suffix:
 ## get POSTFIX for lib install dir
 #if(CMAKE_SIZEOF_VOID_P MATCHES "8")
 #  set(LIB_POSTFIX "64" CACHE STRING "suffix for 32/64 inst dir placement")
 #else()
   set(LIB_POSTFIX "" CACHE STRING "suffix for 32/64 inst dir placement")
 #endif()
 mark_as_advanced(LIB_POSTFIX)
  • Configure, build, and install
 cmake -Wdev ^
     -G "Visual Studio 10 2010 Win64" ^
     -D BUILD_METIS:BOOL=NO ^
     -D SUITESPARSE_INSTALL_PREFIX:PATH="%GridPACKDir%" ^
     -D SUITESPARSE_USE_CUSTOM_BLAS_LAPACK_LIBS:BOOL=ON ^
     -D SUITESPARSE_CUSTOM_BLAS_LIB:PATH=%prefix%\lib\blas.lib ^
     -D SUITESPARSE_CUSTOM_LAPACK_LIB:PATH=%prefix%\lib\lapack.lib ^
     -D CMAKE_INSTALL_PREFIX:PATH="%GridPACKDir%" ^
     ..
 cmake --build . --config Release
 cmake --build . --target install --config Release


ParMETIS

  • Get the official distribution from here.
  • The source distribution needs to modified to
  1. Have CMake look for MPI in the correct way
  2. Edit metis.h in the parmetis-4.0.3/metis/include directory so that REALTYPEWIDTH is 64.
  3. Make sure metis.h is installed
Here is the patch for the CMakeLists.txt file located in the root diretory:
 diff -r -u parmetis-4.0.3/CMakeLists.txt parmetis-4.0.3.fixed/CMakeLists.txt
 --- parmetis-4.0.3/CMakeLists.txt	2013-03-30 09:24:50.000000000 -0700
 +++ parmetis-4.0.3.fixed/CMakeLists.txt	2016-06-30 11:32:38.691121400 -0700
 @@ -15,6 +15,7 @@
  #   message(FATAL_ERROR "mpi is not found")
  # endif()
  # set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${MPI_COMPILE_FLAGS}")
 +find_package(MPI REQUIRED)
  # Prepare libraries.
  if(SHARED)
 @@ -33,6 +34,7 @@
  include_directories(${METIS_PATH}/include)
  # List of directories that cmake will look for CMakeLists.txt
 +add_subdirectory(${METIS_PATH}/include)
  add_subdirectory(${METIS_PATH}/libmetis ${CMAKE_BINARY_DIR}/libmetis)
  add_subdirectory(include)
  add_subdirectory(libparmetis)
 Only in parmetis-4.0.3.fixed/: CMakeLists.txt~
 diff -r -u parmetis-4.0.3/metis/include/metis.h parmetis-4.0.3.fixed/metis/include/metis.h
 --- parmetis-4.0.3/metis/include/metis.h	2013-03-30 09:24:50.000000000 -0700
 +++ parmetis-4.0.3.fixed/metis/include/metis.h	2016-06-30 11:08:21.791160800 -0700
 @@ -40,7 +40,7 @@
     32 : single precission floating point (float)
     64 : double precission floating point (double)
  --------------------------------------------------------------------------*/
 -#define REALTYPEWIDTH 32
 +#define REALTYPEWIDTH 64
  • Configure and build
 cmake ^
     -G "Visual Studio 10 2010 Win64" ^
     -D BUILD_SHARED_LIBS:BOOL=NO ^
     -D METIS_INSTALL:BOOL=YES ^
     -D CMAKE_INSTALL_PREFIX:PATH="C:\GridPACK" ^
     ..
 cmake --build . --config Release
 cmake --build . --target install --config Release
 copy ..\metis\include\metis.h %prefix%\include
 copy libmetis\Release\metis.lib %prefix%\lib
You still need to copy metis.h and metis.lib to the installation directory. For later versions of Visual Studio (e.g. 2015), you may get errors that look like
  C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt\math.h(520):
  error C2059: syntax error: '(' [C:\GridPACK\src\parmetis-4.0.3\build\libmetis\metis.vcxproj]

These can be fixed by going into the parmetis-4.0.3/metis/GKlib directory and commenting out the macro defining rint(x) at the bottom of gk_arch.h

  #ifdef __MSC__
  /* MSC does not have rint() function */
  /*
  #define rint(x) ((int)((x)+0.5))
  */
  /* MSC does not have INFINITY defined */
  #ifndef INFINITY
  #define INFINITY FLT_MAX
  #endif
  #endif

PETSc

  • These instructions mostly follow the discussion here as well as the instructions for building Suitesparse on Windows with CMake
  • PETSc must be built within a Cygwin shell. In this case, Cygwin was installed in C:\cygwin64
  • As mentioned above, PETSc refuses to fetch and build many external packages when configured on Windows. They need to be built individually. See instructions above for the following:
    • BLAS/LAPACK
    • ParMETIS
    • SparseSuite
  • Start a VS Command Prompt
  • Remember to set the Microsoft MPI environment
 set msmpi
  • Within the VS Command Prompt, start a minimal Cygwin terminal and get it ready
 c:\cygwin64\bin\mintty.exe
  • Within the Cygwin shell, review the contents of the PATH environment variable. Take out anything that is not directly related to the PETSc build. Then, make sure Cygwin commands are available:
 export PATH="/usr/bin:$PATH"
  • Configure and build for real values
 prefix="/cygdrive/c/GridPACK"
 ./configure \
   PETSC_ARCH=mswin-cxx-real-opt \
   --with-cc="win32fe cl" \
   --with-clanguage=c++ \
   --with-c++-support=1 \
   --download-f2cblaslapack=0  \
   --with-blas-lapack-lib=[${prefix}/lib/lapack.lib,${prefix}/lib/blas.lib,${prefix}/lib/libf2c.lib] \
   --download-superlu_dist=0 \
   --download-metis=0 \
   --with-metis=1 \
   --with-metis-include=${prefix}/include \
   --with-metis-lib=[${prefix}/lib/metis.lib] \
   --download-parmetis=0 \
   --with-parmetis=1 \
   --with-parmetis-include=${prefix}/include \
   --with-parmetis-lib=[${prefix}/lib/parmetis.lib] \
   --download-suitesparse=0 \
   --with-suitesparse=1 \
   --with-suitesparse-include="${prefix}/include" \
   --with-suitesparse-lib=[${prefix}/lib/libumfpack.lib,${prefix}/lib/libamd.lib,${prefix}/lib/libbtf.lib,${prefix}/lib/libcamd.lib,${prefix}/lib/libccolamd.lib,${prefix}/lib/libcolamd.lib,${prefix}/lib/libcholmod.lib,${prefix}/lib/libcxsparse.lib,${prefix}/lib/libklu.lib,${prefix}/lib/libspqr.lib,${prefix}/lib/libldl.lib,${prefix}/lib/suitesparseconfig.lib] \
   --with-c-support=0 \
   --with-fortran=0 \
   --with-fc=0 \
   --with-precision=double \
   --with-scalar-type=real \
   --with-mpi-include=/cygdrive/c/Program\ Files\ \(x86\)/Microsoft\ SDKs/MPI/Include \
   --with-mpi-lib=['/cygdrive/c/Program\ Files\ \(x86\)/Microsoft\ SDKs/MPI/Lib/x64/msmpi.lib'] \
   --with-mpi-mpiexec=/cygdrive/c/Program\ Files/Microsoft\ MPI/Bin/mpiexec.exe \
   --with-debugging=0 \
   --with-windows-graphics=0 \
   --with-x11=0 \
   --CFLAGS='-O2 -MD -wd4996' \
   --CXXFLAGS='-O2 -MD -wd4996 -EHsc'
 make
 make test
The -EHsc flag is to enable C++ exceptions. Change --with-scalar-type=real to --with-scalar-type=complex to build PETSc using complex numbers (you may also want to change the PETSC_ARCH variable from mswin-cxx-real-opt to mswin-cxx-complex-opt). The complex build appears to have a few problems. If the build stops, keep typing make until it completes.
  • You may have problems using DOS-type directory names as the arguments to options such as --with-mpi-include, particularly if they contain spaces. These can be converted to Linux-type directory names by using the cygpath command inside the Cygwin window. Type cygpath -d "C:\Program Files (x86)\Microsoft SDKs\MPI\Include" to get C:\PROGRA~2\MIA713~1\MPI\Include and use this name for any configure options that have a path listing.
  • We have had difficulties building PETSc using Visual Studio 2015. For PETSc 3.6, we encountered too many compiler errors to get it to build but with PETSc 3.7 we were able to successfully build by modifying the file PETSC/src/mat/order/amd/amd.c. Change
 tval = (PetscBool)Control[AMD_AGGRESSIVE];

to

 /* tval = (PetscBool)Control[AMD_AGGRESSIVE]; */
 if (Control[AMD_AGGRESSIVE]) tval = PETSC_TRUE;
 else tval = PETSC_FALSE;
  • Verify installation by building the tests in GridPACK sandbox/petsc-cmake:
 set path=%path%;c:\cygwin64\bin
 set CFLAGS="/D _ITERATOR_DEBUG_LEVEL=0"
 set CXXFLAGS="/D _ITERATOR_DEBUG_LEVEL=0"
 cmake -Wdev  ^
   -D BOOST_ROOT:PATH=C:\GridPACK ^
   -D Boost_USE_STATIC_LIBS:BOOL=ON ^
   -D BOOST_INCLUDEDIR=C:\GridPACK\include\boost-1_61 ^
   -D PETSC_DIR:PATH="C:\GridPACK\src\petsc-3.6.4" ^
   -D PETSC_ARCH:STRING='mswin-cxx-complex-opt' ^
   -G "Visual Studio 10 2010 Win64" ^
   ..
 cmake --build . --config Release

Global Arrays

  • Get the source from the trunk Subversion repository (see the home page):
  svn checkout https://svn.pnl.gov/svn/hpctools/trunk/ga ga-svn
  • Note that you need to use the CMake build for GA. This has not been officially released but is available from the GA trunk. It is planned for release for GA version 5.6.
  • Make a directory in which to build and change to that directory
  mkdir ga-svn\build
  cd ga-svn\build
  • Configure, build, and install
 set CFLAGS="/D _ITERATOR_DEBUG_LEVEL=0"
 set CXXFLAGS="/D _ITERATOR_DEBUG_LEVEL=0"
 cmake -Wdev --debug-trycompile ^
   -G "Visual Studio 10 2010 Win64" ^
   -D ENABLE_BLAS:BOOL=No ^
   -D ENABLE_FORTRAN:BOOL=No ^
   -D ENABLE_CXX:BOOL=Yes ^
   -D GA_RUNTIME:STRING=MPI_TS ^
   -D CMAKE_INSTALL_PREFIX:PATH="%GridPACKDir%" ^
   ..
 cmake --build . --config Release
 cmake --build . --config Release --target install
  • Check using Global Arrays tests
  • Check w/ GridPACK sandbox. In the sanbox/ga directory, add a build directory and cd into it. Configure the GA test with
 cmake -Wdev --debug-trycompile ^
   -G "Visual Studio 10 2010 Win64" ^
   -D BOOST_ROOT:PATH="%GridPACKDir%" ^
   -D Boost_USE_STATIC_LIBS:BOOL=ON ^
   -D BOOST_INCLUDEDIR="%GridPACKDir%\include\boost-1_61" ^
   -D GA_DIR:PATH="%GridPACKDir" ^
   ..

GridPACK

 git clone -b windoze https://github.com/wperkins/GridPACK.git gridpack
 cd gridpack
 git submodule init
 git submodule update
  • Make a build directory and change into it
 mkdir build
 cd build
  • Configure
   set path=%path%;c:\cygwin64\bin
   set prefix="C:\GridPACK"
   set CFLAGS="/D _ITERATOR_DEBUG_LEVEL=0"
   set CXXFLAGS="/D _ITERATOR_DEBUG_LEVEL=0"
   cmake -Wdev --debug-trycompile ^
     -G "Visual Studio 10 2010 Win64" ^
     -D USE_PROGRESS_RANKS:BOOL=OFF ^
     -D BOOST_ROOT:PATH=C:\GridPACK ^
     -D Boost_USE_STATIC_LIBS:BOOL=ON ^
     -D Boost_USE_DEBUG_RUNTIME:BOOL=OFF ^
     -D BOOST_INCLUDEDIR=C:\GridPACK\include\boost-1_61 ^
     -D PETSC_DIR:PATH="C:\GridPACK\src\petsc-3.6.4" ^
     -D PETSC_ARCH:STRING='mswin-cxx-complex-opt' ^
     -D GA_DIR:PATH='C:\GridPACK\ga-svn' ^
     -D GA_TEST_RUNS:BOOL=YES ^
     -D PARMETIS_DIR:PATH=C:\GridPACK ^
     -D MPIEXEC_MAX_NUMPROCS:STRING="2" ^
     -D GRIDPACK_TEST_TIMEOUT:STRING=60 ^
     ..
Note: this is the contents of gridpack/example_configuration.bat
  • Build
 cmake --build . --config Release
  • Install
 cmake --build . --config Release --target install