614. SIESTA with MPI and acml on debian jessie

One of my students might be using SIESTA for some simulations, and a first step towards that is to set it up on my cluster.

This isn't an optimised build -- right now I'm just looking at having a simple parallell build that runs.

I had a look at http://www.pa.msu.edu/people/tomanek/SIESTA-installation.html and http://pelios.csx.cam.ac.uk/~mc321/siesta.html.
 
NOTE: don't use the int64 acml or openblas BLAS libs, or you'll get SIGSEV due to invalid memory reference when running. NWChem is the complete opposite, and for some reason both the int64 and regulat acml libs have the same names. Not sure how that's supposed to work out on a system with nwchem, which needs the int64 libs.

See here for acml on debian. I've got /opt/acml/acml5.3.1/gfortran64_int64/lib in my /etc/ld.so.conf.d/acml.conf on behalf of nwchem.
 Being lazy, I opted for the debian scalapack and libblacs packages:

sudo apt-get install libscalapack-mpi-dev libblacs-mpi-dev libopenmpi-dev

To get the link to the SIESTA code, go to http://departments.icmab.es/leem/siesta/CodeAccess/selector.html

Then, if you're an academic, you can do:

sudo mkdir /opt/siesta
sudo chown $USER /opt/siesta
cd /opt/siesta
wget http://departments.icmab.es/leem/siesta/CodeAccess/Code/siesta-3.2-pl-5.tgz
tar xvf siesta-3.2-pl-5.tgz
cd siesta-3.2-pl-5/Obj
sh ../Src/obj_setup.sh

*** Compilation setup done. 
 *** Remember to copy an arch.make file or run configure as:
    ../Src/configure [configure_options]

../Src/./configure --help

`configure' configures siesta 2.0 to adapt to many kinds of systems.

Usage: ./configure [OPTION]... [VAR=VALUE]...

[..]
Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
     [/usr/local]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
     [PREFIX]

By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc.  You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.

[..]

  --enable-mpi            Compile the parallel version of SIESTA
  --enable-debug          Compile with debugging support
  --enable-fast           Compile with best known optimization flags

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-netcdf=<lib>     use NetCDF library 
  --with-siesta-blas      use BLAS library packaged with SIESTA
  --with-blas=<lib>       use BLAS library 
  --with-siesta-lapack    use LAPACK library packaged with SIESTA
  --with-lapack=<lib>     use LAPACK library 
  --with-blacs=<lib>      use BLACS library 
  --with-scalapack=<lib>  use ScaLAPACK library 
[..]

../Src/./configure --enable-mpi

checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
[..]
checking for mpifc... no
checking for mpxlf... no
checking for mpif90... mpif90
checking for MPI_Init... no
checking for MPI_Init in -lmpi... yes
[..]
checking for sgemm in /opt/openblas/lib/libopenblas.so... yes
checking LAPACK already linked... yes
checking LAPACK includes divide-and-conquer routines... yes
configure: using DC_LAPACK routines packaged with SIESTA due to bug in library. Linker flag might be needed to avoid duplicate symbols
configure: creating ./config.status
config.status: creating arch.make

Edit arch.make:

# 
# This file is part of the SIESTA package.
#
# Copyright (c) Fundacion General Universidad Autonoma de Madrid:
# E.Artacho, J.Gale, A.Garcia, J.Junquera, P.Ordejon, D.Sanchez-Portal
# and J.M.Soler, 1996- .
# 
# Use of this software constitutes agreement with the full conditions
# given in the SIESTA license, as signed by all legitimate users.
#
.SUFFIXES:
.SUFFIXES: .f .F .o .a .f90 .F90

SIESTA_ARCH=x86_64-unknown-linux-gnu--unknown

FPP=
FPP_OUTPUT= 
FC=mpif90
RANLIB=ranlib

SYS=nag

SP_KIND=4
DP_KIND=8
KINDS=$(SP_KIND) $(DP_KIND)

FFLAGS=-g -O2
FPPFLAGS= -DMPI -DFC_HAVE_FLUSH -DFC_HAVE_ABORT
LDFLAGS=

ARFLAGS_EXTRA=

FCFLAGS_fixed_f=
FCFLAGS_free_f90=
FPPFLAGS_fixed_F=
FPPFLAGS_free_F90=

BLAS_LIBS=-L/opt/acml/acml5.3.1/gfortran64/lib -lacml
LAPACK_LIBS=
BLACS_LIBS=-L/usr/lib -lblacs-openmpi -lblacsCinit-openmpi
SCALAPACK_LIBS=-L/usr/lib -lscalapack-openmpi
COMP_LIBS=dc_lapack.a 

NETCDF_LIBS=
NETCDF_INTERFACE=
MPI_LIBS= -L/usr/lib/openmpi/lib -lmpi -lmpi_f90

LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) $(MPI_LIBS) -lpthread

#SIESTA needs an F90 interface to MPI
#This will give you SIESTA's own implementation
#If your compiler vendor offers an alternative, you may change
#to it here.
MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE=.

#Dependency rules are created by autoconf according to whether
#discrete preprocessing is necessary or not.
.F.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F)  $< 
.F90.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $< 
.f.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f)  $<
.f90.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90)  $<

make
cd ../
ln -s Obj/siesta siesta

I added /opt/siesta/siesta-3.2-pl-5 to $PATH.

To test, edit /opt/siesta/siesta-3.2-pl-5/test.mk:

6 #SIESTA=../../../siesta
  7 SIESTA=mpirun -n 2 ../../../siesta

Then

cd /opt/siesta/siesta-3.2-pl-5/Tests/h3po4_2

export LD_LIBRARY_CONFIG=/opt/acml/acml5.3.1/gfortran64/lib 

make

>>>> Running h3po4_2 test...
    ==> Copying pseudopotential file for H...
    ==> Copying pseudopotential file for O...
    ==> Copying pseudopotential file for P...
    ==> Running SIESTA as mpirun -n 2 ../../../siesta
    ===> SIESTA finished successfully

Also, look at work/h3po4_2.out:

* Running on    2 nodes in parallel
>> Start of run:  24-JUL-2015  21:58:13

                           ***********************       
                           *  WELCOME TO SIESTA  *       
                           ***********************       

reinit: Reading from standard input
[..]
elaps:  optical           1       0.000       0.000     0.00
  
>> End of run:  24-JUL-2015  21:58:20

• Tweet
• Share
• Share
• Share
• Share

About Admin MC3

This is dummy text. It is not meant to be read. Accordingly, it is difficult to figure out when to end it. But then, this is dummy text. It is not meant to be read. Period.

Related Post