Pastix4Calculix - Build and installation issues

Hello,

I was wondering if someone here successfully installed Pastix4Calculix on a CentOS system?
I have explained my issue in detail here (With CUDA-'Threads::Threads' NOT FOUND, Without CUDA - fatal error: pastix_ccores.h · Issue #11 · Dhondtguido/PaStiX4CalculiX · GitHub) .
I am posting the question here again to reach a broader user base.
I have looked in the previous posts, and I did find people facing some similar issues, but their solutions were not quite clear or did not work for me.
PaStix4CalculiX compilation error
Any suggestions?

In addition, during Parsec installation, even though MPI is turned off, some MPI functions were used during the build leading to errors. I could resolve them by just loading the mpich library and include directory paths. But I still find this strange, on why use MPI when I specifically turn it off. Any one faced this too?

Any suggestions would be of great help.

Thanks and Regards,
Hem

My 2 cents is CentOS 7 is too old to install Pastix. The GCC is 4.8.5 which is very old . I would first try building Pastix (non-GPU) per these instructions:

https://solverstack.gitlabpages.inria.fr/pastix/md_docs_doxygen_chapters_Pastix_Install.html

My initital attempt at doing so on Centos 7 failed. It created an executable but the simple example problem says the results are wrong…

@feacluster Unfortunately I do not have control on the OS. Do you mean Pastix instead of Pastix4Calculix?
I use GCC 8.0.2 for my build. And I already tried Pastix4Calculix without CUDA. It fails to create a ton of header files like pastix_ccores.h and the rest in /build/kernel folder and stops compiling.

Correct, yes just Pastix. Usually when I try and install something complex I start with the simplest components to see if they will install. GCC 8 sounds new enough though… Try just Pastix and see if you can get that to work.

BTW, errors about missing include files is usually some path problem…

Okay I will try the standard Pastix. To my understanding the missing headers are to be created by the build. At least these headers were not included in the source code.

1 Like

Okay, so I was at least able to install Pastix4Calculix without CUDA. I had to turn on the MPI though. Not turning on the MPI lead to conflicting MPI function definitions in nompi.h of Pastix4Calculix and mpi.h used by parsec. Some how parsec could not be compiled without MPI libraries, even though the MPI was set to OFF.

1 Like

Interesting ! Can you share your cmake command?

The MKL Library and Open MPI 3.2.1 bin and library paths are exported in to PATH and LD_LIBRARY_PATH environment variables Apriori.
Hwloc 2.1 :

#!/bin/bash
### Source Code https://www.open-mpi.org/software/hwloc/v2.1/
umask 022
./configure --prefix=/usr/PaStiX/hwloc_i8 --enable-static=yes CC=gcc CXX=g++
make -j2
make install

Scotch 6.0.8

#!/bin/bash
### Source Code https://gforge.inria.fr/frs/download.php/file/38114/scotch_6.0.8.tar.gz
umask 022
cd src
ln -sf Make.inc/Makefile.inc.x86-64_pc_linux2 Makefile.inc
sed -i '/CFLAGS/ s/$/ -DINTSIZE64/' Makefile.inc

make -j8 scotch
make -j8 esmumps
make prefix=/usr/PaStiX/scotch_i8 install

parsec:

#!/bin/bash
### Source Code - git clone https://bitbucket.org/mfaverge/parsec.git
rm -rf build
if ! [[ -d build ]]; then
mkdir build
fi
cd build
INSTALLPATH="/usr/PaStiX/parsec_i8"
umask 022

# fixes
sed -i '/-1 == cpu/i return cpu;' ../parsec/bindthread.c

cmake \
    -DCMAKE_CXX_COMPILER=g++ \
    -DCMAKE_C_COMPILER=gcc \
    -DCMAKE_Fortran_COMPILER=gfortran \
    -DPARSEC_DIST_WITH_MPI=ON \
    -DCUDA_TOOLKIT_ROOT_DIR=/usr/CUDA/v10.2/cuda-10.2 \
    -DCUDA_DIR=/usr/CUDA/v10.2/cuda-10.2 \
    -DCMAKE_CUDA_HOST_COMPILER=gcc \
    -DPARSEC_GPU_WITH_CUDA=ON \
    -DCUDA_USE_STATIC_CUDA_RUNTIME=ON \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=${INSTALLPATH} \
    -DPARSEC_WITH_DEVEL_HEADERS=ON \
    -DCMAKE_LIBRARY_PATH=/usr/CUDA/v10.2/cuda-10.2/lib64/stubs \
    -DHWLOC_DIR=/usr/PaStiX/hwloc_i8 \
	-DCMAKE_C_FLAGS="-fopenmp" \
    ..

make -j8
make install

Pastix

#!/bin/bash
### Source Code - git clone https://github.com/Dhondtguido/PaStiX4CalculiX
###
rm -rf build
if ! [[ -d build ]]; then
    mkdir build
fi
cd build
export LD_RUN_PATH=$LD_RUN_PATH:/usr/PaStiX/hwloc_i8/lib:/usr/PaStiX/parsec_i8/lib:/usr/PaStiX/scotch_i8/lib
export INCLUDE_PATH=$INCLUDE_PATH:/usr/PaStiX/hwloc_i8/include:/usr/PaStiX/parsec_i8/include:/usr/PaStiX/scotch_i8/include
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/PaStiX/hwloc_i8/lib/pkgconfig:/usr/PaStiX/parsec_i8/lib/pkgconfig
export PATH=$PATH:/usr/PaStiX/scotch_i8/bin:/usr/PaStiX/parsec_i8/bin:/usr/PaStiX/hwloc_i8/bin

cmake \
	-DHWLOC_DIR=/usr/PaStiX/hwloc_i8 \
	-DCUDA_TOOLKIT_ROOT_DIR=/usr/CUDA/v10.2/cuda-10.2 \
	-DCMAKE_INSTALL_PREFIX=/usr/PaStiX/pastix_i8 \
	-DCMAKE_BUILD_TYPE=Release \
	-DPASTIX_WITH_PARSEC=ON \
	-DPARSEC_DIR=/usr/PaStiX/parsec_i8 \
	-DSCOTCH_DIR=/usr/PaStiX/scotch_i8 \
	-DPASTIX_WITH_CUDA=OFF \
	-DCUDA_DIR=/usr/CUDA/v10.2/cuda-10.2 \
	-DPASTIX_ORDERING_SCOTCH=ON \
	-DCMAKE_C_COMPILER=gcc \
	-DCMAKE_CXX_COMPILER=g++ \
	-DCMAKE_Fortran_COMPILER=gfortran \
	-DCMAKE_LIBRARY_PATH=/usr/CUDA/v10.2/cuda-10.2/lib64/stubs \
    -DPASTIX_WITH_MPI=ON \
	-DCMAKE_C_FLAGS="-fopenmp -I/usr/CUDA/v10.2/cuda-10.2/include -I/opt/ohpc/pub/mpi/mpich-gnu8-ohpc/3.2.1/include" \
    ..
make -j8
make install

Makefile for CCX 2.18 with Pastix :

CFLAGS = -Wall -O2 -fopenmp -fpic -I /usr/SPOOLES_2_2 -I$(PASTIX_INCLUDE) -I /opt/ohpc/pub/mpi/mpich-gnu8-ohpc/3.2.1/include -DARCH="Linux" -DSPOOLES -DARPACK -DMATRIXSTORAGE -DLONGLONG -DPASTIX -DPASTIX_FP32 -DUSE_MT=1 -std=c99 -D_POSIX_C_SOURCE=199309L
FFLAGS = -Wall -O2 -fopenmp -fpic -fdefault-integer-8

CC=cc
FC=gfortran

.c.o :
	$(CC) $(CFLAGS) -c $<
.f.o :
	$(FC) $(FFLAGS) -c $<

include Makefile.inc

SCCXMAIN = ccx_2.18.c

OCCXF = $(SCCXF:.f=.o)
OCCXC = $(SCCXC:.c=.o)
OCCXMAIN = $(SCCXMAIN:.c=.o)

DIR=/usr/SPOOLES_2_2
MKL_LIB=/LPP/OPT/x86_64/intel/parallel_studio_xe_2019/mkl/lib/intel64

PASTIX_INCLUDE=/usr/PaStiX/pastix_i8/include
PASTIX_LIBS=/usr/PaStiX/hwloc_i8/lib/libhwloc.so $(MKL_LIB)/libmkl_intel_lp64.so $(MKL_LIB)/libmkl_sequential.so $(MKL_LIB)/libmkl_core.so /lib64/libm.so \
  /usr/PaStiX/pastix_i8/lib/libpastix.a /opt/ohpc/pub/mpi/mpich-gnu8-ohpc/3.2.1/lib/libmpi.a -lpthread -lm /usr/PaStiX/pastix_i8/lib/libspm.a \
  /usr/PaStiX/pastix_i8/lib/libpastix_parsec.a /usr/PaStiX/pastix_i8/lib/libpastix_kernels.a -lrt \
  /usr/PaStiX/parsec_i8/lib/libparsec.so \
  /usr/PaStiX/scotch_i8/lib/libscotch.a /usr/PaStiX/scotch_i8/lib/libscotcherrexit.a -lpthread -lz -lm /usr/lib64/libstdc++.so.6 /usr/lib64/libgomp.so.1

LIBS = \
     $(DIR)/MT/src/spoolesMT.a \
     $(DIR)/spooles.a \
     /usr/ARPACK/libarpack_x86.a \
     $(PASTIX_LIBS) \
     -lpthread -lm -lc
export LDFLAGS="-Wl,--copy-dt-needed-entries"
ccx_2.18_i8: $(OCCXMAIN) ccx_2.18.a $(LIBS)
	./date.pl; $(CC) $(CFLAGS) -c ccx_2.18.c; $(FC) -Wall -O2 -o $@ \
	$(OCCXMAIN) ccx_2.18.a $(LIBS)

ccx_2.18.a: $(OCCXF) $(OCCXC)
	ar vr $@ $?
2 Likes