Hi there,
I compiled the 2.23 code with MPI to solve with MKL Pardiso. Running a large problem (>5M equations) on a cluster environment, I faced the problem that the master node runs out of memory, i.e. I received the error
Using up to 16 cpu(s) for the symmetric stiffness/mass contributions.
*ERROR in u_calloc: error allocating memory
variable=au1, file=mafillsmasmain.c, line=179, num=6264344448, size=8
Looking into mafillsmasmain.c + the manual I found that the size of the variable au1
NNEW(au1,double,(long long)num_cpus*(nzs[2]+nzs[1]));
scales with the number of cpus and can be reduced by setting the variable CCX_NPROC_STIFFNESS to a lower value. Running the same problem with CCX_NPROC_STIFFNESS=4 actually run without memory problems.
As I understand that, the number of equations is thus limited by the memory of the master node for now. Has someone tried to change the code such that the assembly of matrices can be distributed to different nodes with individual memory on a cluster (using MPI)?
Thanks