PaStiX diverges on Linux

Dear CalculiX users,

we are trying to get the PaStiX solver for CalculiX to work under Linux. We have compiled PaStiX under Ubuntu 20.04 for this purpose. Pastix also runs, but the solver crashes in the first iteration. I have calculated the same case with the version provided by Rafal (same CaluculiX 2.16 and same PaStiX version) and the case runs. In the Linux version, the flow of the solution seems to have changed:
While in the Windows version:

  • iteration 50 :
    total iteration time 0.0102
    error 8.4295e-07
    Time for refinement: 0.4675

CSC Conversion Time: 0.008051
Init Time: 0.000117
Factorize Time: 0.041518
Solve Time: 0.473739
Clean up Time: 0.000000


The Linux version then outputs:

  • iteration 50 :
    total iteration time 0.0295
    error 2.8446e-07
    Time for refinement: 1.1590
    Not reusing csc.
    ±------------------------------------------------+
  • PaStiX : Parallel Sparse matriX package     +
    

±------------------------------------------------+
Version: 6.0.1
Schedulers:
sequential: Enabled
thread static: Started
thread dynamic: Disabled
PaRSEC: Started
StarPU: Disabled
Number of MPI processes: 1
Number of threads per process: 1
Number of GPUs: 0
MPI communication support: Disabled
Distribution level: 2D( 256)

The consequence seems to be that the solution diverges:
Linux:

Using up to 1 cpu(s) for the energy calculation.
average force= 11484.186711
time avg. forc= 11484.186711
largest residual force= 602620.839813 in node 1355 and dof 3
largest increment of disp= 8.271366e-03
largest correction to disp= 8.271366e-03 in node 145 and dof 3
Windows
Using up to 1 cpu(s) for the energy calculation.

average force= 1.435569
time avg. forc= 1.435569
largest residual force= 56.306643 in node 983 and dof 3
largest increment of disp= 1.365542e-05
largest correction to disp= 1.365542e-05 in node 81 and dof 3

Is there anything else that needs to be configured for the Linux version? Can someone put me on the right track here?

Many thanks

Ulrich

I found pastix took a lot longer than pardiso when running with older GPU ( Tesla K80 ). It’s quite tricky to get working so I kind of gave up on it when I saw little performance improvements with older GPU. Maybe I had some wrong settings … But I did chat with the developers and they kind of agreed. See:

Thank you for the information. We have been able to solve the problem with CalculiX 19. In ccx-16 we had problems wih PaStiX. I really hope we will see better speed-up. Our case is relatively small at the moment (about 40e3 equations and 1.5e6 nnz). But we are trying to solve this in FSI with OpenFOAM, can only use direct dynamic analysis and in some cases CalculiX takes 30-40 iterations to converge for one time step. In total we have to run approx. 60000 increments with 2-3 FSI iterations each. Hopefully for cases like this the speed-up by PaStiX is better.

40,000 equations should solve in a second with even Spooles :slight_smile: But if it is taking multiple iterations/increments then that is more a Calculix algorithm issue…