Even iterative solver cannot solve large number of elements?

[hogglife]

Hi, all

I want to investigate deformation of a component due to thermal expansion. Since there are many small features in the modelled object, I got about 14 million second-order tetrahedron elements by Netgen generated by Salome. After assigning boundary condition, e.g. convective film coefficient, body flux, assigned temperature, and fixed movement etc., I ran it with various solvers, e.g. Pastix, Pardiso (from staff.tu.kielce.pl/rokach/), Iterative Cholesky, Iterative scaling. But all solvers told me “error allocating memory”. An example is as follows:

*ERROR in u_calloc: error allocating memory
variable=prestr, file=ccx_2.17.c, line=413, num=-1762920256, size=8

My workstation has 128 GB RAM.

Since I am new to CalculiX, I want to ask is it really such mesh cannot be solved by CalculiX?

I run CalculiX from PrePoMax and I tried a case with similar
boundary condition but only composed of several cubes which are joined together. It was composed of 570 thousands tetrahedron elements and can be solved with iterative Cholesky in half an hour.

By the way, I have not set any environment variable. Is it useful to let CalculiX use more memory to solve my mesh?

Extra two questions:

1. Is there free mesh tool to generate hexhedron except by sweep method? It seems the Hex_i_j_k in Salome uses sweep method and I guess my model cannot be meshed with sweep method.

2. I heard the Pardiso solver has a out-of-core mode. Is it possible my mesh be solved by that mode of Pardiso?

Thank you!

[Calc_em]

Can’t you simplify the model a bit ? Are all those small features absolutely necessary ? It’s better to avoid such huge meshes.

@hogglife

Is there free mesh tool to generate hexhedron except by sweep method? It seems the Hex_i_j_k in Salome uses sweep method and I guess my model cannot be meshed with sweep method.

Try with Gmsh. Many models can be partitioned in such a way that they become sweep-mesheable. Otherwise, you have to stick with tetrahedrons since the hexa block technique is usually not available.

@hogglife

I heard the Pardiso solver has a out-of-core mode. Is it possible my mesh be solved by that mode of Pardiso?

Give it a try: Calculix with pardiso OUT-OF-CORE mode

But most likely you will end up having to reduce the mesh density anyway.

[hogglife]

Hi Calc_em,

I hope to but I currently do not have direction to simplify it. I will try to think it.

Thank you for the information your provided.

Just for making sure, is it real even iterative solver cannot solve such large number of elements?

[JuanP74]

@hogglife

My workstation has 128 GB RAM

According to the manual with this RAM you could solve 4 million equations (not nodes, not elements, but equations, that is dofs after applying all constraints) (p.382 http://www.dhondt.de/ccx_2.20.pdf). 14 million second-order tetrahedrons seem to be a bit far away of your computer capabilities.

[xyont]

@hogglife

Is there free mesh tool to generate hexhedron except by sweep method?

Someone can try hex-dominate meshing in GMSH, I have shown the capabilities in this thread of PrePoMax forums. It almost work for any object with complexity by simple setting and inclusion of 3D CAD files.

Can be large benefit in computational times and significantly reduction in equation by using linear hexahedral (incompatible) and small volume fraction of linear tetrahedral element.

[feacluster]

It should be able to handle 14 million tets. Calculix needs to be built with a different makefile that allows for large integers. Seems that PrePoMax has not been built with that? You can test out your model here to see if it solves for one iteration:

https://calculix.feacluster.com/calculix.php

[hogglife]

Hi JuanP74,

Thank you for the information your provided.

For my case, the output is as follows. Can you point out which is the amount of equations, or what reference I can look into to know the method to calculate amount of equations? Maybe a basic book about finite element method?

The numbers below are estimated upper bounds

  number of:

   nodes:      2442610
   elements:     15629920
   one-dimensional elements:            0
   two-dimensional elements:            0
   integration points per element:           27
   degrees of freedom per node:            3
   layers per element:            1

   distributed facial loads:      1690912
   distributed volumetric loads:            1
   concentrated loads:            0
   single point constraints:          782
   multiple point constraints:            1
   terms in all multiple point constraints:            1
   tie constraints:            0
   dependent nodes tied by cyclic constraints:            0
   dependent nodes in pre-tension constraints:            0

   sets:           37
   terms in all sets:     48387208

   materials:            2
   constants per material and temperature:            2
   temperature points per material:            1
   plastic data points per material:            0

   orientations:            0
   amplitudes:           16
   data points in all amplitudes:           16
   print requests:            0
   transformations:            0
   property cards:            0

 *ERROR in u_calloc: error allocating memory
variable=prestr, file=ccx_2.20.c, line=429, num=-1762920256, size=8

Edit:
To my knowledge, one DOF of a integration point corresponds to one equation. Then

amount_of_elements x integration_points_per_element x DOFs_each_integration_point + distributed_facial_loads + distributed_volumetric_loads + single_point_constraints + multiple_point_constraints
= 15629920 * 27 * 3 + 1690912 + 1 + 782 + 1
= 1,267,715,216

So in my case, there are 1,267,715,216 equations ? Does the above calculation right?

If all tetrahedrons transit into linear hexhedron, since one hexhedron can be decomposed into 6 tetrahedrons, and one linear hexhedron has 8 nodes,

= 15629920/6 * 8 * 3 + 1690912 + 1 + 782 + 1
= 64,211,376

which is much larger than the 4 millions equations according to my 128 GB RAM.

Does this calculation right? If so, it seems I must simplify the mesh even if linear hexhedron is used…

[hogglife]

Hi xyont,

Thank you for the information on hex-dominate meshing in GMSH. I have used mesh method in ANSYS with the same name in previous work and found it great. Maybe the hex-dominate in GMSH is a choice.

[hogglife]

Hi feacluster,

That really a good and convienent website! Due to some other task, I may have a try in future.

[hogglife]

Hi xyont,

In my case, I use the Partition tools in Salome to connect all components to get conformal mesh in order to avoid setting contact in PrePoMax. I want to let the mesh belonging to one component finer compared to other components since that component is nearly 2 orders smaller in thickness versus others. In Salome I can set mesh size on this very thin component but with GMSH in Salome, I do not know how to set the mesh size for this thin component. When using GMSH for the whole partition, submesh cannot be created either (Salome said it is no meaning to create submesh with GMSH method). Do you know the method?

Thank you!

[JuanP74]

Hi, ccx cast the number of equations when starting the step. I think you can estimate it as follows: #nodes*3 (that’s the number of dofs per node)-#constraint equations.

So you’re about 7.5 million equations, not that bad. You should cut it to half.

[Disla]

@hogglife

I do not know how to set the mesh size for this thin component

It is extrudable.?. If it is a very thin component maybe you could try to find the way to perform a good quality quad surface mesh and then extrude it. ?¿?

[xyont]

Hex-dominant use uniform mesh setting by single parameters of element size factor. It’s apply to whole part object, there’s no need to do partition before.

Mesh refinement of GMSH are based on point, not an edge line or surface. So, this make fine tuning become not easy as Netgen does. Also, mapping to CAD models seems not simple task in GMSH even possible.

I did not yet find any approach to be working in point mapping for the purpose of mesh refinement. The documentation seems complex enough for me, maybe a direct discussion to developer at their official forums can give solution.

[mik]

HI hogglife,

I did not understand very well what is the relation of creating a conformal mesh and not having to set contact in PrePoMax. I mean, if you want to have the surfaces of 2 components glued together during the simulation, you would still need to apply some kind of contact with surface behavior or a tie constraint.

But if the mesh size in one of the components is smaller than in the other one, then you would not have conformal mesh anyway, no? I am a bit lost with that description.

In salome, for setting up the mesh size in different components, I set up the 1D algorithm as wire discretisation, and the hypothesis as number of segments or local length. So I create submeshes ono the edges/wires to set up the number of elements on the different edges/wires of the part, and then mesh.

If the 2 parts/components are of the same material, you could then merge the meshes of the 2 components for not having to declare contact/tie later. That is done in salome mesh → mesh → build compound → create new mesh named: → select the meshes of the parts → apply

I attach some figures to illustrate this all:

And here there are 2 parts with non-conformal meshes joined in a compound, and hence creating a new part with this non-continuous mesh.

This particular example has some problems in continuity if used for thermal, but it is just a fast example I built to show the compound capability.

Hope this helps also for reducing the number of elements. 14.000.000 elements are quite a lot of elements… Unless your thermal expansion simulation is of a quite large part…

Regards
M

[hogglife]

Hi mik,

Sorry for my late reply!

Thank you for your explanation on mesh in Salome. However, for GMSH, Salome seems not support submesh.

According to above discussion, for my case, since tetrahedron will lead to huge amount of element and hard to calculate by Calculix, I need to try split the geometry to apply sweep to obtain hexhedron mesh, or simplify the geometry. Hexhedron-dominate mesh in Gmsh is attractable but seems with problem in setting small mesh size for region where I want to have fine mesh, and thus still get huge amount of element and may be failed to solve by Calculix.

---

About not defining contact, I am not sure. Maybe I found a way to model bodies coupled tightly by using conformal mesh. Contact does not setting . Here is an example.

1st. My geometry is as follows.

2nd. Make a partition by selecting all the three bodies as Objects and none as Tools so that the 3 bodies are connected and a body is generate which includes the interface information among the 3 bodies. This is explained in the doc of Salome.

3rd. Create group for the 3 bodies and necessary faces in the body generated by partition.

4th. Mesh the partition with Netgen 1D-2D-3D and set a fine mesh size for the inner box. There will be a mesh size transition at boundary between the inner box and outer box.

5th Export the mesh of the partition as .unv and import into PrePoMax. There will be individual element sets for the group defined on SOLID in Salome. The group defined on faces in Salome does not show as element sets in PrePoMax but shows as Node set.
Different materials can be defined on the imported element sets for SOLID. I have not tried to define boundary condition on the Node set for the face group. For defining boundary condition I manually select in graphical interface in PrePoMax. In addition, there are extra Shell_xxx shown as a part in PrePoMax and I delete them. Contact is not defined.

Thermal-displacement calculation in PrePoMax can be done with this model.

Due to limited time, I have not explored thoroughly this workflow and have not validated against validation case.

[Disla]

Hi hogglife,

You could also try to start from a Tronco conical shape and complete the geometry around the refined core by mirror. Coincident nodes should have to be finally merged.

[hogglife]

Hi Disla,

Thank you for your wonderful demonstration! Unfortunately, the above is a simple example and my real geometry is more complex. There are many curved lines embeded in a plate and many small convexes exist at the surface of the plate. It seems hard work is needed to get hexhedron mesh.

[NorbertH]

Hi Hogglife,

you could try it with cubit.

Simple things like the example above are just a few lines of input.

i am currently working on a component to do the preprocessing for calculix with cubit. heattransfer on solids already works.

if you want to try my component look at my fork on github, its still an alpha version. so bugs will most likely occur ^^

GitHub - NorbertHofbauer/Cubit-CalculiX: development repo

[hogglife]

Hi NorbertH,

Thank you for informing this powerful software. Unfortunately, by searching Cubit, I think I do not belong to its license …

[NorbertH]

Hi Hogglife,
To test it you can either use the learn license without any time limit. Only downside is, that the export is limited to 50.000 elements.
Or you get a trial license for 30 days without limitations.
The only thing you have to do is to register an account on the website. Takes 1 minute.
Learn
trial