Converter ccx2paraview for large FRD files

Dear CalculiX users,

I’ve improved ccx2paraview converter to process FRD files increment by increment, which makes possible to process really big results files. Before making a new release it would be good to test the converter on some FRD files bigger than 1 GB. Unfortunately, I do not have such a big model. Please, if you have, could you share your huge FRD? Thank you in advance.

The code is here:

Related issue:

3 Likes

Hi imirzov,

first, thanks for your work on the converter. It is really helpful.

I tried the new version, but I get an error message when loading the PVD file. The individual VTU files look ok though.

What I like is that the individual time steps are now processed one after the other, so you can see the progress. However, the total conversion time is a bit longer (in my test case with a nearly 800MByte file from 14 minutes to 20 minutes).

This is the error message:

ERROR: In C:\bbd\ecd3383f\build\superbuild\paraview\src\ParaViewCore\VTKExtensions\Default\vtkXMLCollectionReader.cxx, line 444
vtkPVDReader (0000021BFAA6E270): Could not determine the data type for the first dataset. Please make sure this file format is supported.

ERROR: In C:\bbd\ecd3383f\build\superbuild\paraview\src\VTK\Common\ExecutionModel\vtkExecutive.cxx, line 782
vtkPVCompositeDataPipeline (0000021BFF548700): Algorithm vtkPVDReader(0000021BFAA6E270) returned failure for request: vtkInformation (0000021B91025380)
Debug: Off
Modified Time: 324273
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FORWARD_DIRECTION: 0
ALGORITHM_AFTER_FORWARD: 1

ERROR: In C:\bbd\ecd3383f\build\superbuild\paraview\src\ParaViewCore\VTKExtensions\Default\vtkXMLCollectionReader.cxx, line 444
vtkPVDReader (0000021BFAA6E270): Could not determine the data type for the first dataset. Please make sure this file format is supported.

ERROR: In C:\bbd\ecd3383f\build\superbuild\paraview\src\VTK\Common\ExecutionModel\vtkExecutive.cxx, line 782
vtkPVCompositeDataPipeline (0000021BFF548700): Algorithm vtkPVDReader(0000021BFAA6E270) returned failure for request: vtkInformation (0000021B91025380)
Debug: Off
Modified Time: 324273
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FORWARD_DIRECTION: 0
ALGORITHM_AFTER_FORWARD: 1

ERROR: In C:\bbd\ecd3383f\build\superbuild\paraview\src\ParaViewCore\VTKExtensions\Default\vtkXMLCollectionReader.cxx, line 444
vtkPVDReader (0000021BFAA6E270): Could not determine the data type for the first dataset. Please make sure this file format is supported.

ERROR: In C:\bbd\ecd3383f\build\superbuild\paraview\src\VTK\Common\ExecutionModel\vtkExecutive.cxx, line 782
vtkPVCompositeDataPipeline (0000021BFF548700): Algorithm vtkPVDReader(0000021BFAA6E270) returned failure for request: vtkInformation (0000021B91025380)
Debug: Off
Modified Time: 324273
Reference Count: 1
Registered Events: (none)
Request: REQUEST_DATA_OBJECT
FORWARD_DIRECTION: 0
ALGORITHM_AFTER_FORWARD: 1

1 Like

You may want to try CalculiX Extras by Dr. Gustafson:

Hi Nobody-86,

Thank you for testing PVD files. Yep, I’ve found a mistake in a function exporting the PVD. Now it is fixed - you can use the code by the same URL above.

Yes, unfortunately, processing time is now approximately 30% longer. But I believe it could be slightly optimized. To accomplish this task I need your 800 MB file. Please, can you share it?

Hi @imirzov , attached a 3 GB .frd result file from a no lineal analysis (bolt preloading + internal pressure, with plastic material, 20 increments total). CalMED took 5 minuts to convert to .rmed result (compatible with Paraview)

cylinder_lr

2 Likes

To accomplish this task I need your 800 MB file. Please, can you share it?

I could share the model, however I don’t know where to upload such a large amount of data.

Dear @SergioP, thank you for sharing the model!
It took 28.5 minutes to convert your FRD to both VTK and VTU with ccx2paraview on my laptop. I will try to improve the algorithm. While converting your file I’ve got a few ideas :slight_smile:

1 Like

@SergioP, this version is 12% faster, so finally I made a release:

Thank you for 3 GB file. I will continue to improve the speed.

Have you tried the RMED format? I found more easy as they put all the increments in a single file. The last time that I tried to made an animation of a VTK/VTU result set with lot of increments, the computer (a good one Windows workstation) suffer a lot reading the individual increments/frames, but with the RMED was smooth.

Another thing that I found interesting, was a converter for OpenFOAM results, they manage to write the units in the name of every field, would be nice to have something similar in CCX results (don´t know how as they are unitless…)

1 Like

@SergioP,
No, I haven’t try RMED format. Looks like naked Paraview (without Code Aster) does not support it:

I do not want to contribute to Code Aster related software, because it’s not fully open and not present on the GitHub, while CalculiX already is. Moreover, Abaqus is closer to me, so I prefer to support CalculiX, even if I do not use it often.

To solve the issue with many files (a file per time increment) I’m going to use XDMF format, which is fully supported by Paraview. I have quite an old feature request for the XDMF:

Also I do not like the idea to add units during format conversion. Converter has to convert data, it shouldn’t modify it. I’ve added Mises and principal stress calculation, and I’m not sure it was a good idea, since those values could be calculated on the fly directly in Paraview.

Please, in future do not hesitate to create issues on GitHub for the current project.

1 Like

Is there a way to convert an input deck ( .inp ) file directly to vtu format? I am mainly interested in viewing the mesh without any results in paraview.

Currently no. Coding this functionality is not so difficult, but I haven’t planned it. Too busy right now, sorry.

2 Likes

No worries. I did some googling and found this tool:

I tested it on some simple 2D and 3D ( C3d10 ) meshes and it worked.

meshio-convert shape.inp shape.vtu

also try *NO ANALYSIS keyword

2 Likes

Hello,

Just to report a bug on the ccx2paraview converter. Not sure if is the right place. But anyway here it is.
I think the author keeps an eye into the forum, so first of all thanks for coding this very useful tool (at least for the workflow i use).

The bug occurs when placing a perturbation step for a frequency analysis after a static step.

Example:

** == ANALYSIS SECTION ==
**
*TIME POINTS, NAME=T0
0.200,0.400,0.600,0.800,1.000
*STEP
*STATIC
*BOUNDARY
RefNode,6,6,0.03
*EL FILE, TIMEPOINTS=T0
S,E
*NODE FILE, TIMEPOINTS=T0
U,RF
*END STEP
*STEP,PERTURBATION
*FREQUENCY
10
*EL FILE
S,E
*NODE FILE
U,RF
*END STEP

Error:

[...]
INFO: Writing n-twist_model.05.vtu
Traceback (most recent call last):
  File "/home2/jl/workspace/software_devel/ccx2paraview/ccx2paraview.py", line 992, in <module>
    main()
  File "/home2/jl/workspace/software_devel/ccx2paraview/ccx2paraview.py", line 987, in main
    ccx2paraview.run()
  File "/home2/jl/workspace/software_devel/ccx2paraview/ccx2paraview.py", line 904, in run
    result_blocks = self.frd.parse_results(step, inc) # NOTE Could be empty list []
  File "/home2/jl/workspace/software_devel/ccx2paraview/ccx2paraview.py", line 696, in parse_results
    result_blocks.append(self.calculate_principal(b))
  File "/home2/jl/workspace/software_devel/ccx2paraview/ccx2paraview.py", line 780, in calculate_principal
    eigenvalues = np.linalg.eigvals(tensor).tolist()
  File "<__array_function__ internals>", line 5, in eigvals
  File "/usr/lib/python3/dist-packages/numpy/linalg/linalg.py", line 1063, in eigvals
    _assert_finite(a)
  File "/usr/lib/python3/dist-packages/numpy/linalg/linalg.py", line 209, in _assert_finite
    raise LinAlgError("Array must not contain infs or NaNs")
numpy.linalg.LinAlgError: Array must not contain infs or NaNs

So the error occurs when trying to convert the frequency steps.

The procedure finishes without error if we comment the code between lines 694 to 700.

                   if b.name == 'S':
                        result_blocks.append(self.calculate_mises_stress(b))
                        result_blocks.append(self.calculate_principal(b))
                    if b.name == 'E':
                        result_blocks.append(self.calculate_mises_strain(b))
                        result_blocks.append(self.calculate_principal(b))

but there is still a warning:

[...]
INFO: Writing n-twist_model.05.vtu
INFO: Step 6, time 2750.3, U, 3 components, 34432 values
INFO: Step 6, time 2750.3, S, 6 components, 34432 values
WARNING: 69264 NaN values are converted to 0.0
INFO: Step 6, time 2750.3, E, 6 components, 34432 values
INFO: Step 6, time 2750.3, RF, 3 components, 34432 values
WARNING: 48780 NaN values are converted to 0.0
INFO: Step 6, time 2750.3, ERROR, 1 components, 34432 values
[...]

to be honest to my workflow it would be usefull if there was a way to select if to compute these additional quantities or not (misses and principal values). This is because I use the vtk api to further compute additional quantities (including the principal vectors, which make the initial computation of principal values redundant).

Hope this info is useful,
thanks again,
regards,
Jorge

Jorge, thanks for reporting problems.
The right place is here: Perturbation step for a frequency analysis after a static step · Issue #29 · calculix/ccx2paraview · GitHub
Next few month I’ll be extremely busy. So will have a look later. Sorry.

Can you share an .frd result file to try?

Regards