I’d like to propose adding native zstd (Zstandard) compression support to both ccx and cgx.
Using standard gzip or uncompressed files can be slow or storage-heavy. Integrating zstd directly gives us much faster compression and decompression speeds with excellent compression ratios, making reading/writing data vastly more efficient without relying on external scripts or pipes.
I actually have a working version running on my setup. The compressed .frd.zst file opens directly in cgx with no noticeable performance drop, while the file size drops dramatically from 98 MB down to around 13 MB.
Proposed additions:
Native .zst file support for reading and writing in ccx and cgx (e.g., opening .frd.zst directly).
Options to adjust compression levels and multi-threading.
What do you think? Would you find native zstd support useful in your workflow?
How does zstd compression compare to uncompressed binary output? (binary output is smaller than ASCII as well, faster to read/write and ccx/cgx already support it natively)
Is there any performance penalty for large files? (several GB)
Yet my biggest concern is that any downstream postprocessing tool would not be able to natively read the compressed frd. So I currently think it would make more sense to compress/decompress the result files outside of ccx/cgx if necessary.
The differences in timing most likely stem from Turbo Boost technology (AMD Ryzen 7700).
@xyont for me, the .frd.zst format is better (than .frdz) because I can manually add an .frd file to the archive (e.g., using Windows Explorer), and if cgx supports it, I can launch the file without any extra steps.