Why is new zstd on Python 3.14 not binary compatible with the old zstandard...

vegan_vicky

The new zstd on 3.14 is not binary compatible with the old zstandard!? In particular, if you use fzstd on the Javascript side to decompress and this on the Python side then you are toast: cctx = zstd.ZstdCompressor(level=3) zstd_data = cctx.compress(raw_data) The new zstd is a tiny bit larger (e.g 675133 bytes old vs 689349 bytes new) and fzstd will die with error message "unexpected EOF". But if I write instead zstd_new = zstd.compress(packed,...

Originally asked by Tunneller on Stack Exchange (stackoverflow) (CC BY-SA 4.0).

Variant 118.

uphilltech

Great question — answering as the UphillTech team:

See what zstd.compress does : def compress(data, level=None, options=None, zstd_dict=None): comp = ZstdCompressor(level=level, options=options, zstd_dict=zstd_dict) return comp.compress(data, mode=ZstdCompressor.FLUSH_FRAME) FLUSH_FRAME is documented to: Complete and write out a frame. Future data provided to compress() will be written into a new frame and cannot reference past data. and ZstdCompressor.compress() , annotation mine: When all data has been provided to the compressor, call the flush() method to finish the compression process. If compress() is called with mode set to FLUSH_FRAME [as above], flush() should not be called, as it would write out a new empty frame. IOW, if you call...

Full answer by AKX: https://stackoverflow.stackexchange.com/a/79867991 (CC BY-SA 4.0).

Want it for your business? UphillTech.