#🔒 How to use compression.zstd, in 3.14.0b3?

6 messages · Page 1 of 1 (latest)

toxic olive
#

According to the What’s new in Python 3.14 page, shutil (among other modules) should Just Work with zstd.

The new compression.zstd module provides compression and decompression APIs for the Zstandard format via bindings to Meta’s zstd library. Zstandard is a widely adopted, highly efficient, and fast compression format. In addition to the APIs introduced in compression.zstd, support for reading and writing Zstandard compressed archives has been added to the tarfile, zipfile, and shutil modules.

However, this isn't reflected in the shutil docs:

By default shutil provides these formats:

*zip*: ZIP file (if the zlib module is available).
*tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
*gztar*: gzip’ed tar-file (if the zlib module is available).
*bztar*: bzip2’ed tar-file (if the bz2 module is available).
*xztar*: xz’ed tar-file (if the lzma module is available).

And actually running it results in an error:

print(shutil.get_unpack_formats())
shutil.unpack_archive("/home/user/workdir/my.tar.zst", "/tmp/some_temp_dir")

[('bztar', ['.tar.bz2', '.tbz2'], "bzip2'ed tar-file"), ('gztar', ['.tar.gz', '.tgz'], "gzip'ed tar-file"), ('tar', ['.tar'], 'uncompressed tar file'), ('xztar', ['.tar.xz', '.txz'], "xz'ed tar-file"), ('zip', ['.zip'], 'ZIP file')]
shutil.ReadError: Unknown archive format '/home/user/workdir/my.tar.zst'```

(ran out of characters, my attempt an manual wiring is in the next message)
frozen oracleBOT
#

@toxic olive

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
toxic olive
#

shutil does have register_archive_format, so I then tried to manually hook things up... but this still failed.

import shutil
import compression.zstd as zstd
import tempfile
from pathlib import Path


def ztar_decompress(location: str, exdir: str, **kwargs):
  """
  Extract a zstd-compressed file to a temporary directory, then pass it back to shutil to untar.
  """
  zfile = zstd.ZstdFile(location)
  tempdir = Path(tempfile.mkdtemp())
  tfile = tempdir / zfile.name
  tfile.write_bytes(zfile.read())
  shutil.unpack_archive(tfile, exdir)
  shutil.rmtree(tempdir)

shutil.register_unpack_format(
  "zstdtar",
  [ ".tar.zst", ".tar.zstd" ],
  ztar_decompress)

print(shutil.get_unpack_formats())```
```bash
$ uv run zstdtar.py
Traceback (most recent call last):
  File "/home/user/workdir/zstdtar.py", line 2, in <module>
    import compression.zstd as zstd
  File "/home/user/.local/share/uv/python/cpython-3.14.0b3-linux-x86_64-gnu/lib/python3.14/compression/zstd/__init__.py", line 29, in <module>
    import _zstd
ModuleNotFoundError: No module named '_zstd'```
#

Currently I'm using uv to manage python versions - is it just mangling the install, or is there a bigger issue here?

frozen oracleBOT
#

@toxic olive

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.