#How to get the size of the file inside a xz archive?

1 messages · Page 1 of 1 (latest)

past flare
#

.

grand jasper
#

xz is a compression algorithm, i assume you mean .tar.xz, which would mean you decompress the file with std.compress.xz and then use std.tar

worn ether
#

some formats have the decompressed size in the header but i think xz doesn't

grand jasper
#

ah, that's another possible interpretation, the question is a bit ambiguous

worn ether
#

ah each block has the decompressed size but that's going to be nontrivial to use

#

tar has no index so you can't decompress just the beginning and get the answer that way eitehr

#

off the top of my head i think the blocks are bit streams so you can't just iterate through them adding up all the decompressed sizes without decompressing the whole thing. not 100% sure though

#

3.1.4. Uncompressed Size of https://tukaani.org/xz/xz-file-format.txt

  It should be noted that the only reliable way to determine
  the real uncompressed size is to uncompress the Block,
  because the Block Header and Index fields may contain
  (intentionally or unintentionally) invalid information.
grand jasper
#

if the question is "what is the decompressed size of some xz-compressed data", here are two possible approaches using the current standard library:

const std = @import("std");

test "decompressed size" {
    const data = @embedFile("xz-size.xz");

    const decompressed_size = blk: {
        var in_stream = std.io.fixedBufferStream(data);
        var xz_stream = try std.compress.xz.decompress(std.testing.allocator, in_stream.reader());
        defer xz_stream.deinit();

        const decompressed = try xz_stream.reader().readAllAlloc(std.testing.allocator, std.math.maxInt(usize));
        defer std.testing.allocator.free(decompressed);

        break :blk decompressed.len;
    };

    const counted_size = blk: {
        var in_stream = std.io.fixedBufferStream(data);
        var xz_stream = try std.compress.xz.decompress(std.testing.allocator, in_stream.reader());
        defer xz_stream.deinit();

        var counting_writer = std.io.countingWriter(std.io.null_writer);
        const Fifo = std.fifo.LinearFifo(u8, .{ .Static = 4096 });
        var fifo = Fifo.init();

        try fifo.pump(xz_stream.reader(), counting_writer.writer());

        break :blk counting_writer.bytes_written;
    };

    try std.testing.expectEqual(decompressed_size, counted_size);
}
  • the decompressed_size approach decompresses to memory and gets the length (so it allocates enough memory for the full decompressed size)
  • the counted_size approach decompresses to a statically sized fifo and throws away the result using std.io.null_writer, so there's no heap allocation beyond what's needed by std.compress.xz for its internal state
past flare
#

Just a classic .xz archive

#

like txt.xz

grand jasper
#

i don't think that's an archive, just an xz compressed file

#

see #1258963668146585653 message though for how to get the decompressed size

past flare
#

My bad

#

I used the wrong term

past flare
grand jasper
past flare
#

Ok!