tgz decompress | Zig Programming Language | Page 1

signal sluice Apr 9, 2023, 5:43 AM

#

I have a tgz file which i want to decompress using zig but i can't understand how can i use std.tar or std.compress.gzip. can anyone explain me the process how can i do it?

dry wolf Apr 9, 2023, 5:54 AM

#

const allocator = std.heap.c_allocator;

// Open the file
const handle = try std.fs.cwd().openFile("my.tar.gz", .{});
defer handle.close();

// Create a decompresser
const decompresser = try std.compress.gzip.decompress(allocator, handle.reader());
defer decompresser.deinit();

// Untar the archive using the decompresser
try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{});

#

this will decompress the archive and extract the files into the CWD

signal sluice Apr 9, 2023, 6:11 AM

#

i am getting this error

#

tar.zig:15:60: error: expected type '*compress.gzip.Decompress(io.reader.Reader(fs.file.File,error{AccessDenied,BrokenPipe,ConnectionResetByPeer,ConnectionTimedOut,InputOutput,IsDir,NetNameDeleted,NotOpenForReading,OperationAborted,SystemResources,Unexpected,WouldBlock},(function 'read')))', found '*const compress.gzip.Decompress(io.reader.Reader(fs.file.File,error{AccessDenied,BrokenPipe,ConnectionResetByPeer,ConnectionTimedOut,InputOutput,IsDir,NetNameDeleted,NotOpenForReading,OperationAborted,SystemResources,Unexpected,WouldBlock},(function 'read')))' try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{}); ~~~~~~~~~~~~^~~~~~~ tar.zig:15:60: note: cast discards const qualifier

#

@dry wolf

dry wolf Apr 9, 2023, 6:13 AM

#

ah woops

#

make decompresser var

#

as well as handle probably

signal sluice Apr 9, 2023, 6:15 AM

#

now getting this

#

thread 9916 panic: TODO: unimplemented: tar ModeMode.executable_bit_only C:\zig\lib\std\tar.zig:95:13: 0x7ff67dff7a0d in pipeToFileSystem__anon_3818 (tar.exe.obj) @panic("TODO: unimplemented: tar ModeMode.executable_bit_only"); ^ D:\zig\unzipping\tar.zig:15:67: 0x7ff67dff9885 in main (tar.exe.obj) try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{}); ^ C:\zig\lib\std\start.zig:518:80: 0x7ff67dff9f20 in main (tar.exe.obj) return @call(.always_inline, callMainWithArgs, .{ @intCast(usize, c_argc), @ptrCast([*][*:0]u8, c_argv), envp }); ^ Unable to dump stack trace: InvalidDebugInfo

#

@dry wolf

#

the extension of file which i am trying to decompress is tgz

dry wolf Apr 9, 2023, 6:17 AM

#

aight change

try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{});

to

try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{ .mode = .ignore });

signal sluice Apr 9, 2023, 6:18 AM

#

now this

#

tar.zig:15:75: error: no field named 'mode' in struct 'tar.Options' try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{ .mode = .ignore }); ^~~~ C:\zig\lib\std\tar.zig:1:21: note: struct declared here pub const Options = struct { ^~~~~~ referenced by: comptime_0: C:\zig\lib\std\start.zig:61:50 remaining reference traces hidden; use '-freference-trace' to see all reference traces

dry wolf Apr 9, 2023, 6:18 AM

#

oh, it's .mode_mode = .ignore

#

weird option name but

signal sluice Apr 9, 2023, 6:20 AM

#

soory for inconvenience but now i am getting this error

#

error: InvalidCharacter C:\zig\lib\std\fmt.zig:1927:17: 0x7ff7cd96dfd1 in charToDigit (tar.exe.obj) else => return error.InvalidCharacter, ^ C:\zig\lib\std\fmt.zig:1855:23: 0x7ff7cd934614 in parseWithSign__anon_4751 (tar.exe.obj) const digit = try charToDigit(c, buf_radix); ^ C:\zig\lib\std\fmt.zig:1754:5: 0x7ff7cd926e17 in parseInt__anon_3977 (tar.exe.obj) return parseWithSign(T, buf, radix, .Pos); ^ C:\zig\lib\std\tar.zig:40:9: 0x7ff7cd9266d6 in fileSize (tar.exe.obj) return std.fmt.parseInt(u64, rtrimmed, 8); ^ C:\zig\lib\std\tar.zig:117:27: 0x7ff7cd9281bf in pipeToFileSystem__anon_3818 (tar.exe.obj) const file_size = try header.fileSize(); ^ D:\zig\unzipping\tar.zig:15:5: 0x7ff7cd9298aa in main (tar.exe.obj) try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{ .mode_mode = .ignore });

dry wolf Apr 9, 2023, 6:21 AM

#

hmm, im not entirely sure about that, let me do some local testing and get back to you

signal sluice Apr 9, 2023, 6:22 AM

#

okay fine

signal sluice Apr 9, 2023, 10:53 AM

#

Hey @dry wolf do you find any work around?

vast urchin Apr 9, 2023, 1:05 PM

#

signal sluice `error: InvalidCharacter C:\zig\lib\std\fmt.zig:1927:17: 0x7ff7cd96dfd1 in charT...

are you sure the tar file is good?

signal sluice Apr 9, 2023, 1:29 PM

#

Ya

#

I can extract it using winzip easily

vast urchin Apr 9, 2023, 1:30 PM

#

do you mind sending it?

signal sluice Apr 9, 2023, 1:31 PM

#

Wait

#

📎 yargs.tgz

#

@vast urchin

vast urchin Apr 9, 2023, 1:37 PM

#

ah lovely, yeah, this is a bug in std.tar

signal sluice Apr 9, 2023, 1:38 PM

#

oh

vast urchin Apr 9, 2023, 1:38 PM

#

you should file an issue

signal sluice Apr 9, 2023, 1:38 PM

#

okay

vast urchin Apr 9, 2023, 1:38 PM

#

it trims \x00 from the end of numbers but not spaces

signal sluice Apr 9, 2023, 1:40 PM

#

just wondering why is it so

#

anyway i will file an issue

#

@vast urchin we have this error because of std.tar.pipeToFileSystem or std.compress.gzip.decompress

vast urchin Apr 9, 2023, 1:44 PM

#

it's a std.tar issue

signal sluice Apr 9, 2023, 1:44 PM

#

Okay thanks

#

Should i mark this question as completed

#

As i have filed the issue

#

@vast urchin

vast urchin Apr 9, 2023, 1:54 PM

#

up to you if you want to wait for that issue to be resolved

signal sluice Apr 9, 2023, 1:55 PM

#

Okay then I'll wait

signal sluice Apr 9, 2023, 2:31 PM

#

i found the definition of fileSize method

#

pub fn fileSize(header: Header) !u64 {
        const raw = header.bytes[124..][0..12];
        const ltrimmed = std.mem.trimLeft(u8, raw, "0");
        const rtrimmed = std.mem.trimRight(u8, ltrimmed, "\x00"); <----
        if (rtrimmed.len == 0) return 0;
        return std.fmt.parseInt(u64, rtrimmed, 8);
    }

#

do we have problem in the 4th line?

frosty herald Apr 9, 2023, 2:35 PM

#

if random internet person is correct, it'd be a matter of adding (an empty space) before the \x00

signal sluice Apr 9, 2023, 2:36 PM

#

do you have any idea how we can add it

#

can i use mem.join method?

frosty herald Apr 9, 2023, 2:38 PM

#

edit the file, place cursor between " and \, press space bar, save and exit; recompile app run see if errors

signal sluice Apr 9, 2023, 2:40 PM

#

no adding space starts showing those wiggling lines under it

vast urchin Apr 9, 2023, 2:41 PM

#

it needs to be " \x00"

signal sluice Apr 9, 2023, 2:42 PM

#

now we have another error

#

error: FileNotFound
C:\zig\lib\std\os\windows.zig:131:39: 0x7ff67794abef in OpenFile (tar.exe.obj)
            .OBJECT_PATH_NOT_FOUND => return error.FileNotFound,
                                      ^
C:\zig\lib\std\fs.zig:1385:23: 0x7ff67791579e in createFileW (tar.exe.obj)
            .handle = try os.windows.OpenFile(sub_path_w, .{
                      ^
C:\zig\lib\std\fs.zig:1268:13: 0x7ff677909496 in createFile (tar.exe.obj)
            return self.createFileW(path_w.span(), flags);
            ^
C:\zig\lib\std\tar.zig:133:28: 0x7ff6779085cd in pipeToFileSystem__anon_3818 (tar.exe.obj)
                var file = try dir.createFile(file_name, .{});
                           ^
D:\zig\unzipping\tar.zig:14:5: 0x7ff67790989a in main (tar.exe.obj)
    try std.tar.pipeToFileSystem(std.fs.cwd(), decompresser.reader(), .{ .mode_mode = .ignore });

signal sluice Apr 9, 2023, 3:30 PM

#

@vast urchin @frosty herald any idea?

vale matrix Apr 9, 2023, 10:00 PM

#

seems like the path isn't being created. could use makePath() for that.
i was able to get the tar file from above decompressed with these changes:

--- ~/Code/zig/zig/lib/std/tar.zig 2023-04-09 14:57:50.227554929 -0700
+++ ~/Code/zig/zig/download/0.11.0-dev.2213+515e1c93e/files/lib/std/tar.zig  2023-04-09 14:57:06.687171566 -0700
@@ -35,7 +35,7 @@
     pub fn fileSize(header: Header) !u64 {
         const raw = header.bytes[124..][0..12];
         const ltrimmed = std.mem.trimLeft(u8, raw, "0");
-        const rtrimmed = std.mem.trimRight(u8, ltrimmed, "\x00");
+        const rtrimmed = std.mem.trimRight(u8, ltrimmed, " \x00");
         if (rtrimmed.len == 0) return 0;
         return std.fmt.parseInt(u64, rtrimmed, 8);
     }
@@ -128,8 +128,10 @@
             .normal => {
                 if (file_size == 0 and unstripped_file_name.len == 0) return;
                 const file_name = try stripComponents(unstripped_file_name, options.strip_components);
-
+                if (std.fs.path.dirname(file_name)) |dirname|
+                    try dir.makePath(dirname);
                 var file = try dir.createFile(file_name, .{});
+
                 defer file.close();
 
                 var file_off: usize = 0;

#

not sure this is correct, but it does allow it to succeed.

#

note: i also tried replacing line 132 with try dir.makeDir(dirname); (as is done a few lines above) but it fails if the directory already exists. so i think makePath() is the better alternative.

vast urchin Apr 9, 2023, 10:20 PM

#

this is probably the solution, makeDir is also problematic because the parent directory isn't guaranteed to exist or be defined above a child

vale matrix Apr 9, 2023, 10:21 PM

#

is there already an open issue?

vale matrix Apr 9, 2023, 10:23 PM

#

vast urchin this is probably the solution, `makeDir` is also problematic because the parent ...

do you think that the makeDir() is potentially problematic a few lines above too?

#

this one i mean: https://github.com/ziglang/zig/blob/master/lib/std/tar.zig#L125

vast urchin Apr 9, 2023, 10:38 PM

#

yes

vale matrix Apr 9, 2023, 10:55 PM

#

looks like this is the issue https://github.com/ziglang/zig/issues/15222. i'll create a pr for this.

vale matrix Apr 9, 2023, 11:12 PM

#

created a pr here https://github.com/ziglang/zig/pull/15228

signal sluice Apr 10, 2023, 1:56 AM

#

thanks @vale matrix @vast urchin @frosty herald @dry wolf for discussion and help

#

its working!!

timid kelp Apr 21, 2023, 6:39 AM

#

signal sluice its working!!

The PR or your code?

vale matrix Apr 21, 2023, 6:55 AM

#

btw, i created a much more comprehensive pr for std.tar https://github.com/ziglang/zig/pull/15382

signal sluice Apr 21, 2023, 3:27 PM

#

@timid kelp my code

#tgz decompress