#Http url Download file.

1 messages · Page 1 of 1 (latest)

warm mortar
#

Hi all this code for downloading from url.

const ZigetError = error{
    CreateSockFail,
    InvalidAddr,
    ConnectError,
    SendError,
    RecvError,
};
pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();
    var args_it = try std.process.argsWithAllocator(allocator);
    defer args_it.deinit();
    _ = args_it.skip();
    const host = (args_it.next() orelse {
        std.debug.print("no host provided\n", .{});
        return error.InvalidArgs;
    });
    const remote_path = (args_it.next() orelse {
        std.debug.print("no remote path provided\n", .{});
        return error.InvalidArgs;
    });
    const output_path = (args_it.next() orelse {
        std.debug.print("no path provided\n", .{});
        return error.InvalidArgs;
    });
    std.debug.print("host: {s} remote: {s} output path: {s}\n", .{ host, remote_path, output_path });
    var conn = try std.net.tcpConnectToHost(allocator, host, 80);
    defer conn.close();
    var buffer: [256]u8 = undefined;
    const base_http = "GET {s} HTTP/1.1\r\nHost: {s}\r\nConnection: close\r\n\r\n";
    const msg = try std.fmt.bufPrint(&buffer, base_http, .{ remote_path, host });
    _ = try conn.write(msg);
    var buf: [1024]u8 = undefined;
    var total_bytes: usize = 0;
    var file = try std.fs.cwd().createFile(output_path, .{});
    defer file.close();
    while (true) {
        const byte_count = try conn.reader().read(&buf);
        if (byte_count == 0) break;
        _ = try file.write(&buf);
        total_bytes += byte_count;
    std.debug.print("written {d} bytes to file '{s}'\n", .{ total_bytes, output_path });
}

but when i run it using this command.

the output file looks like a response header not the actual file.
Whats going on here? I'd like to know.

molten bramble
#

not the actual file
To my knowledge, that's not how HTTP 1.1 works.
I'd expect the result to look something more like

200 OK
Content-Length: <SIZE>
Content-Type: text/plain
More: Headers here

Content of file here (also notice the two \r\n's)
#

You don't post the output you get, so I can't really help you much beyond that.

warm mortar
# molten bramble > not the actual file To my knowledge, that's not how HTTP 1.1 works. I'd expect...

Thank you for caring to help .Sorry to not having posted it. here is the output file

Connection: close
Content-Length: 0
Server: Varnish
Retry-After: 0
Location: https://raw.githubusercontent.com/lun-4/ziget/refs/heads/master/src/main.zig
Accept-Ranges: bytes
Date: Sun, 13 Oct 2024 11:28:00 GMT
Via: 1.1 varnish
X-Served-By: cache-tyo11954-TYO
X-Cache: HIT
X-Cache-Hits: 0
X-Timer: S1728818880.202469,VS0,VE0
Access-Control-Allow-Origin: *
Cross-Origin-Resource-Policy: cross-origin
Expires: Sun, 13 Oct 2024 11:33:00 GMT
Vary: Authorization,Accept-Encoding```
And also if I remove one \r\n the program just hangs without doing anything.
And if you can, will you suggest the proper way to download a file from url like wget in zig using just the std lib? Thank you.
molten bramble
#

It's an HTTP equivalent of a symlink, basically

molten bramble
#

Although... isn't that URL the same as the one you used originally? 🤔

warm mortar
#

You mean like this right.

```                     (host)                                                  (location)
molten bramble
#

So I'd expect it to give you that alternative location

#

I figured that was the Location header value, but... I think that's the same URL isn't it?

warm mortar
#

yeah when i give that same link to wget it works just fine

molten bramble
molten bramble
#

The point of a symlink is to Just Work, after all

warm mortar
#

what is that second http request can you please specify. I am a beginner

molten bramble
#

You repeat the original request, only this time to the URL it gave you

#

"That URL is outdated. For that resource, please use <this> one instead, going forward."

warm mortar
#

Yeah but its the same url right? what to do about that? Sorry if I sound dumb.

molten bramble
#

A good question. I'm not sure off hand.
I would say "just trying it again", but that doesn't seem right. 🤔