"Rewinding" to previous argument with the new Args.Iterator.Posix

[robust obsidian]

With the old arg iterator I enjoyed using the index field to "rewind" the iterator to a previous argument while parsing options for CLIs. I can't see how to achieve this with the new implementation.

1. Any suggestions on how to replicate this, while keeping the iterator pattern?
2. Is there a "hard" reason for this change? Could it be reversed if my use-case proves worthwhile?

An example:

⁨⁨```ts
var args = std.process.ArgIteratorPosix.init();
const progname = args.next();

// Parse options
while (args.next()) |arg| {
// THIS: If arg isn't an option, rewind to process it again
if (arg[0] != '-') {
args.index -= 1;
break;
}
}

// Process file arguments
while (args.next()) |path| {
...
}

[tepid salmon]

you can just save the whole iterator state, instate instead of relying mutating the internal data, this is also more potable.

an alternative is collecting the args into a slice, which there is a helper function for toSlice (on Args not the iterator).

[robust obsidian]

I checked ⁨toSlice⁩ and it seems to require an allocator, which I would like to avoid (I assume because it also deals with Windows stuff).

Right now I'm curious as for the reason for this change, if it maybe helps avoiding footguns or something.

[tepid salmon]

yes its about footguns, (regarding posix specifically) previously there was a global variable and the iterator had an index into it, now the iterator has the data directly and the global has been removed. Same story for environment variables. but since the logic for obtaining that is contained within init parameter of juicy main, and the globals are removed, they no longer have to cater to libc.

std posix/os apis mimicked libc alot, as that allows std to trivially switch to libc when its linked since it implements all the same stuff no reason to include it twice in the running program.

other platforms may not have used globals, or they did, idk, regardless std was likely still copying libc for the aforementioned compatability.

[robust obsidian]

But would the args iterator need to be changed in order to achieve this? It seems like it would just be taking the ⁨⁨Args⁩⁩ from the init rather than as a global.

I didn't dislike the idea of using toSlice, but it could have a variant that don't require an allocator, similar to the current ⁨⁨iterate⁩⁩ and ⁨⁨iterateAllocator⁩⁩. I guess I would still prefer the old iterator though, unless the new one has some advantage that I'm not noticing.

[tepid salmon]

the internal state does need to change, as it needs to be able to access the actual args which are no longer in a global variable.
the api hasn't changed, only the internal state.

you can just save the whole iterator state, instate instead of relying mutating the internal data, this is also more potable.
this method works for both new and old implementations, and across platforms.

an alternative to that, which might be preferable to reduce the amount of memory being copied (windows arg iterator is quite large), is to store the arg instead of the previous iterator state.

both are very similar to the example you gave, in case it isnt clear how saving the arg doesnt add a bunch of complexity:

var next: ?[]const u8 = args.next();
// options
while (next) : (next = args.next()) {
   if (not an option) break;
}
// file args
while (next) : (next = args.next()) {
}

[robust obsidian]

the api hasn't changed, only the internal state.
I guess you could say it changed in the sense that it doesn't expose the index field anymore.

the internal state does need to change
Yes. What I don't understand is why it discarts the previous args, rather than keeping them and using the index to manage the position.

in case it isnt clear how saving the arg doesnt add a bunch of complexity
I was having trouble figuring this out. Saving just the arg together with this pattern is pretty neat, will probably use it if the new iterator is here to stay.

[tepid salmon]

Yes. What I don't understand is why it discarts the previous args, rather than keeping them and using the index to manage the position.
The only benefit I can think of is its smaller this way, neither way is more or less complex, they are just different.

While it may make sense for posix specifically, not all platforms do args in a way that allows you to go to the previous, at least not without much effort. The api is designed with crossplatform in mind, so they likely just didnt think to make a specific platforms implementation more flexible when other platforms are limiting the api.

Windows for example seems quite complex, and std also converts it to utf-8 on top of that. even if it could go back (I dont think it can without a full state save that I suggested) it would be expensive to recompute and allocate the arg when the user could instead just store it in a variable like I suggested.

the api hasn't changed, only the internal state.
I guess you could say it changed in the sense that it doesn't expose the index field anymore.
while public state can and should be considered part of the api, zig has the awkward position of not having private state.
I think platform specific implementation details in a cross platform api are quite clearly "internal" and a bad idea to depend on.
So i dont think you can blame them for assuming changing it wouldnt affect most people.

[robust obsidian]

The thing is that this one doesn't seem to be cross platform. ⁨⁨Args.Iterator.init⁩⁩ is for posix specific and ⁨⁨Args.Iterator.initAllocator⁩⁩ for Windows and such.

The funny thing is that the Windows implementation aparently have an index field.

[robust obsidian]

As you said the only benefit of the current one seems to be the smaller size at the cost of expressivity.

I was checking some other iterators in the std lib and most of them use the index implementation, one even has a ⁨previous⁩ method. So I imagine they are not against an iterator providing this.

What do you think, am I overthinking this or is it actually valid? I tried the alternative you suggested but it just didn't feel as right. Also sorry if I'm bothering you.

[tepid salmon]

it is a cross platform api as it abstracts over platform specific implementations.

I dont think offering functionality that is not available on all platforms negates its cross platform purpose. That being said, idk why a non allocating init is supported, i would guess its just a remanent of old code.

I think something like Args should offer the same functionality on all platforms, so if some platforms dont offer going backwards (even just for a single arg) or if its expensive to do so, then i dont think it should be supported.

especially since its trivial for the user to get, and do so efficiently, that functionality themselves.

btw I cant find a previous function on any of the implementations

[robust obsidian]

What if I use the specific one instead, with ⁨⁨Args.Iterator.Posix.init⁩⁩?

That being said, idk why a non allocating init is supported, i would guess its just a remanent of old code.
I think posix just doesn't need to alocate somehow.

The previous function is from ⁨⁨fs.path.ComponentIterator⁩⁩

[tepid salmon]

What if I use the specific one instead, with ⁨Args.Iterator.Posix.init⁩?
Im not sure why you would do that, since args are no longer in a global you have to get them through juicy main, at that point you have the cross platform Args, im not sure why you would bypass that to create and use the implementation directly.
I think posix just doesn't need to alocate somehow.
it does not, but Args needs to cater to platforms that do and non allocating init is the only thing it has that isnt available on all platforms. initAllocator doesnt allocate for platforms that dont need to, its just takes an allocator for the platforms that do need to.

The previous function is from ⁨⁨fs.path.ComponentIterator
I am beyond confused why you are bringing that up, or even using it having a previous function as a reason why Args.Iterator should????

[robust obsidian]

im not sure why you would bypass that to create and use the implementation directly.
What do you mean by bypass? It seems like I can access the specific implementation using the ⁨Args⁩ from juicy main.

I was thinking that if I only cared for posix I could get a more "featureful" implementation. I don't see how this could break the cross platform API.

I am beyond confused why you are bringing that up, or even using it having a previous function as a reason why Args.Iterator should????
I was pointing that at least one iterator had it, so maybe it's not unfeasible for others. Although I didn't check how much this one differs from others, so maybe it is an exception.

[tepid salmon]

I was thinking that if I only cared for posix I could get a more "featureful" implementation. I don't see how this could break the cross platform API.
you can do that, and it wont break anything, it just that you dont get anything you dont already have.

I was pointing that at least one iterator had it, so maybe it's not unfeasible for others. Although I didn't check how much this one differs from others, so maybe it is an exception.
the reason I am confused is because they are different things, one iterates over path components, the other over program arguments.

Iterators are a simple and very flexible pattern, defined as being able to step through work getting a value that may signal that it is done. They do not have a standardised api, the closest you'll get are some common function names like next, and perhaps the usage of optionals which are sometimes replaced with error unions or a sentinal value.

its analogous to saying apples should taste like oranges because they are both fruit. yes they are both fruit (iterators) but that is a broad category, they are still distinctly different.

[robust obsidian]

it just that you dont get anything you dont already have.
What do you mean? I would get an iterator with the possibility to check previous values, as it doesn't need to worry if other platforms support it.

one iterates over path components, the other over program arguments.
Is the concrete thing it iterates really that important? I imagine it wouldn't hurt to allow rewinding when use-cases appear for each iterator.

If I'm not mistaken Zig has some sort of conditional compilation, so adding a ⁨⁨previous⁩⁩ wouldn't bring overhead. Of course the devs may not even want to maintain it in the std, but them I would be curious as to why.

[tepid salmon]

What do you mean? I would get an iterator with the possibility to check previous values, as it doesn't need to worry if other platforms support it
the posix implementation doesnt give you that, and you dont need to get it separately to access its state either. since you'd have to save its state, or the arg you dont even need to interact with the posix specific state.

Is the concrete thing it iterates really that important? I imagine it wouldn't hurt to allow rewinding when use-cases appear for each iterator.
not to be rude, but i feel like you havent been listening to me. again, Args is a cross platform abstraction, not all implementations can or should support that so Args cant/shouldnt either, especially for something you can trivially do yourself.
a different unrelated iterator supporting it has no meaning to this conversation.

At this point I have repeated myself multiple times, I dont know how to explain this to you.
And I dont know why you keep bringing up unrelated iterators.

[robust obsidian]

the posix implementation doesnt give you that
Yes, the current one doesn't. Previously I mentioned the old one (from 0.15.2) which used the index implementation, which in a way allowed rewinding. But recently it has changed, and I was wondering why.

If I understood correctly you are saying it isn't possible anymore because ⁨Args⁩ is cross platform, but at the same time it continues to expose a posix especific? I'm not sure if that's it but thanks for trying.

I understand if you don't want to continue. Sorry if I misunderstood you.

[robust obsidian]

I'm still interested in this, in case anyone wants to continue the discussion.

I don't know if I'm missing something obvious or if what I'm proposing is somehow absurd. I would appreciate if someone could help me clear this up.

[polar osprey]

You could avoid rewinding and have the same logic with a state machine. Or else put the file arg handler in a function and call it in your if statement; instead of rewinding.

[polar osprey]

    var state: ArgsState = .init;
    while (args.next()) |arg| {
        sw: switch (state) {
            .init => {
                procName = arg;
                state = .options;
            },
            .options => {
                if (arg[0] != '-') {
                    state = .files;
                    continue :sw .files;
                }
                // handle options
            },
            .files => {
                //handle files
            }
        }
    }

idk maybe it's a little messy but rewinding shouldn't be necessary

plus with a switch state machine you can switch back into parsing options if they put some after the positional args

[tepid salmon]

you can combine the while and switch by either having the arg in a var or making the state a tagged union with the arg as data for each field that needs it.

I thought about giving that solution, but if they just want to match what they had previously then the switch is quite verbose compared to what I gave earlier.

var next: ?[]const u8 = args.next();
// options
while (next) : (next = args.next()) {
   if (not an option) break;
}
// file args
while (next) : (next = args.next()) {
}

[polar osprey]

Oh nice yeah I missed what was going on there

[steel sun]

"Dynamic Programming" is the classical term for memoizing your states in a single pass, which is suitable for single-pass arg parsing. Might want ot look it up.

[robust obsidian]

plus with a switch state machine you can switch back into parsing options if they put some after the positional args
For my purposes I wanted just the really minimal behavior I posted in the beggining. Thanks for the suggestion though.

Or else put the file arg handler in a function and call it in your if statement
I liked not having to put things in functions in the original, it felt simpler.

The best alternative seems to be Vei's recommendation, but I still like the one I was using before more.

I recognize that I'm being stubborn, I guess I'm just a bit frustrated that it got swaped by another implementation that don't allow it anymore nor seems to be necessary or a big improvement.