#Actor / memory safe framework

1 messages · Page 1 of 1 (latest)

primal viper
#

would it not be possible to create a set of memory safe primitives using zig?

like an actor framework, where all types that cross boundaries are just handles/copyable, and based on messaging?

then zig would be entirely memory safe, and without defer, as long as that system is used only

basically like Swift's actors

does something like this exist? does anyone know of any discussions about this?

cuz i saw a couple actor libraries but they still work with []u8 and such, which is unsafe / relies on defer

jolly agate
#

sure, you could build a safe framework where everything is managed by some runtime. though nothing about []u8 is inherently unsafe, it could be provided to the user in a way that doesn't require them to manage memory, and don't have a way to sneakily store a reference to it later - though only within reason, since you can still just go out of your way to violate requirements of the API, using things like inline assembly, globals, extern functions, and whatever else

fluid ivy
#

Hmmm, I kinda do actor patterns

primal viper
#

it'd be more about the idea of auto deallocating stuff after "turns", so no defer is needed, and making retrieval return optionals/nullables/whatever zig calls these

jolly agate
#

it's not really possible in just bare zig to make it impossible to do "unsafe" things, since unsafe operations are just woven into its fabric

jolly agate
primal viper
#

right but say i have a magical linter that prevents use of anything but my little actor frameworks types

#

unless i wrap them in // unsafe start and // unsafe end

#

just hypothetically

jolly agate
#

sure, you could force usage of only a subset of zig code

fluid ivy
#

So

#

see, safety is a property you win with proving states are all correct

jolly agate
#

essentially a way to re-invent rust's unsafe on top of zig through additional tooling

primal viper
#

right

#

that's exactly what i want

#

IDGAF about performance, just safety, i wanna use zig for comptime/its ingenious type system but dread defer and discipline (im not disciplined)

fluid ivy
#

So you cannot win without making sure your own things, I wouldn't try to impose one language into another but get your safety needs

jolly agate
#

whether or not what you're saying leads to reasonably "safe" code, while still being useful, is questionable

primal viper
fluid ivy
wild atlas
jolly agate
#

it's tricky to design a safe subset of code without making it overly restrictive, or without missing edge cases by being too permissive

fluid ivy
#

Because what are you trying to be safe against:

  • Memory errors?
  • Safe inter-thread interactions?
  • Logic bugs?
  • Side effect safety?
primal viper
jolly agate
#

also, want to note: there's technically nothing "unsafe" about leaking memory (which is what would happen by forgetting a defer), it just means you leak resources, which won't be a problem until you do that too much. in fact, in rust, leaking memory is something you can accomplish entirely in safe code

fluid ivy
#

Leaking memory is just a runtime issue, not a safety one

primal viper
#

right, im not talking only about UB or UAF

jolly agate
#

if you consider leaking memory to be unsafe (which is also valid to do), you will need to do some even more complicated stuff than even rust does

wild atlas
fluid ivy
#

unless you want to be security hardned, then secrets and memory leaks can leak information

primal viper
jolly agate
#

as you can see, "safety" is not a binary switch that you can turn on or off with one fix, it is a very large problem space with a huge number of considerations, angles, solutions, caveats, and tradeoffs

primal viper
jolly agate
primal viper
#

not in practice

#

in theory yes

jolly agate
#

no, they are the same thing in practice

#

"freeing" memory often is just "removing an item from a list"

wild atlas
# primal viper data races, UB safety. not deadlocking.

if u can force zig code to go through certain paths (maybe with an external sanitizer) then you can get the first two. Cant have no deadlocking with actors, unless you go the "behavior" approach from ponylang or similar. Idk how youll ensure UB safety exactly (without pulling like an ASAN)

jolly agate
#

to prevent the former, you need to basically know how to prevent the latter

#

and as you can imagine that is complex

#

if were easy to do, someone would have already done it

#

it's the fact that there are so many tradeoffs and caveats that it's such a contentious design space

#

you can achieve some limited form of what you want by constraining zig, but it will be nowhere near "perfect" by any reasonable definition

fluid ivy
#

There is no perfection over safety

jolly agate
#

indeed. there's just "attack vectors that you care about", and preventions for those vectors

fluid ivy
#

My Windowing Library is basically actor model based lol

jolly agate
#

zased

fluid ivy
#

It is safe over threading but still leaks and I still gotta care about race condition (because zero allocation post init sucks sometimes)

#

Let me see if I can extract the gist of it

primal viper
#

hm, look, my main issue is that rust eliminates this whole class of bugs entirely w/ its way of doing RAII and its Send/Sync markers, but then async falls back to "something like actors but not really" with channels, or sprinkling Arc<Mutex<T>> everywhere, or some variation thereof, where everything ends up being refcounted. it's worse than using a GC.

i wonder if there's a solution in zig that could be a bunch of primitives that are designed to work in a channel/message passing system only, and because we constrain lifetimes to knowns like "foo isonly valid for this message", i could also sidestep defer .deinit()'ing all of those types all the time

it seems very obvious to me, which is why i must be missing something, because i haven't found any discussions about this, or any attempts to build such a framework on gh/codeberg/ziggit

jolly agate
#

zig isn't geared towards solving those problems in the same way

fluid ivy
#

There is no automatic way

primal viper
#

i get that im kinda mangling two problems here, one being thread safety and the other resource cleanup, but i believe they could both be fixed by the same system

fluid ivy
#

there is no free lunch

#

your design needs to take account and move around for it

primal viper
#

no of course - there'd be a performance penalty

#

its basically elixir in zig or something

jolly agate
#

we usually solve these problems by instead decreasing the amount we allocate altogether, often through DOD, and establishing upper bounds on sizes that are appropriate for our applications

primal viper
jolly agate
#

these are just kind of the ways that exist

#

if you want static memory safety, zig is just not the language

#

we deal with hardware and low level systems where we want direct control of those details

fluid ivy
#

The major way to do it without automatic is generating safe data structures you proof are safe over their usage

jolly agate
#

again, you can design a framework to abstract some of these details away

fluid ivy
#

and squat as many usability and half states as possible

jolly agate
#

but unless you're going to undertake the task of running a prover on zig code all NASA-style, there are definite limits in the approach you have described

primal viper
primal viper
#

or am i misunderstanding

primal viper
#

thank you.

jolly agate
#

we have APIs like std.Build, where memory usage & safety is a non-concern due to the context in which it is used, which is a short execution time, compiled on-demand

fluid ivy
#

I am asking you to kind of do proving of those data over TLA+ or COQ

#

in a way that half states don't exists lol

jolly agate
#

but it still will never be rust levels of static safety (without first constraining and/or extending the language as a whole)

fluid ivy
#

nope

#

It's not static even

#

Casual Verification used to test big distributed system can help you if you want to test your code follows the proven behaviour, but that only asserts that at certain stages it's doing what it should

primal viper
fluid ivy
#

TLA+ and coq are languges to generate formal specifications

#

TLA+ is for concurrent systems, coq is more for formal math, which is a superset of data structures if you will (Also there is lean)

jolly agate
#

lean is pretty fun

fluid ivy
#

completed the whole number game

jolly agate
#

yee same

#

trying to use it every now and then to "prove" certain things for myself lately

#

not successfully, but trying :p

fluid ivy
#

yes

#

Generally my views on software design @primal viper is that you should design on formal verification or some kind of scratch place where you can test your ideas, from there you get your invariants and expectations

primal viper
# jolly agate but it still will never be rust levels of static safety (without first constrain...

with what i'm proposing, it'd all be at runtime, so objectively worse performance. wrong states are then representable at the type level, not trying to prevent that. i dont think there's any (good) way to fix that without introducing lifetimes to the type system like rust does it

none of them could lead to UAF, leaks, and so on, though, they'd at worst be null. because you're using only the frameworks primitives, which all neatly stash this stuff away

in essence using zig to build a "higher level" framework. no free lunch, yes, it's all runtime based. but so far it doesn't sound like it's impossible. just inflexible, probably as soon as you have to step outside the confines of the framework

fluid ivy
#

At runtime you can definitevely reduce representation of invalid states

#

at comptime too

jolly agate
#

yeah sure, wouldn't say that's impossible

#

if you feel intent on it, I'd say go for it. you'll definitely learn a thing or two, one way or the other

fluid ivy
#

For UAF you can track memory, generational handles are big on this

#

Instead of giving memory away, you give a ticket for that memory, that ticket will hand the memory as long is valid

primal viper
#

they still use zig slices or stdlib ArrayList etc

jolly agate
#

I certainly don't think it exists to your specifications :p

fluid ivy
#

this will not eliminate all UAF issues, but as long as you treat it as a RC kind of thing, you get safe over memory in a good subset

jolly agate
#

if you're avoiding usage of things like std.mem.Allocator, they are no more "unsafe" than an integer

#

though to be clear, to design something of this magnitude, you yourself will definitely need to get into the weeds with writing "unsafe" code :p

primal viper
fluid ivy
#

You cannot enforce it more than that

jolly agate
#

like, strings are "slices"

primal viper
#

:P

fluid ivy
#

after you pass the data to the user they can do invalid things over memory

jolly agate
#

specifically they are of type *const [n:0]u8

primal viper
#

yes, but working with them invites you blowing up your program

jolly agate
#

but they are no more "unsafe" than integer literals

#

how?

#

working with integers also "invitkes you blowing up your program", due to the mere existence of integer overflow and casting

primal viper
#

as soon as the memory behind the pointer gets freed you could be reading garbage

primal viper
fluid ivy
jolly agate
#

you cannot free a string literal

primal viper
jolly agate
#

and since we have established your framework would not include dynamic manual allocation, you couldn't even accidentally free slices

fluid ivy
#

Thus why I mention it as a contention
Is contradictory

primal viper
#

oh, maybe i misunderstood what you meant

jolly agate
#

there is nothing more fundamental than integers

#

if you hate integers, you hate programming

primal viper
#

not integers themselves

fluid ivy
#

@jolly agate Technically High and Low voltage

primal viper
#

but the fact that most programming languages let you add to integers unchecked

jolly agate
jolly agate
fluid ivy
#

You can define all set of data actions on your own api, but cannot be generic over the user design without boilerplate

primal viper
fluid ivy
#

boilerplate induces the user to not interact with the system itself

primal viper
#

for instance i think rust should disallow foo + 1 without wrapping in unsafe{}

jolly agate
jolly agate
#

I think you have a fundamentally skew understanding of what "safety" is

primal viper
#

yes but there's a difference between "checked by zig and it panics" versus "im trying to catch that case and have my code handle it gracefully" no?

fluid ivy
#

You can have at runtime tagged information on data, and you can have some thread trying to detect all illegal behaviour, but this requires desinging a runtime and some way of dealing over with it...
Suddently you are writting a programming language

primal viper
#

figured that's essentially what im doing :P

jolly agate
#

under your pretenses, the safest thing for you to do would be to simply never turn on your computer. you have to accept some level of real world volatility to get anything done

primal viper
#

right, i care about practicality here

#

i'm fine with panicking on running out of memory

jolly agate
#

then you will likely need to make some practical concessions about what you do and don't accept

fluid ivy
primal viper
#

i'm not fine with panicking on common language constructs

jolly agate
fluid ivy
#

See Vale

#

(Vale is a really interesting language, love the concepts)

jolly agate
#

allocation failure can be handled, sometimes integer overflow simply cannot be, often because the assumptions have already been programmed into the code preceding it

primal viper
jolly agate
#

if you choose a bigger integer, it wouldn't overflow, so those two sentences seem fairly contradictory :p

primal viper
#

do you not think its weird that rust is super careful around pointer arithmetic, which is just arithmetic, but then it doesn't care a whole lot about integer arithmetic?

jolly agate
#

fwiw, if ranged ints are accepted and implemented, the default will be non-panicking, with panics only introduced with explicit casts

primal viper
#

right!!! that's fine

#

that's perfect

#

gotta bounce for a lil while but thank you so much for the discussion guys y'all are awesome

jolly agate
primal viper
#

feel free to keep chatting i'll catch up later

jolly agate
#

software engineering is all about making the right trade offs

jolly agate
fluid ivy
#

(I could use IO, but didn't want to imply is attached to any design)

wild atlas
# fluid ivy Here is an example of an actor model created by yours truly, bugs ahead: https:/...

the SyncResponse could store .value without atomic. it could also avoid doing a wake if theres no waiter with something like

fulfill(v):
  value = v
  if ready.swap(2, .release) == 1: futexWake(&ready)
wait():
  if ready.swap(1, .acquire) != 2:
    while ready.load(.acquire) == 1: futexWait(&ready, 1)
  return value

in receive() the store(PARKED, release) needs to happen-before the tryReceive()'s head.load(acquire). Can do that by upgrading both to seq_cst. SImilar situation between trySend()'s head.store(release) and wakeReceiver's state.load(acquire). https://ziglang.org/download/0.14.0/release-notes.html#StoreLoad-Barriers

fluid ivy
#

Yea, make sense

#

You mean this?:

pub fn SyncResponse(comptime T: type) type {
    return struct {
        value: T,
        
        ready: std.atomic.Value(u32) = .init(0),

        pub fn init(default: T) @This() {
            return .{ .value = default };
        }

        pub fn fulfill(self: *@This(), io: std.Io, value: T) void {
            self.value = value;
    
            if (self.ready.swap(2, .release) == 1) {
                io.futexWake(u32, &self.ready.raw, 1);
            }
        }

        pub fn wait(self: *@This(), io: std.Io) T {

            if (self.ready.swap(1, .acquire) != 2) {
                while (self.ready.load(.acquire) == 1) {
                    io.futexWaitUncancelable(u32, &self.ready.raw, 1);
                }
            }
            return self.value;
        }
    };
}