#how to deal with arraylist of allocated items

1 messages · Page 1 of 1 (latest)

lunar cedar
#

I have a function that returns an ArrayList of a Token tagged union. Token will always contain data that has been allocated. So, is it possible to ensure that the user doesn't have any problems with freeing the memory? Will I have to create a deinit?

wraith compass
#

The simplest path is to simply avoid the need to.
Say, by having the allocated data not controlled by the tokens -- instead the parser does, for example.

#

So parser.deinit() would free all that, for example.

upper cosmos
#

Ensure, no. Even if you provide a deinit, you can't make the user call it.

lunar cedar
wraith compass
#

Well - how you do it depends entirely on how it's allocated, of course.
But honestly, I think that unless you have a very specific reason, I wouldn't care about it

#

In fact, I wouldn't have tokens control any memory at all

#

I'd just keep the input string alive, and have the tokens point into that

#

I assume that's the data the tokens have that is allocated?

lunar cedar
wraith compass
#

The essence of memory management is that you should know the lifetimes of the data you have, and you use that to your advantage.
In this case, you haven't told me what the data that's allocated is, but, if that data is str: []const u8 field that says what string this token comes from, then you don't need that string to be separately allocated.
You could basically just do .str = input_string[offset..][0..token_length], which means that str field points into the input_string, and thus has the same lifetime as that input string.

lunar cedar
#

oh i completely understand now

#

i like having to think about memory. in rust it was way to easy to just slap a Vec::new on everything and call it a day.

wraith compass
#

It's important to understand that managing memory manually doesn't mean that you inherit the problem that a GC solves --- it's that you get to do whatever you want in order to manage it, because you know what lifetimes you need.

#

You may, for instance, never free anything at all; the OS will always reclaim the memory anyway.
In fact, trying to free everything right before you quit is a bit of a bad form really -- you're just wasting your user's time.

wraith compass
#

because you know what lifetimes you need.
On this part though, part of "knowing what you need" also means that you understand that "well, technically I need X to live for A, Y to live for B, and Z to live for C, but I can just make them all live for A because that works and is much simpler."
An arena allocator is built to do exactly this; make it very cheap to allocate a bunch of stuff and then free it all at once, quickly -- having to track literally every single thing and free it is so 2012 🤣

lunar cedar
#

*In release mode?

wraith compass
#

I'm saying that freeing everything just before you know your program is about to quit, is a sign that you don't know what you're doing more than it is necessary 😄

lunar cedar
#

also, unrelated question but kinda related to what im doing, and you seem decently knowledgable on memory management: is reference counting a decent way to handle memory for a language that will have similar uses to something like javascript, server side web and otherwise connected applications?

#

or at least better than a gc would be?

wraith compass
#

My impression is that refcounting is generally a fairly slow approach which also cannot deal with cyclic datastructures, like linked lists.

#

But I've not personally worked with it that much, and I wouldn't want to say that it shouldn't ever be used.

#

I would however suggest that one purposefully doesn't make things more complicated than absolutely necessary.

lunar cedar
#

cyclic data structures would preferably be avoided in this language. I don't think they're necessarily something i want to support

upper cosmos
#

Server side web and similar transaction-oriented things would often use an arena for everything to process that request then deinit that when done.

wraith compass
#

I would generally suggest using an arena instead of refcounting. Though, I would actually suggest using an arena for basically everything that you can.

# 
var req_arena = make_arena();
defer req_arena.destroy();
const arena = req_arena.allocator();

// allocate to your heart's content, because everything will be freed at the end of the scope with no need to
// track everything at any point
lunar cedar
#

what if a block is particularly chunky? i feel like an arena might introduce some issues with oom

wraith compass
#

There's not really a relation between those things

#

If there's ever an OOM, you just give up on the request and reply with a 500 or whatever

#

For example

#

You can always have a secondary allocator for things that resize a lot or whatever -- the point of custom allocator types is that you can pick exactly what you need for what you're doing, as you see fit. Or, build your own custom amalgamation; an allocator that incorporates both arena-like an GPA-like behaviour, or whatever.

#

An arena for most shit, and a more general-purpose one for growing stuff, as a random example.

#

But you may well not even need that

lunar cedar