`free` crashes

[marsh jackal]

I want to parse a file from argv into a LIST of TOKENs. The only problem is, whenever I try to free the value of the existing token inside of LEXER so I can write a new value to it, the program will abruptly crash. I have shortened the code to around 150 relevant lines (including helpful comments and debug prints and excluding error checking to reduce line count). I am compiling with gcc on windows 10 using gcc -m32 -O2 test.c -pedantic -Wall -o test.exe and running using .\test.exe test.
Expected output (excluding debug stuff):

id      2

id      2

16      3

16      3

69      3

Actual output (excluding debug stuff):

id      2

id      2

16      3

Any tips or hints will be greatly appreciated.

[vivid walrus]

When your question is answered use !solved to mark the question as resolved.

Remember to ask specific questions, provide necessary details, and reduce your question to its simplest form. For tips on how to ask a good question run !howto ask.

[stone turtle]

AddressSanitizer:DEADLYSIGNAL
=================================================================
==1==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000074 (pc 0x7ff06dccb42d bp 0x000000000000 sp 0x7ffdac5ed8c0 T0)
==1==The signal is caused by a READ memory access.
==1==Hint: address points to the zero page.
    #0 0x7ff06dccb42d in fgetc (/lib/x86_64-linux-gnu/libc.so.6+0x8b42d) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
    #1 0x402f73 in main /app/example.c:145
    #2 0x7ff06dc64082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
    #3 0x4011cd in _start (/app/output.s+0x4011cd) (BuildId: 7c94ee8eee41f3f6fc9391b0710d6e6d3d40447c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/lib/x86_64-linux-gnu/libc.so.6+0x8b42d) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee) in fgetc
==1==ABORTING

I'd suggest the additional compile options (for debugging only!):

-g3 -Og -fsanitize=address,undefined

Ah, and the error arises at my end because I can't open the file right now

But if you have memory related issues, then sanitizers are always your way to go

*As a first step

[marsh jackal]

thank you for your help I'm using these option now, but the last compile option -fsanitize=address gives an error:

collect2.exe: error: ld returned 1 exit status```

[stone turtle]

!wiki How to Use Sanitizers

[vivid walrus]

How to Use Sanitizers

Sanitizers are tools which generate additional code in your program that can catch many common programming mistakes, such as:

• accessing arrays out of bounds (caught by ASan)
• signed integer overflows (caught by UBSan)
• data races (caught by ThreadSan)
• accessing indeterminate memory (caught by MemorySan)

General Advice

Not all sanitizers can be combined, but when they can, use e.g.:
-fsanitize=address,undefined to combine them. Always compile with debug info to get line numbers, variable names, etc.

GCC

• -fsanitize=address
• -fsanitize=undefined
• -fsanitize=thread
• -g for debug info

clang

• -fsanitize=address
• -fsanitize=undefined
• -fsanitize=thread
• -fsanitize=memory
• -g and llvm-symbolizer for debug info

MSVC

• -fsanitize=address
• -Zi for debug info

Sample Program

int main(void) {
    int x;
    return x;
}

`-fsanitize=memory -g` Output

SUMMARY: MemorySanitizer: use-of-uninitialized-value /tmp/test.cpp:3:5 in main
Exiting

(3:5 is the line and column of return)

Note: The sanitizer lists for GCC and clang are not exhaustive

[stone turtle]

Ah, it's because of MinGW

Idk, never used that compiler (or C under Windows in general)

[marsh jackal]

I see, I'll look for alternatives in the meantime, thanks

[stone turtle]

I can also quickly boot to Linux and check it out there

[vivid walrus]

@marsh jackal Has your question been resolved? If so, run !solved :)

[stone turtle]

Once as a message, once as an image so the coloring is preserved

So this is the line in question:

list->items[list->count++] = item;

[marsh jackal]

so youre getting a heap-buffer-overflow?

[stone turtle]

yup

This is the output I get up until that point:

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
id    2

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
id    2

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value

[marsh jackal]

is that the entire output?

because this is what I'm getting

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
id      2

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
id      2

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
16      3

[dbg] `tokenWrite` called

and this is how i was able to trace the error back to that call to free

[stone turtle]

That's the entire output, at least with sanitizer enabled (which aborts upon the first memory issue, whereas C isn't so strict and just lets you have undefined behaviour until everything kinda crashes eventually)

Yeah, without the sanitizer I get

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
id    2

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
id    2

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
16    3

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
16    3

[dbg] `tokenWrite` called
[dbg] Freed old token value
[dbg] Wrote new token value
realloc(): invalid next size
Aborted (core dumped)

So actually a bit more than you

But yeah, as I said it's undefined behaviour, so anything can happen, which is why it's completely normal that our codes have a different output (because different operating system and compiler)

[marsh jackal]

I see, i guess i should try working on linux from now on until i can figure out where the UB is coming from

[stone turtle]

It's also UB on Linux. The OS isn't the issue, the code is

Just because mine shows more in this case doesn't mean it's generally better

(Although arguably Linux is much better for C)

Oh, I found the bug I think

It's actually in the listPush function

[marsh jackal]

right, thats whats causing the heap overflow on your end right?

[stone turtle]

list->items = realloc(list->items, sizeof(list->items) + sizeof(item));
```This is the same as doing
```c
... sizeof(void **) + sizeof(void *) ...
```which then probably evaluates to:

... 8 + 8 ...

[marsh jackal]

ohhhh thats absolutely right

so the correct way should be to reallocate it to a block of memory with (n items * sizeof each item) + (sizeof new item) right?

[stone turtle]

Also the listInit function can be shortened to this:

LIST* listInit(void){
     LIST* list = calloc(1, sizeof(LIST));
     return list;
 }
```Since it makes no sense to `calloc(1, sizeof(void*));` as you can just leave it `NULL` for now.

Then in `listPush` you should change it to this:
```c
LIST* listPush(LIST* list, void* item){
     // reallocate `list->items` to the appropriate size
     list->items = realloc(list->items, (list->count + 1) * sizeof item);
     // append the item and then increment `list->count`
     list->items[list->count++] = item;
     return list;
 }

[marsh jackal]

thats brilliant thank you so much

[marsh jackal]

!solved