Consistency Tracker Data Wiped After Crash

[wooden yarrow]

My PC basically crashed while i was playing Celeste, and I came back in and noticed all my times for the previous rooms were wiped. Just wanted to know if this was recoverable. If not, oh well

[untold sinew]

i doubt backups are made for these but @gusty crane could prove me wrong

[gusty crane]

there are backups IN THEORY and from what i heard from people they usually work. but i know of 1 person who said the backups were unrecoverable

[gusty crane]

go into Celeste\ConsistencyTracker\stats\ and find the file for the map youre playing

if there is a .bak.1 file with a very similar name, thats the backup

so you delete the original file, then rename remove the .bak.1 extension and the _backup from the name (so that the name matches the original files name), then start the game

(that also means close the game first)

in theory, this backup file should be impossible to get corrupted, but i've been proven wrong more than once on this topic

[untold sinew]

it can get corrupted if the game freezes while saving to it

renaming it should be a different story though

[gusty crane]

no, thats not how it works in cct

theres 2 separate files:

• regular stats file
• temp backup file
  when the regular stats file is written, instead the temp backup is written to first, then after that its copied over to the regular stats file. in this process, at least to my knowledge, only 1 of the files can become corrupt due to power outages, PC crash, w/e

then, the next time you load into the map, cct can have 2 things happen:

• temp backup file is corrupt, regular stats file is fine -> just load normally
• regular stats file is corrupt, temp backup file is fine -> move the backup file to .bak.1, then create a new blank file

[gusty crane]

so unless my assumption for this process is wrong, there should always be at least 1 copy thats working

[untold sinew]

ah so write ahead backups

[wooden yarrow]

alr ill rename the highlighted file and try that

[gusty crane]

make sure to also remove the _backup from the name, or else it will just get overwritten immediately again

maybe make a safety copy first

[wooden yarrow]

got this lil warning, gonna proceed but just putting that out there

[gusty crane]

yea thats fine

[wooden yarrow]

hmm that didn't work

[gusty crane]

[untold sinew]

whats inside the json

[gusty crane]

i would really like to know how this process can fail

probably corrupted

[untold sinew]

how big is the json file

maybe send it

[gusty crane]

there is something in the way c# does file IO that i am not understanding

[wooden yarrow]

json file of Normal is 3kb

[untold sinew]

would this be... normal?

[gusty crane]

3kb sounds like there data in there

actually maybe not

[untold sinew]

oh what about log.txt

[wooden yarrow]

where's that

[untold sinew]

uhh look at the log history folder in the celeste folder

find the one closest to before the crash

i mean i doubt there will be something but yea

[untold sinew]

i swear i remember having a file just open also made it prone to get corrupted

which is. yea

[wooden yarrow]

that's a couple days ago, i would go in LogHistory?

[untold sinew]

uhh have you started celeste more than 3-4 times

if so then its gone

[wooden yarrow]

loghistory has like 3 entries so i guess it might be gone

[untold sinew]

yeah that's why i'm asking

damn

i doubt itd have much info anyway

[wooden yarrow]

unfortunate i guess its confirmed then

[untold sinew]

question: is the write stream closed/flushed before copying?

[gusty crane]

it doesnt use a stream writer, i only use that for logging. stats writing uses File.WriteAllText

[untold sinew]

Events.Events 🔥

[gusty crane]

or can a move operation make the file corrupt??

[untold sinew]

i mean writealltext has to use a stream

[gusty crane]

because there is a point where only 1 file exists, when the backup is moved over

[untold sinew]

uhhhh if the underlying implementation makes a copy and deletes orig then theoretically

[gusty crane]

i would doubt that? why would it do that

[untold sinew]

i have no clue

[gusty crane]

that seems very inefficient

[untold sinew]

because it is

just changing the file name in the inode(?????????? i think?) makes more sense

i have no clue how this could corrupt files

but i swear long ago i got a file to corrupt just by reading it

[gusty crane]

idk

i will say, after i implemented the system in this way, people mentioning data corruption became a lot rarer. from ~2-3 per month to 1 every like 2 months, so it Probably helped

[untold sinew]

id say almost definitely

but yeah it uses a native call

im at a loss then

[tardy cosmos]

@gusty crane @untold sinew hi from another person whose backups doesn't work out too
it's sad to loose 100h of stat but ofc i don't blame you, sometimes programs doesn't work how we want it
here is some logs idk if it somehow will help but anyways
122332, 131128 are before crash and 133627 after

also is it okay that .bak.1 file contains only null's inside it?

[gusty crane]

im afraid if the .bak.1 file is corrupted then the stats are gone for good

[tardy cosmos]

yeah, I'm over it already, but is bak should normally contain readable json? or null's are okay due binary writing or smth?

[gusty crane]

it should be readable json, but its only created when the original file is found to have issues, as a copy of the _backup file. a long time ago when i implemented this i thought it was impossible for both the original and the _backup file to become corrupt at the same time, but apparently i was wrong

[arctic juniper]

Just a heads up please make your own help post for your issues in the future.

If you want to reference some other help post you can send a link to it.

[tardy cosmos]

oh, okay, it was just same thing so i decided write it here

[golden lion]

Only just now saw this because the post was necrod; you're using a copy instead of a move, and only the latter is atomic afaik

There's still the issue with OS-wide filesystem buffers / the drive itself being corrupted, but of you're intending to use a classic "write to alt file then atomicallly replace" then you definitely want a move instead of a copy

[gusty crane]

i also used Move in a previous iteration of this system, it still happened

[golden lion]

Then it's probably an OS file buffer thing + an abnormal shutdown

Or like full blown disk corruption

[gusty crane]

i plan on switching the system entirely, so that cct just keeps 2 files and doesnt move/copy/delete anything, and when loading the stats it looks for the most recently edited file and when saving just saves to each file slot in a 1 2 1 2 pattern

which SHOULD make corruption impossible

^huge asterisk here, because im pretty sure i said the same thing like 6 times already

[golden lion]

Also, for your consideration: https://learn.microsoft.com/en-us/dotnet/api/system.io.filestream.flush?view=net-9.0#system-io-filestream-flush(system-boolean)

[gusty crane]

idk, i no longer trust microsoft documentation on this matter. i'd rather find a solution that logically cannot fail, and i hope i got it with this one

[gusty crane]

and yea its always abnormal shutdown

[golden lion]

I see

[gusty crane]

always after power outages / pc crashes

[golden lion]

You should be able to adress that using that overload though

[gusty crane]

thats the thing: should

maybe, but will it actually work? idk

impossible to test sadly

[golden lion]

If that doesn't work then you're just screwed period

But adding that call in should reduce failure cases down to only drive malfunctions, and may your deity of choice help you in that case

Just wanted to point that out as a low-effort fix for something like this

[gusty crane]

eh, ill switch to my aforementioned thing, which should also work in any other cases like drive malfunction

but thanks for the advice anyways

[gusty crane]

speaking of this tho, how does that logically work btw? how can a copy operation corrupt the file that is being copied?

shouldnt that just read, never write?

or im guessing you mean it crashes here (or during the copy, but before backupPath was properly written to disk)