Sync Drift on Windows OS

[balmy cobalt]

Preface

I am a volunteer ITGmania developer. The ITGmania community has been investigating sync drift in the engine for ~10 months. We're at a place where have mid-song drift completely elimated, but still have a variance of ~6ms per song.

We have several users (myself included) who have periodically tested OutFox test builds as well as other clients to see if anything is truly drift free.

The thread on the ITC server is VERY long so I will summarize the findings and general needed information.

• It is hard to demonstrate, because in many cases, it only occurs every few songs, so someone has to be able to repeatedly get a FEC with a low excellent count or better. It is easier for better players to demonstrate this as there is less noise in their scatter plots. It's most easily noticed when playing one song repeatedly, as is common in ITL.
• Most players who were helping test ITGmania alpha/beta builds, as well as other clients such as OutFox, are capable of getting quads or quints reliably. I'm not quite that good, but I can reliably get a 2-star or 3-star.
  • All of these players, myself included, are not insistent on using any particular client. In other words, everybody who's been participating in this testing is willing to use an engine for daily gaming besides ITGmania.
• Sync drift was rarely reported/not much of an issue until ITGmania version 0.8.0, which had some commits that ended up causing drift more frequently.
  • Many users reported that 0.8.0 would be drift-free for several songs, and then the sync would be iffy on another song, but then be drift-free again after the problematic song.
  • This is the kind of behavior I noticed the two times I tested OutFox. When it drifts, the player suddenly feels like they have to compensate for the timing window slightly moving from where it's expected. It's enough to push from the Fantastic window into the Great window for 1-2 consecutive notes, which is about the same as ITGmania 0.8.0.

• Some fixes were made for ITGmania 0.9.0 which generally made the sync better, but still had drift mid song. This version no longer stayed off sync when it drifted, it would get back on sync.
  • The 0.9.0 build would slightly drift late or early mid-song but then "snap" back to the proper sync quickly.
• 0.8.0 and earlier were prone to getting "stuck" off sync for a while, or sometimes a whole song. 0.9.0 resolved this.
  • 0.9.0 resolved an issue where the sync would drift late around an hour into uptime (RageTimer bug due to weird bitshift math in GetTimeSinceStart).
• Within the last few months, we made enough changes to the engine that many users started using the beta build as their daily driver. This would be mostly drift free within a song, but the visual delay may need to be adjusted between songs. The offset per song seemed to be consistent, so if you wanted to repeatedly play a song, you wouldn't have to adjust the visual delay between plays.
• An ITGmania test build called 0.7.1 made ~3 months ago has become very popular, since it doesn't drift mid song but might require the global offset to be slightly adjusted once or twice per hour. This has been used in at least one tournament, on Windows, with no drift issues.

It should be noted that the remaining drift we have in ITGmania is Windows-specific. Mac users and Linux users are, generally speaking, experiencing NO drift on the release version of ITGmania 0.9.0. My testing below, and the testing of other users I've shared, are all on Windows.

My test plays

• Ryzen 5600U with Windows 11
• The latest OutFox test build multiple people tried on our server was OutFox-0.5.0-pre043-a24-Win64-date-20241127.
  • There are at least 2 others who tried this build, I'm not sure if their screenshots have made it back to the OutFox team or not. I've shared one other user's scores below.
• I used the Digital Dance theme v1.0.55 since it's compatible with SM5.1+, and I was familiar with it.

First set

I played the 7 of "Clean It Up" from "Lobster Roll" because I can reliably play it over and over and get a two star with 515 +/-5 Fantastics. I used this for my testing because of my consistency on this song.

• My first play (the 95.66) was the most drifty of all my 8 plays, it seemed to settle in pretty well after that. I definitely felt that I had to adjust to the sync moving around on the first play, but the other 3 felt perfectly solid and drift free. This is a similar behavior to how ITGmania 0.8.0 drifted. You can see it get "stuck" in the excellent window for a while, followed by a noisier section. I can elaborate on how we resolved this
• I was already warmed up, and the scores after it were in line with what I usually get on that chart, so I don't think it was a me issue. That play I felt like I had to adjust for the sync moving a few times, whereas my next three plays felt absolutely perfect with no drift at all.

Second set

• Two drifty plays followed by one almost perfect play (Single little drift at the end) and one perfectly solid play. These were quick instances of the judgements not matching up for a few notes and then going back to where it was. I also included a screenshot showing total uptime.
• The one with 3 greats, those are my fault. The other greats were drift; i was playing as if i was AutoSyncing (judgements shut off, just stepping to the beat).

This picture shows the uptime - old versions of ITGmania (0.8.0 and before) would drift more as time went on, so we tried to do a set, leave the game open for a few hours, and then keep playing. OutFox did not drift more as time went on.

The following scores with the default OutFox theme are not my scores, but scores from one of the volunteers who has been helping test ITGmania for many months now. This tester thinks, like I do, that OutFox will be fully drift-free for several songs, but occasionally it will drift very briefly before getting back on sync.

[main flare]

what windows version? and looks like you were playing with vsync off. (you should enable that) I'll install windows and see if I can recreate anything like this. I would expect input to go early if it was 'going out of sync' but the fact they go late is weird since that would imply something on the input handling side, not the sound side. All the screens of itgm I have seen all go super early which in my mind makes sense. (squirrel can probably correct me if this is wrong since he knows the sound and input stack inside and out)

Also is there anything unique with these files or test setup? cbr oggs, vbr mp3s, etc? what input are you using?

[balmy cobalt]

what windows version?

Windows 11

and looks like you were playing with vsync off. (you should enable that)

In ITGm or SM5 our community recommends not using vsync. Many of the top players are strongly against using vsync, but I understand that could be a SM5.1 pecularity that does not affect Outfox. Anyway, that is why I shut it off, but I can try again in OutFox with vsync on.

I'll install windows and see if I can recreate anything like this. I would expect input to go early if it was 'going out of sync' but the fact they go late is weird since that would imply something on the input handling side, not the sound side.

We don't think it's anything input side, at least in ITGm/SM5.1. There's been an extensive amount of testing done on our server to rule out the input side of things as being related to drift, but if you (as in, the dev team) know of any issues with SM5's input handling, I'd be very interested to hear any relevant findings. We believe it's either audio-position related (ragesound) or visual (arrows may stutter - ArrowEffects is the significant place using time as a float, the rest of ITGm is using integer based time in milliseconds, the theory is this is due to the floating point time in seconds not matching up perfectly with an integer milliseconds time).

All the screens of itgm I have seen all go super early which in my mind makes sense. (squirrel can probably correct me if this is wrong since he knows the sound and input stack inside and out)

I will go through our thread in the other server a bit and see if I can find any evidence of drifting late. I do not remember off the top of my head if it only drifted early. Our thread is here if anyone would like to take a peek themselves. https://discord.com/channels/292111865658474496/1210363069235204156

Also is there anything unique with these files or test setup? cbr oggs, vbr mp3s, etc? what input are you using?

Nothing unusual here, we're generally taking care to use CBR OGG by step artists who are known for making very well synced charts. We've done tests on mp3 and don't find it to be noticeably worse than ogg, but everyone is pretty much using ogg.

Let me know if you think of anything else for me to check on! I'll try to get another test set in with vsync on before the weekend is up.

Also, I know squirrel and I discussed this a long time ago, but our most stable builds are not using low_sample_count_workaround and have also (mostly) reverted to sm5.0.12's GetHardwareFrame/ClampHardwareFrame

[main flare]

yeah, vsync was not really an option in sm, thats long fixed in outfox. audio position explains the issue on sm/itgm, but doesn't explain it on outfox where you start hitting late, that seems backwards to me, which is why I brought up input (in reference to us, not sm). I'll setup a windows computer and try to replicate it, can single digit excellent most stuff so should be pretty obvious if it feels different then my linux setups.

[balmy cobalt]

I forgot to answer your last question btw. I'm using a StepManiaX Gen 5 stage

[small zinc]

Hello! I don't test all the builds since I don't have time and I'm also trying to maintain some level of competitiveness, which is difficult to do while testing tons of diff builds/etc. But I do test the new builds that @balmy cobalt puts out every like... 2-3 beta releases or so?

I actually used to main Outfox too! Back during 2020ish? Somewhere around there. But for a good little while I used Outfox as my main program for playing, and it felt solid. ITGM/SM at the time also had very little issues. But this was also before my main switch to the new blue window (15ms) and I think having that window made the issue much more apparent since there is more precision involved.

Like mentioned above, the issues began to start around the era of ITL2023ish iirc? 0.8 or so. Not sure what build that was for Outfox because I haven't tried Outfox since testing it earlier this year. I grabbed whatever version was on the website and that also had drift as well as ITGM's most recent build. Someone mentioned to me that the version on the website is not the most up to date one and to get the most recent Outfox build from the Discord and try again.

I have zero affiliation to either program and don't particularly care which is better for what because I don't play mods or do anything engine specific. I basically just want whichever has no drift lol. And so far, the most recent builds (that I have access to, that is) for both Outfox as well as ITGM have very noticable sync drift on multiple platforms (Windows 11, Windows 10, two separate laptops w/ diff specs, a desktop w/ AMD hardware as well as a desktop w/ Nvidia/Intel hardware). I do not have access to test all of these anymore as I'm going through a very lengthy move, but I do still have access to my all AMD desktop setup. So that's my main play PC now and will be for the foreseeable future. It's W10, fwiw.

Happy to help provide w/e's needed to assist, but I'll prob be pretty busy til after holidays.

[main flare]

thanks for the info, will setup a windows test bed and play on that, and see if I can recreate anything

[quaint stratus]

Indeed. We also had to rewrite mix buffer and merge due to apple issues, but we are closer to sm4's audio setup than sm5s in the position calculation section.

We still have workaround, but it's not functional anymore

[main flare]

@quaint stratus what audio config should I be using for windows? rtaudio or just whatever is default?

[quaint stratus]

Waveout as default as itgm does not have po/rtaudio

[main flare]

first test has nothing to do with audio, just checking to see if the chart, game clocks drift, compared to an externally timed input. Results were perfect there. Did a bunch of tests. I have a micro controller pressing every 250ms, and a 120bpm chart with all 8ths. I even reset the controller a few times in the last run to try to get something to gltich out. outfox handled input reconnections like a champ.

I am thinking about a test bench for the sound, and I thinking just making a chart with a constant beeping noise. the microcontroller will press a key on every beep, if the audio drifts the graph would reflect that

from scipy.io.wavfile import write

sample_rate = 44100
duration_beep = 0.05
duration_silence = 0.2
frequency = 1000
total_duration = 120

# Generate the beep sound
t_beep = np.linspace(0, duration_beep, int(sample_rate * duration_beep), endpoint=False)
beep = 0.5 * np.sin(2 * np.pi * frequency * t_beep)  # 50% amplitude

silence = np.zeros(int(sample_rate * duration_silence))
cycle = np.concatenate((beep, silence))
num_cycles = int(total_duration / (duration_beep + duration_silence))
audio = np.tile(cycle, num_cycles)
audio_int16 = np.int16(audio * 32767)

file_path = "beep_every_250ms.wav"
write(file_path, sample_rate, audio_int16)

file_path
``` iamthedjofthisgig

[quaint stratus]

https://tenor.com/view/dj-stove-aoki-dj-kalan-dj-fire-gif-19349344

[main flare]

perfect

[quaint stratus]

banger

[main flare]

just to compare i tested a bunch of times on the latest itgm, there is def an issue there.. every result is the same, slowly getting earlier and earlier, this is only 2 minutes

same thing with vsync btw, slowly just goes outta sync early

[balmy cobalt]

Is this the release version, or our current beta?

[main flare]

Release 0.90

[hasty cliff]

You got me curious, does this happen on sm5/5.1?

[balmy cobalt]

Thank you, I'm gonna dm you our latest beta also if you could please check if that's also affected?

[main flare]

Yup

[main flare]

This was a quick functional test, I'll refine and actually run it across the different clients in two/three days, going camping for now. Just awesome it worked first try

[main flare]

i will get some more results later, but I accidently left itgmania running for a few days and reran the test... it was sigificantly off when doing the sound reative test. But when restarting the app, the sound reative test was actyually really good. So I think there are two different problems going on. The game timer gets off, but the sound is likely compensating (which would explain why a sync''d input slowly goes out, but a sound reative hit actually works well), but eventually that sound also gets way off.

this was after a a few day uptime, restarting brought it back on sync

Going to do the same tests with outfox and let it sit running for a couple days too

[balmy cobalt]

Thank you for all your efforts on this issue, Cube! (And belated happy holidays 🙏) Do you mind if I screenshot your few posts above to share it with other itgm people?

[main flare]

yeah thats fine.

I also accidently made the best machine autosync possible 😄

outfox, vsync off, waveout, latest test build, windows 7

itgmaniaa same test on fresh boot, I tightened up the sound reactive test to only take about ~1ms which yeilds much more accurate results.

[balmy cobalt]

Curious if you were ever able to test Outfox on Windows 10 or 11? I believe a lot of people who have dedicated stepmania PC's in the ITG world are running Windows 7 or Windows 10. We have some users on Windows 11 but very few compared to like Win 7 or Linux. It seems like people have been avoiding Windows 11 when possible, I know ITGmania will momentarily stutter when Windows 11 does its news/weather update in the taskbar if you're in Windowed mode, but Outfox doesn't.

I guess if anything, Windows 11 should be the one to test, because it seems like we have had many more reports of drift issues in ITGm from people on Windows 11. I know a lot of people who have had this issue and have a dedicated machine for the game are usually willing to switch to some Linux or older Windows. For various reasons some people may not be looking to switch OS's, so we've been meaning to see if there are any reproducible issues that exist in W11 but not older Windows (such as the aforementioned news/weather update stutter).

Sync Drift on Windows OS

[quaint stratus]

there is a provable stutter with the notifications system, if it comes from Edge, or Chrome browser, oddly not with firefox for some reason, which can really cause issues with SM

someone sadly had a browser 'notification malware' where this thing kept on recommending shopping channgels and such every 30s even offline, and it played havoc with games

i've had it myself on my win11 machine where it'll basically make you have to restart the game

not just an SM issue though, it's across many games

[main flare]

I wont be testing on win11, but I might be able to get win10 setup for an outfox test. So far all results on win7 are pretty much perfect.

[balmy cobalt]

Thank you both 🤝
Cube, yeah a Win10 setup would be great if possible. As far as I can tell, Win7 is pretty much problem free. I've seen in my own testing that ITGm is more sensitive to Win10/Win11 specifics than Outfox is, but that's not hugely surprising because the Windows specific code is mostly unchanged from SM5.1. Though, I'm also curious if Outfox isn't affected by some of this Win11 behavior like ITGm is because you are generally using different drivers than us, or if that's something on the Windows arch side?

[quaint stratus]

well, we use different systems to mainline SM5, i did discuss these before you went off on a massive tangent about 300 days ago or so lols

one of the biggest issues with legacy SM5 is it's ctime

it doesn't know enough about the clocks to adjust

not dissing you, it's your game and direction, you were free to take or ignore us, so no hard feelings

but you focussed way too much on the chrono side, rather than the math i was talking about and discussing

[quaint stratus]

I don't wish for anymore us and them, as i told you before

[balmy cobalt]

Sorry, but not sure what you're referring to @quaint stratus ? No offense taken or anything, just confused. We did focus hard on the RageTimer side and tried to make all instances where time is called more accurate by using the microseconds time directly instead of converting to floating point, and I was even thinking to look back to ArchHooks side of things again to see if it was related to system clock stability, but it sounds like I might have overlooked something important. However, even so, the very best accuracy players using ITGm think Linux/Mac no longer drift since 0.9.0 release. But we still have Windows users (who do also play on Linux setups they think are drift free) saying neither ITGm or Outfox is perfect. And I know Cube said his Linux setup exhibits no drift issues. So that's why I have been trying to get an idea of what to rule out as possible explanations for why neither game seems perfect on Windows.

[quaint stratus]

outfox does drift

but it's only on older hdmi devices.

this drift otherwise

is not the same as what ITGM is exhibiting

it's why i polled our server about it, because the claims were getting annoying and eating into our development time, as we check these things

we could not replicate any drift or skew on any of the 20+ devices the devteam has

i was able to get a drift on a 2004 monitor, when the audio was played through it on a 20year old cable

but it was a slow 1ms / 7min or so drift

not mid song or such like ITGMs

changing the monitor/cable fixed it as well, even on the 2006 era machine

but this edge case is not going to affect other SM/ITGM users sadly so nothing we can rule out for your skew issue

Cube's testing will be a bit more logical than ours, i've done programmatic test cases, but it's better if @main flare handles this one, i will chip in with item's we've seen, but yea, i hope we can solve it.

[main flare]

@balmy cobalt I got some weirdness with the version on the website (v42), but that one is pretty old at this point, so if users are using that I believe they might have some issues. So far my tests on the 43 build has been great, im going for one more day uptime to see if it does the same thing itgm did. but so far its been spot on.

[balmy cobalt]

Okay, thanks for this! I'll check back to see the results after one more day uptime. Also I'll check with our users who made reports to see if they're using HDMI.

[main flare]

was not able to reproduce the drift issue with long running outfox (ran into other issues unrelated though). will see if I can get win10 setup, but I think I can comfirm we don't have any measurable drifting. I am going to redo one more test by taking my sync chart and making it an hour long

[balmy cobalt]

OK, sounds good. It seems to me that you're at the same place we were after we resolved the major drift issue. (ITGm no longer has the mid song drift where it will drift out of sync and stay there. that has not been an issue in our engine since before the 0.9.0 release). No long term drift, no mid song drift that doesn't resolve very quickly, and the sync test using the microcontroller for input comes back perfectly.

If the automated sync test comes back perfect, that's very good, but that still leaves us with the very slight drifts which resolve quickly, as shown in the screenshots I provided when opening the thread. And this is a common behavior between our two games,** AND even then only on Windows 10/11**, so I am beginning to suspect Windows to be the culprit more and more.

This is because

• The methods to get a monotonic clock from Windows are............ not as good as Mac or *nix
• People really only report it happening in W10 or W11, making me think it's something to do with that newer OS
• Our testers are saying the same very slight drift still exists in both Outfox and ITGm's newest builds on Windows but do not exist on Linux or Mac

So, this is the drift we're still struggling with tackling in ITGm - the very slight one as i reported initially in this thread with screenshots. I do recognize it's slight enough that it will only be noticed by players playing on a tight timing window (15ms or tighter) so realistically many players are not affected by this amount of drift.

My theory at this time is that it's either related to Windows method of high resolution timestamp retrieval or is a very slight inconsistency caused momentarily when converting the in-game time to fp for ArrowEffects.

I'm happy to go into greater depth about what we've done either here or in vc if that helps.

[main flare]

will have win10 setup with the hour long test this weekend. You keep saying both clients experience a sync drift that resolves? are you baseing this off other tests? (or win10/11 reports?) The tests I have done on win7 show outfox does not have the long running sync drift that sm5/itgm have (which does not resolve), and itgm also has the weird timed input (not the sound reactive) drift too. Those are two different issues.

anyways, win10 test coming soon

[balmy cobalt]

To be clear, I'm not referring to the long term drift in here. I'm aware the long term drift as exists in sm5 is not fully resolved in itgm 0.9.0 but we think it's solved as of the September-October test builds.

This is an issue that I believe is somewhere between the timer and the visual side. Because the sync tester is not affected by it. But the sync tester doesn't care, or know, what's visually happening. That's why players with extremely good timing (sometimes better than AutoPlay) can reliably demonstrate this.

Again this is very win10/win11 specific, and affects both our clients, but only enough to really be noticed within a 15ms or tighter timing window. So users who are playing on 22ms or wider windows are probably unaffected. Mac, Linux, Win7 are not exhibiting this behavior.

[main flare]

I have win10 setup and the sync tester running. It's perfect for a while but had a random few minutes where it was terribly early. And all over the place, then recorrected. Im trying to record it happening to see wtf happened

[balmy cobalt]

That's wild, we've never seen anything like that gigantic block of noise (in testing or reported), but definitely have seen stuff like that stray Excellent like 4/5 way into the density graph.

[main flare]

That excellent was likely due to background noise

[balmy cobalt]

Oh makes sense, I forgot yours uses a microphone

[main flare]

I think I checked in and bumped the table haha

I'll run it tomorrow on outfox and while capturing too

And I wanna run the timed input test too

[balmy cobalt]

Fwiw someone on our server made a test that detects audio like yours, and someone else independently took an an old LTEK PCB (one of the 125hz ones) and had a microcontroller simulate an timed arrow press, no issues with either. And I had my own timed input test initially which I verified was a steady clock. So that's why we think the input side is not the issue, and mainly suspect something with the visual timing side or the system time retrieval. Not sure what you guys are doing, but we are using QueryPerformanceCounter on Windows.

Until we got the timer side stuff dialed in, there would be a slight "stutter" as the arrows scrolled up the screen sometimes, but it would correct itself, so it wouldn't surprise me if it's just getting slightly off and then back on track and that's what we're seeing as the cause of the drift that's common to both our clients.

[main flare]

this still happens on win10 as well.