perf_prof_branch

[whole cloud]

Thank you for showing me what I have already seen a dozen times, please don't send me more of that crash, I have all I need, and don't ping me for it twice in the middle of the night I see all messages here anyway, k thx bb.

[maiden sequoia]

I believe it pinged two times because i edited the message to add a second link. It crashed 2 times in 5 minutes.

[whole cloud]

2.06.148581 new PROFILING branch with PERFORMANCE binaries, v6, server and client, windows 32/64-bit, linux server 32/64-bit

• Fixed: Potential crash related to a performance improvement
• Tweaked: Character orientation on tilted ladders is improved - https://feedback.bistudio.com/T83397
• Fixed: Unloading of a specific cargo with setVehicleCargo was not working - https://feedback.bistudio.com/T162101

if you don't want to use the Steam branch, the files are also available for alternative download here:
https://drive.google.com/drive/folders/15p9j7C2nHUt6NoVfChX4YFuqzFXzblJh

[empty goblet]

[whole cloud]

If someone finds something badly broken in this prof build, please find it till thursday, thank you

[silk pewter]

what, we can't report it before? 😁

[stark falcon]

does that mean 2.08 before winter break?

[feral harness]

unlikely

[whole cloud]

No, what sparked that idea?

[gritty wasp]

Maybe time limit for bug reports?

[whole cloud]

I just want to get bugs fixed before the weekend

[maiden sequoia]

🦧

[kindred radish]

Somehow we lost about 10-25 Server FPS over the last updates. (Stable/perf doesn't make any difference)
I talked to other admins with high pop servers and they have similar problems.(Exile & KotH (vastly different (much less objects/no AI)) 🤔

Quote from a KotH Admin: "Its dropped my KoTH server at 80 players from 40fps to 16fps, no AI and less than 50 vehicles total"

Day it started should be the - 6/11/2021.

---

We shutdown the server every 3 hours, so "mission" runtime was ~60 minutes.
13.11.2021 - https://gyazo.com/496b1ecfeb9663ccb92ad795cea49cc9
30.11.2021 - https://gyazo.com/07c5af8b6d921ae18d7b4803f347a2af
^ - lower player numbers worse server FPS)

• Made a test with no AI at all on the server and there was almost no difference in server FPS. I expected a huge difference since our server is pretty loaded (8k mission objects, 120+ AI, up to 70 players, no HC).

[mellow lark]

No mod updates in the meantime..?

[kindred radish]

Ofcourse there were updates but normal KotH for example doesn't use mods and the other servers are vastly different in terms of mods but all suffer a server-side performance loss.

[maiden sequoia]

@kindred radish is you using those files? #perf_prof_branch message

[empty goblet]

remember you can manually download the older server binaries from the google drive so you can verify your claim (e.g. if the 148560 perfoms better than 148581)

also another quesstion, how many CPUs that server has available or is it as some shared VM resource ?

[kindred radish]

i know but as i said, there is no difference atm in stable/or perf and our clients need to be able to join, so i couldn't revert to a older stable/perf version before

dedicated machine only for one arma server, ts server (mostly empty) and discord bot
AMD Ryzen 7 3800X 8-Core Processor @ 4.25 GHz
64 GB DDR-4 2666
NVME SSSD

And this quote from a much more experienced server admin with multiple servers (different game-modes)
"Its dropped my KoTH server at 80 players from 40fps to 16fps, no AI and less than 50 vehicles total"

[empty goblet]

i'm confused by no difference in stable / perf comment ... and what do the clients have with it ... anyone with main branch or performance binary of profiling can join server running main or (performance) profiling branch

the comments need to be clear ... which build number was fine and which ain't

e.g. 2.06.148470 is actual main branch, 2.06.148581 is actual profiling, 2.06.148560 was previous profiling etc.

also the MT fix which was returned to the profiling build 148581 already existed before, it was just temporary reverted due to the crashes i observed on our official servers

so, i would expect complains about performance to happen in the previous builds too

that's why we need the exact build numbers .... when the FPS was high, when it wasn't etc.

[kindred radish]

Similar bad perf in recent stable/perf. I know but server client version must be compatible so i can only go back some revisions ?!
The admin of the quote was from states it started for him at the 6/11/2021.
I think it was this hotfix update near this date (don't remember for sure).

[empty goblet]

build numbers, dates aren't helpful cause one can't really tell what the admin is running

is he running main branch, is he using profiling branch (performance or profiling binary) , is he using dev branch ?

Day it started should be the - 6/11/2021.
this is also quite annoying if i understand it properly ... that the low performance problem is happening for month already but we hear about it month later ?

why, why not report it asap

and i get no KOTH admins complain to me at KOTH discord either , so i'm not aware of any problem

[kindred radish]

We are used to getting blamed on the mods we use (Exile), make lots of changes etc.
So i always wanted to be sure about such feedback.

I try to get some more info and will reply back here.

[empty goblet]

ok, it definitely helps to narrow the case (to specific build / range of builds) if it ain't your mod/mission/server/scripting changes

[kindred radish]

And while looking at the patch-dates i think he got the date wrong in the message to me (https://steamdb.info/app/107410/patchnotes/)
I pretty sure he means the update from 16 November 2021, since he wrote that in the Exile Discord
— 16.11.2021 "Anyone else seeing a server performance drop since the Arma 3 update today?"

I felt some worse perf before (was running uptodate perf build all the time) so i can't say which build it was exactly

[empty goblet]

maybe he was before that on profiling branch and the update for WS overwrite it with main branch and he isn't on profiling branch since ?

in short, this is pure example why exact build numbers and branch IDs are needed (it's in .RPT)

Type: Public
Build: Profile
Version: 2.06.148581
this is profiling branch, profiling binary

Type: Public
Build: Stable
Version: 2.06.148581
this is profiling branch, performance binary

[kindred radish]

I am aware of that, i will add a extra log to our server with some basic needed performance metrics, so in the future i can pinpoint it more easily.
productVersion command returns this atm (perf client) ["Arma 3","Arma3",206,148581,"Stable",true,"Windows","x64"]
What that be suffice for the future ?
And i try to get the missing information this day, thanks already 👍

[empty goblet]

Type: Public
Build: Stable
Version: 2.06.148470
this is main branch, stable binary

[empty goblet]

ye that would be ofc helpful, if you run non-default memory allocator that might be useful to know too and OS version too (ideally full build number)

[whole cloud]

Would be awesome if you could definitely say that it was the mini update.
But as far as I can see the update had no performance relevant stuff in there.
Mostly just minor fixes

[hexed verge]

for the ladder orientation fix, i noticed this on the tanoan building type Land_Shop_City_04_F

[feral harness]

@hexed verge could you give coordinates please?

this is in editor

[hexed verge]

will do in a bit currently eating dinner

[empty goblet]

btw. just FYI ... W10 and W11 treats Arma 3 just on par to notepad.exe in priorities (not as Games) hence that's another problem compared to W7/W8

[hushed seal]

🤦‍♂️ Setting process priority to high should "fix" that, correct?

[quaint flame]

Perhaps, but you need to do that every time you start A3, except you use a program.

[fickle geyser]

TIL, why is it so?

[silk summit]

Someone at BI forgot to update microsoft on the fact that Arma3.exe is a game and not a regular program/app?

[fickle geyser]

I do not think it works like this.

[silk summit]

Of course, it probably works by setting a flag when compiling the executable so that newer versions of windows can detect it as a game and therefore giving it the priority it deserves resource wise 😉

[fickle geyser]

do you need to ping me all the time?

[silk summit]

Nain

[quaint flame]

In case you didn't know. You can turn off pinging

[silk pewter]

TIL

[quaint flame]

You learn something every day I hope

[fickle geyser]

unfortunately discord does not remember that and you need to click it every time.

[hexed verge]

so testing again, @feral harness. it is not near as bad as before for some reason.
position world [5755.25,10254.6,12.7227]

the one that i reported is the one at position world [5785.78,10276.1,15.476]

terrain both coords are from tanoa

[hushed seal]

Personally using process lasso to do that + assign cores.

[empty goblet]

ye, that with some twists

[thin wyvern]

You can add priority to registry for every program if you need HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\arma3_x64.exe\PerfOptions] "CpuPriorityClass"=dword:00000006 // 6=Above Normal

[thin wyvern]

And to assign cores start game with batch file ```@ECHO OFF

START "ArmA 3" /D "C:\Igre\steamapps\common\Arma 3" /affinity 3F "C:\Igre\steamapps\common\Arma 3\arma3battleye.exe" 2 1 0 -exe arma3_x64.exe

@exit```

https://www.eightforums.com/threads/cpu-affinity-shortcut-for-a-program-create-in-windows.40339/

3F = 0-5 on my 5600X

[feral harness]

does it look off if you spawn building on VR?

[hexed verge]

wait one will check

[feral harness]

also maybe you need to verify files, as it looks inconsistent

[hexed verge]

it is consistent on the second cord it looks the same as the first screenshot at least

[feral harness]

I’ll check tomo on the map

[hexed verge]

looks fine in vr, even with the direction change

also looks fine if you tilt the building sideways in vr

[feral harness]

should work fine even upside down

[inland dew]

Tested in YAAB priority normal vs. above, high and real time.
FPS isn't higher at all

If not even slightly lower vs. normal

[empty goblet]

it ain't just normal priority thing what i got in mind ...

[gritty wasp]

maybe 12 gen intel related. I can test. But YAAB is so random run to run.

[inland dew]

That's why you run it 7-10 times, each time saving min and avg FPS values and then make average of min and average of avg FPS

[gritty wasp]

Still sounds like average temperature of universe.
Ok.
Win 11. 12 Gen Intel

1. Stable perform better than perf.
2. (perf + Pcore affinity + above normal) vs perf =
   +2.2 avg min (+5%)
   +1.23 avg (+2%)
   +6 avg max (+6.5%)

[inland dew]

Because of FPS variation from run to run

And min and max FPS can vary a lot from run to run

Only avg FPS is +- fine from run to run, although can vary to + or - 2-3 FPS

[gritty wasp]

so +1.23 avg (+2%)

I think stable vs perf will give more. In favor to stable. But I don't want to test it today

[inland dew]

CPU model? Cores and cache frequency? Windows version? E-cores enabled? RAM frequency and CL?

[gritty wasp]

12700k + 4800CL38 DDR5. Full stock.
I don't have proper cooling for now. It is under 70 in arma, but hit 98 in cinebench. I need proper backplate and spacers. I think current one not tight enough.

[inland dew]

)))

[inland dew]

FPS with patch 2.04.147719 (LEGACY)
81.3 min. | 111.1 avg. | 196.2 max.

FPS with patch 2.06.148373
79.7 min. | 106.7 avg. | 181 max.

FPS with patch 2.06.148470 (STABLE)
80.1 min. | 109.8 avg. | 188.7 max.

FPS with patch 2.06.148581 (PERFORMANCE)
78.6 min. | 107.7 avg. | 185.1 max.

Only Arma and Steam were running, with all known and unnecessary background processes off and no other soft running to take screenshots, not even Steam overlay to press F12.
Restart of PC for YAAB runs of each build.
Waiting few minutes so Windows starts all the processes, to avoid lower/different FPS, in case I start YAAB runs, although there are some processes that still are starting.
Fixed CPU and GPU frequency, to avoid discrepancies coming from possible frequency variation.

[random isle]

@inland dew u disabled HT in bios ?

[inland dew]

To sum it all up, with patch 2.04.147719 (LEGACY) the performance is the best.

Patch 2.06.148470 (STABLE) comes 2nd with:
1.48% worse min. FPS vs. 2.04 (LEGACY)
1.17% worse avg. FPS vs. 2.04 (LEGACY)
3.82% worse max. FPS vs. 2.04 (LEGACY)

Patch 2.06.148581 (PERFORMANCE current/latest) comes 3rd with:
3.32% worse min. FPS vs. 2.04 (LEGACY)
1.87% worse min. FPS vs. 2.06 (STABLE)
3.06% worse avg. FPS vs. 2.04 (LEGACY)
1.91% worse avg. FPS vs. 2.06 (STABLE)
5.66% worse max. FPS vs. 2.04 (LEGACY)
1.91% worse max. FPS vs. 2.06 (STABLE)

Patch 2.06.148373 comes last

[inland dew]

no

[random isle]

disable and run again

[inland dew]

doesn't bring anything

[empty goblet]

oh, remember to share with us the startup command-line

[inland dew]

-skipIntro -noSplash -enable HT -malloc=cma_x64 (AVX2) -world=empty -noPause -noLand -showScriptErrors

[inland dew]

so, for now, using STABLE, gets you more performance than latest PERFORMANCE build

unfortunately, it's impossible to use 2.04 LEGACY, that gives the best performance, by far, to play on 2.06 servers (((

[patent dome]

Most performance related changes on the perf/prof branch right now are related to large AI numbers, far away ai, large triggers, and non-local ai. None of which really apply to YAAB, but multiplayer and larger missions.

So I don't think just going of YAAB is really what you should be using to determine if perf/prof is worth it.

[random isle]

id check that test mission code lol, 90% of mods in workshop are made by people who have no clue how arma works

[empty goblet]

he was just checking the perf diff since 2.04 legacy

[patent dome]

Yea, but he makes the conclusion that stable gives better peformance just based on YAAB, even though that barely shows any of the performance differences brought with recent perf/prof updates. Just hoped to bring some nuance to the results, as YAAB only covers singleplayer gameplay.

[gritty wasp]

And still would be better if it will not degrade

[inland dew]

so if all the changes made apply mostly to multiplayer, then why performance in singleplayer has noticably degraded then?

and that's not just me that has noticed it and it's also in multiplayer

one can say whatever one want's but numbers are there

[patent dome]

I know, I also can't give an explanation for it. But a lot of things are still getting added, fixed, changed. It could be anything.

[inland dew]

fixes/optimizations are really appreciated, but before moving forward, one should find the cause of lower performance with already made/implemented changes

[patent dome]

I'm also not saying it should not be looked at, I'm just trying to nuance that current perf/prof improvements won't show up in YAAB. And all other stuff is mostly anecdotal evidence, without the reproducible numbers YAAB provides.

[inland dew]

it won't show, sure, but performance has degraded and it's a fact

don't know how multipalyer related improvements could have degraded performance in multipalyer, as well as in singlepalyer at the same time, instead of improving it

you're the experts guys

we see/observe - we report

[patent dome]

I also can't explain it, I don't work on the code for it or have even studied it to know what could be the cause. I'm just trying to give my view on the observations.

[inland dew]

like people reporting lower performance on newer perf builds on their KotH servers, although there are 0 bots to begin with

[patent dome]

Just for this point, I don't think personally it is just the performance improvements for multiplayer that caused any overall performance degradation. There are many changes to existing commands, features and new things being added as well. Any of that could add overhead, which would need to be profiled.

[inland dew]

so we have lower multiplayer performance, although 0 bots and lower singleplayer performance, although it's not multiplayer

anyways, these issues need to be tracked down, somehow

[patent dome]

Profiling would be the tool for that, which Dedmen regularly does. But so far it is all based on known performance bottlenecks. I don't think we have a clue where to start looking for the small performance degradation from 2.04 to 2.06 (perf or stable)

With time it can probably be figured out

[inland dew]

as one can see, even latest perf is slower than stable

and current stable slower than previous stable

well, 3 to almost 6% degradation, I wouldn't call that small, to be honest

[patent dome]

I'm saying small because a 1-3 fps difference in min and avg framerates is not really noticeable at 80+ fps.

[whole cloud]

You have to keep context in mind.
5% at 80fps are alot less than 5% at 20fps

Who cares when you get 78 instead of 80 fps. Right. No one.

Who cares when you get 50fps instead of 12... Yeah.

[inland dew]

anyways, one should talk about % only

well, it's "just" 3-4 FPS less, but that's on my system

on older/weaker systems it can easily be more than that

[whole cloud]

Older/weaker systems never even get that high.
If something is slower, it's probably one small subsystem that makes up a small part of the frame, and not literally the whole engine

[inland dew]

well, you certainly know better

I'm not an expert

but improvements/optimizations/fixes should only improve performance and not degrade it, no matter buy how much

[whole cloud]

I wanted to do some YAAB focused profiling.
But didn't think it's really worth the time investment.
I don't care about squeezing out 1 percent more at 80fps, while the majority of the players plays in completely different MP scenarios at like 20-30fps with occasional outright freezes

[inland dew]

if performance goes down, how can one call that optimization/improvement?

that's right - one can't

[whole cloud]

If you want performance. I can just strip out all AI features for you if that's what you want..

[inland dew]

no, but if there are perf issues, one should admit it and see what can be done to solve it, instead of saying I don't care

and the definition of improvements is to improve, not to degrade. correct me if I'm wrong

[silk pewter]

a bug fix can take performance away yes

[whole cloud]

Oh you prefer to get 90 fps in a Singleplayer scenario, instead of getting towing and moddable keybinding and better ladder movement and steam rich presence and better ladder movement and more sound modability and more graphics settings control.
Ok then. Just go back to a older Arma version and have your fun there.

[inland dew]

what all of this has to do with lower perofrmance?

[whole cloud]

You don't get heated car seats for free.
Either more fuel efficiency, or heated seats. Not both.

[whole cloud]

Adding new features, making the engine do more work, takes CPU time

Sorry if you feel offended that I don't care about your few % fps degradation at 80fps (which is well beyond playable) but.. i don't.

I have other fires to put out than that are more important

[inland dew]

well, don't forget that it's singlplayer only, with not that many bots and armor, in a medium sized village
in multiplayer FPS is lower, of course

I just wanted to report my findings

I did it

it's up to you guys

whether you will (be able/willing) do something about it

[whole cloud]

Yep and thank you for that.
But not thank you for behaving like a few percent at that top end should be our top priority and we should outright drop everything were doing for your well-being.

[inland dew]

well, that's how you interpreted it for yourself. I only pointed out performance degradation, not knowing what causes it

I didn't say anything about you all having to outright drop everything for my well-being

[whole cloud]

You are telling us here we should stop "moving forward" until we investigated the very slight degradation you found.
No. We won't do that.
We will keep moving forward, and on the side keep an eye on performance (where it matters, and that's not squeezing out one or two more fps when you're already at 80+)

[inland dew]

but ok, I guess I won't spend time anymore on such stuff in the future, seeing the reaction

[inland dew]

it's logical. If there are issues and one continues to add/change stuff, later it might be even more difficult to find what's causing it or maybe not at all, since there will be so many changes after that

but ok, I'm off

Just so you know, there were already such situations at BI, in the past, when there were issues that weren't addressed timely or simply were not discovered and more and more builds/patches were released since and it was impossible to find when and because of what certain issues appeared, that weren't there before.
And I'm talking about the game itself, not a problem in an outdated custom mission

[maiden sequoia]

I don't mind if Arma 3 loose some performance for the map crash fix 🦧 🦧 🦧 🦧 🦧

[gritty wasp]

I don't want to be offensive and definitely appreciate your work and I would prefer moving forward instead of profiling 1 fps drop. But keep in mind that he is reporting with sanitised environment(no one play like that) and have heavy overclocked high end system. So where he have 80+ fps I think most of regular players have under 50. So drop can be bigger for lower performance systems.

[gritty wasp]

Did you try to use markers system but with less marker TYPES as it was suggested?

[gritty wasp]

Can't say about KOTH but for 150+ TvTs server fps better on perf build. I can't feel the difference, but can see it on graphs.

[inland dew]

It gives some hope

[vagrant zodiac]

I’m running AMD and have noticed a steady degradation of YAAB for some time. However, I can’t say that I’ve seen the same drops in MP environment- although that’s a lot more subjective than YAAB. Also without benchmarking regularly, I can’t say how much is directly down to A3 updates. One big step I mentioned here (as it was supposed to be a big performance step forwards- I think it might have been possibly particle related) but I never had time to do proper back to backs.
But over time, the loss is much more dramatic. My 1080p standard baseline not long after building my current machine was 94-97 fps average. Now getting over 80fps is not easy.
(And 84 FPS used to be my 1440p personal/ultra style settings average) How much of this is caused by windows security updates etc, I don’t know as I don’t have time to benchmark regularly. But this is despite other benchmarks actually improving over time with small updates.

But thought I should mention it given the discussion above.
We are obviously hugely grateful for all the fixes and cool new features like IR lights but perhaps more help is needed from the community to run things like YAAB for additional feedback?

[feral harness]

when the measurements were taken, recently all on the same day or you take one each version release?

[inland dew]

All the builds were tested yesterday, in 2 hours time, almost 30 mins of YAAB runs per build

[restive pilot]

Well if you tested the builds in that order one after the other maybe the performance loss was due to overheating

[inland dew]

As one can read, people with less powerful builds lost more than just few FPS.
But who cares, right

[inland dew]

No heat problems and PC restart for each build and several minutes of time given to Windows to load all the processes

I have good cooling and CPU and GPU frequency didn't drop or vary

It just is how it is. Don't look for issues in my testing or my system - there are none

[burnt turret]

Hey there! I'm a newbie and I just found out about this branch, I've been reading some stuff but I still feel somewhat bogged down by a bunch of technical jargon, can I have some ELI5 explanation? sorry if it's a dumb question!

[maiden sequoia]

Did you try to use marker system as suggested?
Yes, i tried, before the suggestion. It was 85% markers system and 15% drawIcon. My move to 100% drawIcon was to try to stop the crash. I never tried 100% marker system, btw.
Did you try to use less Marker TYPES?
When using markes system i used more or less 5 marker types with color variations.

[whole cloud]

early preview build that gets bugfixes and experimental performance tweaks early.

[burnt turret]

Oh! I was wondering if this branch basically had a lot of performance boosts that the stable branch didn't have, but it seems those improvements just get here first, and then on the stable branch?

[whole cloud]

usually yes. There are some performance things that are exclusive to profiling branch until they are sufficiently tested

[feral harness]

could you do reverse testing next time. ie recent builds first then legacy? For scientific purposes

[inland dew]

What's the difference in doing it this way exactly?
I checked files integrity each time I changed the build and then restarted the PC before starting to test

[feral harness]

I will tell you what is the difference, I would like to eliminate the possibility that windows shits all over itself the longer it runs. Because the difference in degration is consistent I’d like to be able to make sure it is not due to the OS

[inland dew]

Only for build 373 I upgraded from 2.04 Legacy to 2.06 stable (470) and replaced the .exe by 373

[feral harness]

I dunno what this has to do with what I asked

[inland dew]

But there were restarts of PC before testing each build.
It's not like I've tested all builds without restarting PC, just by switching branches and the one I tasted last is then therefore the slowest

[feral harness]

Ok, makes sense

[mellow lark]

Also why is almost 6% difference considered 'nothing'? That's pretty significant in my eyes

If only we had a built-in benchmark 😄

[inland dew]

I was switching the branch, then verifying the integrity, then restarting the PC and only then starting to test, closing all known and unnecessary processes, only Arma and Steam running, no soft to momitor FPS or to take pics, not even Steam overlay enabled + Arma launcher closed, once Arma started.

• I waited several minutes after restarting so that Windows starts all the processes, before starting to test, so processes that still were starting wouldn't influence FPS

[mellow lark]

Differences will likely be even bigger on lower end systems

[inland dew]

And no CPU/GPU frequency variations

[inland dew]

Some people with slower PCs then mine already experience exactly that

[mellow lark]

Mhm maybe my lower than before YAAB results aren't just random then

[inland dew]

And we have several players that complained on our servers, that their FPS is noticably lower than it was before , players that play on our servers on daily basis.
So they know +- the FPS they're getting usually when server full or closer to the end of the round or when it's Kavala or Pyrgos etc.

[inland dew]

You're not the only one

[mellow lark]

Well it could even be magnified if there are regressions on both clientside and serverside processed stuff in MP

I could provide some YAAB runs for reference if you'd like

[whole cloud]

At 80fps, a 6% difference is about 1.3ms frame time.
At 20fps it's 3.2ms.

Optimizing several millisecond big problems is usually easier than trying to hunt microseconds.

[inland dew]

But in MP and on average or below average PC people never experience 80

[whole cloud]

I can get huge improvements at the low end and with freezes within a few days.
But optimizing at the top end when the game is already running smoother than it needs to be is very hard and takes alot more time

[mellow lark]

Makes sense I guess

[whole cloud]

I don't want to invest months if work to get people up from 80fps to 120fps.
I'd MUCH rather spend the time to get people from 20 to 40fps, and in the meantime also add new features

[inland dew]

Most people play at sub 30 or slightly past 30 FPS in MP, just so you know

[whole cloud]

Yes, more frametime difference, but less fps difference.
It will be barely noticable if you are already low

[mellow lark]

I mean if you get people from 20 to 40 it's likely that top end systems will see improvements too

Maybe not as much, though

[whole cloud]

Exactly.
Which is why I don't care about minor problems on the high end that most people don't even reach

[whole cloud]

Yes. I know that.
That's the area I'm optimizing in

If I can do something that gives +5fps from 20fps baseline, but -5fps at 80fps baseline, then I will do that.

[inland dew]

You don't get it, at all.

It's just few FPS difference, but it's just for me, with my highly OC'ed PC and because it's singleplayer.

The difference is bigger than that in multiplayer and with much cheaper/older PCs, like most of users have

[whole cloud]

And i did quite a few of such optimizations recently.
Increase the base cost, but decrease the scaling alot.

For example ViV cargo, loading a object into empty vehicle is now slower, maybe even +100% slower.

But when you load into a full vehicle, it's like 40+ second game freeze, down to <1 second game freeze.

[mellow lark]

groove's system is like the 0.01% rig, if not less. Would be valuable to see results on an average system

[whole cloud]

Latest profiling also has the attachTo tweak.
It has actually a performance deficit in situations where you already run very high fps and do lots of rendering.
But we measured +10fps in contact campaign at I think 40fps.

[inland dew]

New features is very very nice, especially that useful and at that late state of the game.
And that they don't come for free, but put more load on the CPU, it's to be expected.

I have nothing to say about that.

But when someone is talking exactly/specifically about optimisations and people observe noticable performance loss, that can't be call optimisations.

Hope you understand what I mean.

Don't mix new features with optimisations.
These are completely different things

[whole cloud]

If you run at 80fps, then you are simply not my target group for optimizations.
If you loose performance which doesn't impact your playability, but that makes me improve other players playability then that's perfectly fine for me.

I don't know how often you want me to repeat myself.
Yes there might be slight performance deficits at the high end, as long as the low end sees improvements, i don't care about your deficit.

Once the low end issues are fixed, and the people who today run 15-20fps can run 60, then I start to care about your high end issues.

Your benchmarking is very welcome to see how the tweaks impact the high end, but in the end it's not something we need to act on, it's not a priority.
If you can show that there are significant deficits at the low end, then I'm very interested.

[pallid island]

Which version added the attachTo tweak? 👀

[fickle geyser]

You don't get it, at all.
You don't get it too, 3% in high FPS might not translate to 3% on low fps.

[whole cloud]

But "I get 3-5% lower on very high end, i ASSUME that PROBABLY also impacts the low end" is not very useful for me

[fickle geyser]

That's what Dedmen is explaining in the MS example.

[whole cloud]

Last I think, isn't it in the changelog?

Ah no, it was in v5

[fickle geyser]

So yes there's some FPS degradation in high fps scenario, but the question is Does it affect lower end machines and lower fps scenarios?

[inland dew]

Again, last time.
It's my system and it's singleplayer, 2 minutes run with not that many AI and vehicles, in a mid sized village.

in MP, with a lot of players, AI, vehicles, many many hours of non stop gameplay, with a lot of corpses and wrecks and buildings destructions and syncing stuff between players + their much much slower/older PCs, with 2.06 stable and especially with latest perf build, these players observe more than just few FPS lost, at their sub or above 30 FPS

[pallid island]

Ah, I see it now, thanks. Didn't put together that this was referring to attachTo

 - Tweaked: Potential performance optimization when attaching static objects

[inland dew]

As already said, there are players with much weaker PCs that report more than just few FPS lost, in MP

[whole cloud]

Groove.
You are not understanding me.
Yes, your Singleplayer scenario probably spends the most of it's time in rendering.
The MP scenario that applies to the majority of players spends it's time in simulation and networking and stuff.
They are completely different systems.

It doesn't matter to the low end players if rendering gets 20% slower, if rendering actually only makes up 4% of their total frametime.

But for you at your high end, with few AI and vehicles, you spend maybe 80% of your frame at rendering, and that 20% slower is HUGE for you.

It literally is irrelevantly small difference for low end players, but a HUGE difference for high end players.

But.. i don't care! High end players are already high enough

[inland dew]

Nevermind...

We're talking past each other

[whole cloud]

Performance degradation in one small (of dozens) subsystem that may impact you alot, may not matter at all to others.

I do see your performance difference, but "it probably also scales for lower fps" is just plain incorrect

[inland dew]

Other people report also FPS degradation, in MP, on weaker PCs and it's not just few FPS, but ok

[whole cloud]

That's also what NeilZar tried to tell you.
There were lots of improvements for AI.
Some even making differences like 3->50fps.

Meanwhile we did changes to pip rendering and stuff, that for you make the difference of 80->76 fps.

If you can show low end benchmarks or profiling data in exactly these problematic situations and comparisons there, that would be HUGELY useful.

But your 80fps benchmarks saying "hey 3% lower there" are useless when you're trying to talk about these completely different scenarios.

[inland dew]

Well, I was testing in 1080p standard video preset.
There PiP quality and draw distance are relatively low

[whole cloud]

You cannot compare your SP YAAB performance with MP performance.
They are vastly different scenarios with the bottlenecks being in completely different places.

[inland dew]

That I understand

But just so you know, I'm not an expert.
I was just told by several people that their MP experience was worse lately than it used to be no so long time ago.
And I simply decided to test in YAAB, although it's a completely different scenario.
But even in YAAB, although it's different from MP, I could see lower FPS.

Just to say it.
I understand it's different than MP.
Just my 50 cents

[whole cloud]

Again thank you for your benchmarking.
But stop telling us that your issues should be our top priority.
They are not and they won't be.

If people report lower fps at the low end, that's a priority.
But it's also VERY difficult to compare at that end because there are no standard benchmarks there, basically every run will be different.
Maybe the server was running for longer and there were garbage entities lying around, maybe the player had chrome in the background, maybe a windows update was installed.

Your clean benchmarks are nice for the high end, but they just don't help with he low end

Like currently we have reports on noticable fps drop on serverside, that's a big thing and people are helping to narrow it down and we will work on that.

But some people saying "my fps feel lower" and you measuring the high end and telling us we should make it a priority aren't useful.

[inland dew]

Never said it's only my issue and that it has to be top priority for you.
I think you see what you want to see, even if it isn't there.

Let's move on

[silk pewter]

anyways, these issues need to be tracked down, somehow

before moving forward, one should find the cause of lower performance with already made/implemented changes

I think those made it look like "I lost perf on my side, then it's not a fix"

I agree, let's move on
both want the best for the game, just that the POV & data is different of course

[inland dew]

AI and triggers multi-threading ❤️

I remember there were attempts to do something with the sound.
Don't know if it's still in the works, abandoned or already done

[whole cloud]

The sound thing never moved beyond idea stage.
But i think I still have it on the list.

Although...... Maybe it partly exists.
There is a prof branch exclusive tweak.
Normally Arma waits for previous frame rendering to be done, then does sound and AI, then renders new frame.
Or maybe I'm swapping some things, it was something like that.
I changed it to already do the stuff first, before waiting for rendering, so that it renders in background and does other stuff in foreground.
But there were no clear results.
It was the "waitBG" scope in profiling

Instead of waiting, it should do other stuff.

[empty goblet]

i will just chime In, to understand the work DD does in background (with help of kk, me, community)

i've got experimental 2.06.148592 server binary but most of the optimizing went into threaded AI and triggers and similar, so if anyone want to try , just say (use mention or DM me, note, it's unsigned executable, only windows 64bit, only profiling type of binary (rpt spammy), not performance)

so if someone here run windows 64-bit server with lot of AI or triggers and want to test it ...

[inland dew]

will you use it on WL EU?

[empty goblet]

it's already used there since friday, same on some Zeus serves, so far no oddities

[inland dew]

poor harddrive(s) )))

[empty goblet]

in relation to performance : unfortunately the fully threaded / split from waiting for primary/simulation thread renderer exists only in DayZ (fall 2016, it was still RV engine) , the soundengine multithreading similar, it was started but then moved on Enfusion

e.g. now , i just wish to figure / repro and fix those mostly AI/vehicles bugs next year but it will be hard

• friendly MG randomly execute teammates (including player leader) via FF (still unsure if it's related to crossing LOS&LOF / attempt to lay suppressive fire going wrong)
• friendly non-heavy AT ignores enemy wheeled/tracked APC/tanks (even while given specific order to fire at it while direct line of sight)
• AI instantly knowsabout you while firing launcher (often)
• AI instantly snap on your position (random, its either fine (they search for you, miss some shots) but when NOT then you dead in 1-2 shots, first burst)
• vehicle bumps/flops/fly away when you replace dead corpse in gunner/driver/commander/passenger with yourself or your AI (the corpse seems become unmovable colliding object for frame or two)
• flipping vehicles way too often (you will find vehicles literally littered alongside roads)

[inland dew]

It makes me mad...

[empty goblet]

and ofcourse the usual neverending goal of improved performance for larger player numbers in MP ...

feel free to add whatever you deem important enough ...

[silk pewter]

say3D FPP vs 3PP 😁

[empty goblet]

now i base my AI observation on 'suffering playthru of whole WL map at Livonia, Malden and Altis in past week (being ill allows me to just watch the horrors)

[quaint flame]

I wanna expand that list by:

• Prevent tanks from flying into space 😄

[restive pilot]

but that's 's best feature

[quaint flame]

Okay then I add:

• AI instantly knows your position when explosives are used.

[whole cloud]

Lol do you know that we had something like that?

If youd shoot mortar near enemy, and get out of the mortar before impact, AI would instantly know your exact position

[silk pewter]

oh!

I think in ArmA as well there was this "use satchel on tank" and even if you are behind the hill the enemy instantly knows your whereabouts

hopefully fixed since 😁

[inland dew]

Since Operation Flashpoint from 2001 )))

[inland dew]

Still there, since 2001 )

[silk pewter]

well it had been fixed at one point (maybe re-broken afterwards)

I did not encounter that in A2 nor A3, I should check again

[quaint flame]

It's definitely there in A3

[whole cloud]

Might be annoying to fix, they don't know how the explosive was triggered.
But I think generally explosive damage shouldn't reveal who caused it, since they cannot know that anyway.
Does that sound safe?

[restive pilot]

but would they be alarmed?
or just go about their business...

[whole cloud]

Even for a grenade, if you see it thrown you know from where and who.
But if it just explodes next to you, you have no idea

Yeah alarmed but not reveal enemy

Lol that'd be funny tho

[silk pewter]

tested

…it hurts

evidence that ArmA > Arma 3

[whole cloud]

[fickle geyser]

OFP > Arma

[empty goblet]

i was even funnier, if you used the remote detonate , the target and it's AI group knew all about you anywhere on map (and if there was mod on information sharing, everyone knew about you)

[silk pewter]

yep 😭
but I swear I saw it fixed in A1

[empty goblet]

ye that one was fixed ages ago ... guess something similar creeps back

[maiden sequoia]

@inland dew i'm having 65 fps in YAAB with a watercolled i9 10900KF (10 cores) / 32 GB DDR4 3600 Mhz / RTX 3080 Ti with 1080p and standart settings. This is bad?

[minor silo]

Yes.

[inland dew]

with good CPU OC and good RAM OC (RAM quality must be good + a lot of knowledge how to tune it), your FPS could have been closer to 90, but this section is not for such discussions, unlike #hardware_vs_arma

[feral harness]

I reproed it, is indeed caused by the changes but it shouldn’t, quite an interesting bug, thanks

[hexed verge]

heh, is it because of the un even terrain beneath?

[feral harness]

could be

I fixed it but am puzzled by it, the problem might lie deep in the engine

[silk pewter]

rotation speed at the time of action?

[hexed verge]

doubt it, when i tried, i was standing perfectly still. the angle was also always the same per ladder

[empty goblet]

in case someone want to try the server binary with performance improvements (no builds till 2nd week of January), not-signed, experimental!
pw: testbuild https://www.dropbox.com/s/t8tyu4dtvx3fxox/Arma3RetailProfile_Server_x64_windows.7z?dl=0
there are also new entries in RPT, which start with timeLimiter, e.g,

2021/12/05, 3:05:18 TimeLimiter, Category AITraceFireSector exceeded time limit, runtime 15589, limit 15000, denied requests 10
2021/12/05, 3:05:18 Ent ba83aba040, dbg R Charlie 2-6:2/I_LT_01_cannon_F [9215.067383,21692.531250,16.342579], time 933
2021/12/05, 3:05:18 Ent ba841c8080, dbg R Alpha 1-3:3/I_MRAP_03_hmg_F [3820.666016,13778.958984,23.315659], time 769
this means is that unit for listed operation exceeded allocated amount of ms per frame and it will be delayed till next frame
it should be in way that units which failed previous frame gets priority over new units for the next frame to avoid endless cascade
we didn't yet fully investigate why some (usually crew/gunner) causing this, so that's something for next year headache
you should observe uptick in better multi-core utilization with this binary related to AI (dedicated server/HC)
if mission / mods got lot of triggers (both in sqf and fsm), there should be noticeable improvement too
it ain't optimal, some of the AI and triggers optimizations didn't make it in, so that's something to expect in January

[tawdry gazelle]

Do you have a known issue list so that we may concentrate more on those problem and may give more detailed feedback?

[autumn timber]

Is this the same binary as the one you told people to DM you to get it or something else?

[whole cloud]

That timelimiter thing may also be the cause of some server deaths. Where fps drop to close to 0, most players get kicked, and then suddenly the server recovers and runs fine again

[empty goblet]

yes

[empty goblet]

not seen that as this version didn't drop to 0 yet lol

[whole cloud]

Well it should prevent that from happening

[kindred radish]

Is that build compatible with stable client ?
If so i would throw it on our server server, can't make our situation worse i guess.

[whole cloud]

Yu

[tawdry gazelle]

Agreed. We have had our server crashed without any crash log or dump once.

holy ****? that server dont even have any log???

[maiden sequoia]

Run server with -limitFPS=10 or 5 is good way to simulate a overloaded server? I wan't to see how a new feature will run on a full & overloaded server.

[whole cloud]

No

[maiden sequoia]

I'm using profilling, what is this?
total/wSimu/perAI/aiAll/aiGrp/aiGCT/tgTrk/tgTrF/tgSee

I have many tgSee causing a small freeze.

Using mods.

https://imgur.com/cLcp44z

[whole cloud]

Ai targeting

[feral harness]

121ms, how do you know this is what causing freeze?

[whole cloud]

tgSee is a recurring problem, i don't know if that's capped in the time limiter, i think it's not

[maiden sequoia]

This image shows it?
https://imgur.com/cLcp44z

I used:diag_captureSlowFrame ['total',1/10];

I'm using profilling from Steam. Not the last one Dwarden released above.

I removed all mods but not the custom map and CUP mods (can't remove CUP because there is CUP stuff in database). Same result.

Map: Chernarus Redux. I don't have this problem in Altis with no mods.

[empty goblet]

use the latest and you may know more

[maiden sequoia]

My servers are on GTX Gaming, so i can't update server exe 🙁

I'm just using client profilling.

[kindred radish]

20:03:08 Ent 23295895880, dbg R Alpha 1-3:1/Exile_Unit_Player [11251.826172,9436.665039,290.549286], time 18207
20:03:08 Ent 2332173da00, dbg B Alpha 3-6:2/RyanZombieC_man_polo_5_Fwalker [1704.375000,8879.375000,143.831253], time 13
20:03:08 Ent 2318cd64100, dbg B Alpha 1-3:2/RyanZombieC_man_polo_4_Fslow [12194.375000,8580.625000,4.676875], time 5
20:03:08 Ent 2331249ccc0, dbg B Alpha 1-3:1/RyanZombie22slow [12124.374023,8805.625977,29.488594], time 4
20:03:08 Ent 23184294780, dbg B Alpha 1-2:1/RyanZombieC_Orestesslow [12213.125000,14114.375000,43.978123], time 3```

Do you guys need logs with these ? I guess it doesn't make sense for non vanilla/close to vanilla scenarios ?

[whole cloud]

Exile_Unit_Player? But that's a AI unit? Why is it named player?

[vale shoal]

Exile_Unit_Player is the playable character of the Exile mod

[maiden sequoia]

Why it have time 18207 while others have time 3, 4, 5 and 13?

[maiden sequoia]

What client i need to connect to lastest Dwarden server Performance Binaries (2.06.148592)?

I'm using Steam Profilling branch and can't connect to the 2.06.148592 server.

[empty goblet]

148470 main and any 2.06 profiling/performace client shall be able connect

[empty goblet]

because it list first the unit which exceeds the allotted time

[whole cloud]

So is that unit a player?
There should only be AI units listed there, not players
Is it possible that there is AI in that unit?

[maiden sequoia]

Dwarden, GTX put your last EXE in my server and it became "red" on the launcher for me, i tried all branchs.

Server version is 2.06.148592.

Client is Steam Profiling Branch: https://imgur.com/A9sozFD

Same happens on my local server.

[maiden sequoia]

This post is from 10 December 2021. But the EXE date is 3 December 2021.

[empty goblet]

that's correct ...

[kindred radish]

possible, sometimes on disconnect (mostly on connection losses) the playerobject is not getting deleted and "AI" taking over.
"Exile_Unit_Player" is intended only for players tho.

[whole cloud]

Thanks that's very interesting

[feral harness]

yeah it wont get deleted by default unless you set it. Are you using HandleDisconnect?

[empty goblet]

such units are in WL too ... the SquadLeader is playable but when leave , the AI takes over (unless disabled in settings)

[maiden sequoia]

This is on a server with this binary #perf_prof_branch message ?
You just replaced the old server binary with this one? What client you used to connect?

[kindred radish]

Yea Exilemod is using HandleDisconnect to delete the unit, doesn't always work tho.

[kindred radish]

Yes. Just replaced the exe, like normal.
Iam running profiling build atm (client) (most of the other clients are using stable)

Here is another incident of the TimeLimiterlog from the same server period. (These were the only two so far)
https://pastebin.com/KY6XVjFn

Was around the time we usually got a mass kick lately (2h ~45m runtime) while also having low server fps (<15) and loads of stray entities (~15k+ total / 10k just from two clients).
Server didn't stall and didn't kick players tho, iam not 100 % convinced it will stay like that, but one can hope 😆

[whole cloud]

Do you have repro for stray entities yet?

[kindred radish]

not yet, i was too busy to find a "vanilla" repro

[whole cloud]

we usually got a mass kick lately
Please keep eye on such things, that info is really valuable, if the time limiter kicks in when you would usually get freezes or mass kicks that's very important for me

[tawdry gazelle]

Yes, happens on my server quite often.

As for the mass kick, my server would have 4-6 players kicked out of 20 total when server is sometimes overloaded. I wonder if there's any way to tweak or remove the time limiter?

[tawdry gazelle]

server overloaded: especially on some network broadcasted scripts (remoteExec I believe), or massive different types of units spawns.

[empty goblet]

overloaded with the 148592 ?

[tawdry gazelle]

No, with dedmen's perf binary. I believe it should be 2.06.148581.

I have 5 server instances, two on Dwarden's 148592, three on dedmen's 148581, one on stable. 🤣

haven't notice that happens on 148592 tho.

[empty goblet]

there is no timelimiter prior 148592 ...

[tawdry gazelle]

emmm then the mass kick issue was not caused by time limiter? May be normal network thing? It happens quite often when we play zeus missions... 🤔

[maiden sequoia]

On the left of the server name, in the launcher, you have a green filled circle, a red filled circle, or a red "X"? My server have a red "X" with this binary and i can't connect.

I also just replaced the exe.

[empty goblet]

you misunderstood, TM now prevents it, only if the server gets stuck at 0 fps it may have negative effects as DD mentioned

there could be other causes where TM fails but that's to be seen, from my PoV the servers seems more than less stable/reliable (FPS wise)

[whole cloud]

I think the server is so low fps that i cannot do network processing for so long that it exceeds the timeout and thinks players aren't connected anymore and thus kicks them

[empty goblet]

that's correct dedmen , i many times mentioned this as the infamous 0 fps freeze stall hickup ...

there is also variant of it, where it affects just BattlEye operations or Steam client ticket processing (most likely timeout on calls/mt)

so far, the TimeLimiter introduced in 148592 actually seems to help resolve (to be precise, it's work-around the stall issue) some of the trigger cases

[maiden sequoia]

🦧

[edgy comet]

Had that issue for 3 years due to bad code 🙃

[empty goblet]

ye ofc you can get other freezes/stalls due to scripting and some other bugs ... aka there is more than 1 trigger case

[random isle]

yep thats like 90% use case, a shit code.

optimize your code before you post here

[maiden sequoia]

When i create a mission on-fly i use scheduled code and even periodic uiSleeps between objects spawn just to not cause freeze on players.

I'm good at the "art of sleep" lol

[tawdry gazelle]

Yes, I misunderstood... I thought it was a common feature on every version...

[maiden sequoia]

This is why i was having problems with last unofficial server performance exe:requiredBuild = 999999999;

🤬

There is no client on that version, so NO-ONE was able to connect until i reduced requiredBuild. I set it to zero.

[rain moth]

do you need to set it at all?

[maiden sequoia]

@rain moth not set it is the same as set it to zero, you get:
Required Version 2.06.*
in the Launcher. So no, no need to set it.

[autumn timber]

If requiredBuild is set to a large number, like requiredBuild = 999999999; for example, it will automatically be lowered to the current server version.
Heh, interesting. Didn't know that.
Could it be that the definition of a "large number" has changed? 🙂

[random isle]

this new? Tried to AccessTargetList for non-local AI group while non-local targeting was disabled! Engine will now enable non-local targeting which will hurt clientside performance.

[whole cloud]

Ye

[random isle]

Is that cause of findNearestEnemy and forgetTarget function ?

i use that for mortar fire script fire and forget

[whole cloud]

ye, for example.

If you run it on remote AI

[random isle]

the loop runs on HC for local HC units though

[whole cloud]

There are also some other reasons that i need to check.
I think if it's script caused it should log which script it was

[random isle]

cant even replicate it, but i do see that error few times in 5-6 hour games

[plain trout]

is it possible to change the default memory allocator to the windows (system) allocator for windows 10? At least on Livionia the performance double or tripples and the weird tree groups popping in and out is gone

[empty goblet]

each major release of OS changes the kernel and memory allocator behavior

also, this is first time i hear about N times gains ... not even the SSE4 vs AVX1/2 was able go past 10-20%

[tawdry gazelle]

There might be some bugs related to memory allocator though. Some players in my server have reported terrain objects flickering and I tell them to change memalloc to system, all issues gone... 🤔

Although the best solution to the problem is reinstall the OS... but they wouldn't love to. 😅

[random isle]

i re-install server os on every game end 😄

windows 2019 core with docker containers managed from gitlab along with code updates to mission, full automation ....

[full nova]

how to revert back from Perf_prof build?

[inland dew]

Same way you did to switch to it

[empty goblet]

it's possible as the default TBB4 allocator isn't latest of v4 nor newest (2021 or w/e is the build now) and Intel done several fixes in meantime

also you can try use the CMA allocator, it has 32/64bit libs too (SSE, AV1, AVX2)
download here: #end_of_life_arma message

[verbal mist]

Hi @empty goblet an @whole cloud thanks a lot for your work!
Since I bought a 12900K and did some benching in YAAB my question: Is it possible to enable AVX512 / are there plans to enable it in the Arma3 memory allocator ? If you disable the E-Cores the P-Cores of 12th Gen. Intel CPUs are capable of AVX512 and most mainboard manufacturers allow it to be enabled, ASUS has this option ; MSI will allow it in the next beta bios.

So any chances of you enabling it in a future memory allocator?

[empty goblet]

AVX512 will give nearly no gains compared to AVX2 except turning your cpu into roasting pan

[whole cloud]

No. All default allocators need to work with the steam listed minimum system requirements.
And AVX512 isn't in there

[verbal mist]

@empty goblet and @whole cloud thx for your very quick and clear answers. 👍
Highly appreciated.

So I will start memory tuning and AVX2 instead of AVX1.

[empty goblet]

ye the difference between avx1 and avx2 is small but still there is ... someone like @inland dew may tell you his findings results

[verbal mist]

I really want to get beyond my last result of 128 fps in YAAB but the suggested memory timings in MSI Dragon Ball 1.0.0.8 produced too much errors. AVX2 might be enough to crush the 130 wall. A project for the holidays.😀

[autumn timber]

128 fps
ONLY 128fps? How can you even hit something with such a slideshow! 🙉

[mellow lark]

No, we will spend 50 hours for that 1% more fps

Hehe xd

[autumn timber]

I dunno... dude is probably playing only COOP missions because 128 fps is literally unplayable

[empty goblet]

maybe he has 144Hz display and trying to match it in fps 🤣

then, at 24th gets 200Hz monitor as present and needs to start tweak again

[verbal mist]

oh no that sounds like too much pain
3440x1440 @100Hz it is.
And Benching is a worthwhile endeavour in itself - I am still a beginner in this art.

[elder torrent]

Thats insane I've never seen higher than 50 with standard settings. What arma settings, system settings and hardware are you running?

[tawdry gazelle]

Hi, how may I know if the custom alloc works? I can see its name in rpt file, but seems no other more obvious information?

[empty goblet]

it works ... you will see other name instead if it was using another allocator and there would be no game if it didn't work 😉

[tawdry gazelle]

Ah, that's good. I've follow the instruction provided in the CMA.zip to compiled 2019_U3 version, it seems work, too.

If you are intrested, may I post it here for more people to test?

[tawdry gazelle]

Okay... seems 2019_U3 version does not gain any performance at all... even worse than system alloc...🥲

[tawdry gazelle]

I wonder if the memory allocator really uses AVX512 even though I compile it with the feature enabled in Visual Studio...

[restive pilot]

why would it use AVX512?

it should use AVX2

[tawdry gazelle]

just curious about that... 😅

[mellow lark]

Lol, latest Alder Lake with good ram...

[tawdry gazelle]

Hi, I've found that Intel's tbbmalloc has been discontinued in its original repos. Instead, they are continue updating it in oneTBB repo (https://github.com/oneapi-src/oneTBB).

I've followed the CMA's source code and made an implementation of the oneTBB 2021.5.0 branch, and run YAAB for test with heavy mod loaded.

[tawdry gazelle]

I've run YAAB several times, and found the new version has a 5% average fps boost than Dwarden's CMA alloc.

And the new version seems better handling worst case fps, and has more stable frame interval. (or what do you call time between frames? frame pacing? frame duration? I dont know what it is in English)

Though I'm not sure if I implemented the hugePages part right... 😅

Binary: ~~https://1drv.ms/u/s!AmQv-K9BLIUolWBIzN2atugwVk3m?e=ftbGDU~~ Newer and more: https://1drv.ms/u/s!AmQv-K9BLIUolWF61t7GrOU5rxZL?e=CfAShV

[empty goblet]

it's with the source included ?

[tawdry gazelle]

source code is here, just published.

https://github.com/GoldJohnKing/oneTBB/tree/Arma-3-v2021.5.0

Ah, I have to hit the road now... (going home, going home...) 😆

[autumn timber]

I was surprised to see only 19 commits in your branch, but then noticed that you branched out of the onetbb_2021 branch and not master 🙂

[tawdry gazelle]

emmm I only pushed one commit... you sure you are on the right link? 🤔

[autumn timber]

Yes, I'm on the right link. And onetbb_2021 has 18 commits so it's alright

[tawdry gazelle]

https://github.com/GoldJohnKing/oneTBB/commit/ace8e550cb5fe58ad79426838f6f5c5f468e40b7

[autumn timber]

I meant that I saw 19 commits in total in that branch, which made me go "huh?"

[tawdry gazelle]

Ah get your point🤣

[autumn timber]

Especially that master has 191 commits 😄

[tawdry gazelle]

im not native English so... 😅

[tawdry gazelle]

By the way I compile it with Windows SDK 22000, Visual Studio 2022, MSVC v143, and the same compiler parameters (/MT, AVX2, Speed, ...) as the original CMA.

[verbal mist]

https://www.computerbase.de/forum/threads/arma3-performance-of-ryzen.1672767/page-23 more specs on the next page

https://www.computerbase.de/forum/threads/arma3-performance-of-ryzen.1672767/page-24

[empty goblet]

@inland dew more frames and on Intel, this is for you 😁

[tawdry gazelle]

Here are my YAAB test results with no mods loaded, captured by Nvidia FrameView:```
Avg FPS 90th % 95th % 99th % RenderPresentLatency (ms)
Default 62.172 48.512 45.262 40.29 2.202
2021.5 68.658 55.056 51.535 45.554 2.415
Boost 10.43% 13.49% 13.86% 13.07% 9.67%

Significant, right? With heavily mods loaded and post-process effects provided by ReShade, it also has stable 5% performance boosts. 🥳

[fickle geyser]

Default as in arma default allocator or the earlier CMA?

[tawdry gazelle]

arma default.

Compared with CMA, it also has 5% performance gain. (just not shown here.)

But it's just YAAB in singleplayer. I haven't test server side and multiplayer performance... I hope someone can help testing this.

really interesting finding...

[empty goblet]

i do wonder if you compile just AVX1 one, what will the performance gain would be (e.g. if it's just fixes in the allocator)

[tawdry gazelle]

yep, I will give a try tomorrow.

[empty goblet]

and ofc compare the result of those vs system allocator

[restive pilot]

Btw where did you get the CMA sources?

[empty goblet]

#perf_prof_branch message

it's part of the download, both sources and libs

[tawdry gazelle]

Dwarden sent me

[restive pilot]

I get a 5% boost on my potato laptop

[restive pilot]

does it even use AVX2?

I searched for it but all I saw was AVX
I enabled it anyway but I don't think it'll use them

also why is mine much smaller than @tawdry gazelle's? (and all other mallocs in dlls dir)
https://github.com/Leopard20/oneTBB

[restive pilot]

looks like I built it wrong

[tawdry gazelle]

I guess that should be expected.

[tawdry gazelle]

I have enabled some performance optimization compiler parameters, it may copy assembley codes to its caller, so it could be larger than default.

(Why cant I post pictures!!! 😭 )

[restive pilot]

it was because I forgot to compile with /MT (it was MD)

[tawdry gazelle]

Ah, that should be it. 😂

I've compiled more version as dwarden suggested, here's the binary link. (Source codes still in my repo)

[tawdry gazelle]

https://1drv.ms/u/s!AmQv-K9BLIUolWF61t7GrOU5rxZL?e=CfAShV

[restive pilot]

yeah I tried yours first

it gives better boost than the one I compiled ...

gonna try again with /MT

[tawdry gazelle]

what MSVC and windows SDK do you use? mine is at VS2022, MSVC v143, WinSDK 22000

I wonder if compiler brings those performance boost.

[restive pilot]

v142 (VS2019)

[tawdry gazelle]

As far as I know, 5% is a massive boost for game performance, even overcloking could hardly give that much. 🤔

[restive pilot]

yeah now it's on par with yours

[restive pilot]

actually it's 8%-10%

it was the MT one that was 5%

so yeah, pretty cool

[tawdry gazelle]

Yes, this replicates my results. 🥳

:

[restive pilot]

@tawdry gazelle btw if you unset AVX does it make any difference for you?

it compiles the exact same lib for me

[tawdry gazelle]

I believe VS should use AVX for 64bit as default, SSE for 32bit as default?

[restive pilot]

dunno

[tawdry gazelle]

windows dont have something to directly see the dll's instruction set...

I'm considering decompiling it and see if it uses any new instruction...

[restive pilot]

I just compared their sizes...and they were exactly the same

[tawdry gazelle]

yes, mine too. I dont think they should...

[restive pilot]

actually nvm

it doesn't even care

#if (defined(__AVX__) || (_MSC_VER >= 1600 && defined(_M_X64))) && !defined(__sun)

it enables it automatically if _M_X64(x64 bit) is set and _MSC_VER >= 1600

[tawdry gazelle]

Yes, that's what I said, 64bit default uses AVX. But what about AVX2 and AVX512?

[restive pilot]

I didn't find any reference to AVX2, let alone AVX512

[tawdry gazelle]

me either...

the easiest way is to get a PC with AVX support but not AVX2 support to test.

if the AVX2 is true avx2, it would crash as cpu dont support it, but avx one would run. 🤣

[restive pilot]

looks like it doesn't use any vectorization at all (except maybe some auto vectorizations?)

[tawdry gazelle]

Found a MSVC document about that: https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?redirectedfrom=MSDN&view=msvc-170

[tawdry gazelle]

I've wrote a program to detect if compiler would use those instruction set marcos.

It shows that in MSVC, x64 would use IA32 (no extension instruction set at all) by default, and x86 would use SSE2 by default.

So the compiler paramter should taking effects, we just dont know if TBB actually use them...

The code is simple:```
int main()
{
#ifdef AVX512BW
std::cout << "AVX512BW\n";
#endif

#ifdef AVX512CD
std::cout << "AVX512CD\n";
#endif

#ifdef AVX512DQ
std::cout << "AVX512DQ\n";
#endif

#ifdef AVX512F
std::cout << "AVX512F\n";
#endif

#ifdef AVX512VL
std::cout << "AVX512VL\n";
#endif

#ifdef AVX2
std::cout << "AVX2\n";
#endif

#ifdef AVX
std::cout << "AVX\n";
#endif

#if _M_IX86_FP == 2
std::cout << "_M_IX86_FP == 2, SSE2 Probably\n";
#endif

#if _M_IX86_FP == 1
std::cout << "_M_IX86_FP == 1, SSE Probably\n";
#endif

#if _M_IX86_FP == 0
std::cout << "_M_IX86_FP == 0, IA32 Probably\n";
#endif
}

[restive pilot]

based on a brief look I had it doesn't

it does use vectorizations (SSE and AVX) in other parts, but not for the malloc

[tawdry gazelle]

okay... that's a pity...😅 so we shouldn't have noticeable performance differences on different instruction set, major performance impact should be caused by different TBB version...

By the way I'm also interested in microsoft's mimalloc https://github.com/microsoft/mimalloc, it prefers small pages instead of large pages, I wonder if I may migrate that one...

[restive pilot]

looks interesting

[restive pilot]

I tried this too. Gives roughly 6% boost. Not sure if it uses large pages tho

[tawdry gazelle]

could you share the source code? I haven't got time to do that...

[restive pilot]

also it seems that the new TBB has about 5% more memory usage as well (needs more testing tho)
after the benchmark had ended, new TBB was using ~4.5 GB memory for me, while the default TBB and mi were ~4.2

[restive pilot]

I just threw the CMA sources in there and made some adjustmets to use mi functions instead of TBB ones 🤣

[tawdry gazelle]

Ah... 🤣

As I run a more carefully performance test, I've come to a confusing conclusion...

https://s2.loli.net/2021/12/22/kF4LvuRfAzns1Vm.png

There seems no more gain than CMA this time...

[restive pilot]

what about this one?

your PC is better than mine, so maybe you can tell more accurately
P.S: it's broken.

[tawdry gazelle]

Here's another benchmark result of a low end laptop... It gets 10% performance gain.

https://s2.loli.net/2021/12/22/klzrRKUQn6ecWmo.png

Ah, I know why! All the dll I tested today was compiled by myself, so all of them have the similar performance gain!

So the performance gain could be brought by the compiler, not the TBB code...

[tawdry gazelle]

Could you please post the cma_api.cpp file, so I may double check if huge pages is activated? 😁

[restive pilot]

actually I just noticed it has mem leak 🤣

let me see if I can fix it

[tawdry gazelle]

🤣

Alias        Avg FPS    Min FPS    Max FPS    90th %    95th %    99th %    Time (ms)
Default        62.575    25.883    106.097    50.265    47.4    42.726    139.136
Gain        0.00%    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%
mimalloc    62.019    23.332    108.136    49.133    45.919    41.86    138.983
Gain        -0.89%    -9.86%    1.92%    -2.25%    -3.12%    -2.03%    0.11%

mimalloc seems not that good, at least on my PC.

[empty goblet]

large (huge) pages shall show benefit cause you don't waste your l2/l3 cache on memory tables

now that might be where W10/11 system allocator shows some progress, as since certain build of W10 MS enabled LP as always on (system wide)

the kernel evolution brings more too , so when doing system allocator tests, always include full OS build

[inland dew]

On latest Win 10 and Win 11 build it's not enabled

[empty goblet]

it's enabled for years (i just don't remember the build since they did it)

[inland dew]

It's not, otherwise enabling it wouldn't have given me any FPS boost

FPS would have been as high as it is, without having to do "lock pages in memory" for my username

[silk pewter]

you may have installed your Windows before LP was enabled by default?

[inland dew]

No

[empty goblet]

unless MS broke something and reverted the change

[verbal mist]

I can confirm your findings on a 12900K :
your (tbbmalloc_2021_AVX2_hugePages.dll) against CMA AVX1: 124 vs. 117 FPS EDIT: NOW 130.1 FPS new record😀

• a friend in my ARMA Community tested on Ryzen 3 and Gen 10 Intel and came to similar results.
  Will try to fix my RAM settings during the holidays and post my findings again.

THX for your good work man!

[empty goblet]

weird, i swear i seen somewhere some year ago (2019 or earlier), that MS released W10 update which switched the LP lock to enabled by default including home edition https://docs.microsoft.com/en-us/windows/win32/memory/large-page-support

[fickle geyser]

It always needed special permission on user and running as admin.

At least on non server versions.

[empty goblet]

one of reasons they changed it because W10HE had no gpedit, another was that the kernel support was ready (since windows 8.1)

[fickle geyser]

That's why it's funny seeing these performance guides recommending enabling huge pages in launcher.

As it does nothing for 99% of the player base.

[empty goblet]

ye seems like MS reverted that change and the blog/docs about it are goner

behold OS as service where what was ain't what is

[fickle geyser]

I don't recall anything about this so... are you sure it wasn't only in preview versions or smth?

[inland dew]

@tawdry gazelle @empty goblet tested CMA AVX (2016) vs. (2021) Intel 2021 5 AVX and FPS is exactly same.

Now testing default Intel vs. Intel 2021 5 AVX (no lock pages in memory in Windows)

[inland dew]

@tawdry gazelle @empty goblet tested default Intel vs. Intel 2021 5 AVC (no lock pages in memory in Windows) and FPS is exactly the same

So, to conclude, Intel 2021 5 AVX gives exactly same FPS as default Intel.
And Intel 2021 5 AVX (with lock pages in memory in Windows) gives exactly same FPS boost as CMA AVX (with lock pages in memory in Windows).

[kindred radish]

So iam still working on the stray entities problem. And i managed to write a hacky script to delete these entities.
Just had a period with 15.000+ of these and server was running at ~15 FPS with 30 players.
Managed to delete ~12.000 and the server FPS went up to ~50 FPS again even after almost 3 hours runtime.
Big downside is you can only delete these if the clientOwner is still online

#logEntities https://gyazo.com/b25e35f833fe6185df9b719340bf8e8c
Perf Metrics https://gyazo.com/685d5d1e33d18ddcd4bc0ca3c086346f

[empty goblet]

erm, if the owner disconnects the entities shall transfer to server locality (also always erase first in the local owner locality not remote (so what's owned by server, on server, what's owned by client, on that client))

and if you have 12000 rogue entities you don't need , i think you have serious problem of cleaning stuff over the time ... why wait for the bloat treshold

get rid of stuff far away from players (out of sight out of memory), too old based on timers (FIFO) and then limit thresholds cause always some genius figure how spawn too much at once

[kindred radish]

The thing is iam pretty sure its also the cause for other servers bad perf. Got a entities log from another server with just 4 players and 30 minutes runtime and 3 players caused around 90 each of these with the exact same pattern as on our server. Iam gonna get some more logs from other servers after new year and iam pretty sure it will be the same there.

> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:3, Pos: [0.000000][0.000000][0.000000], N:Supply30
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:4, Pos: [0.000000][0.000000][0.000000], N:Supply80
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:5, Pos: [0.000000][0.000000][0.000000], N:Supply0
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:6, Pos: [0.000000][0.000000][0.000000], N:Supply20
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:7, Pos: [0.000000][0.000000][0.000000], N:Supply40
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:8, Pos: [0.000000][0.000000][0.000000], N:Supply40
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:9, Pos: [0.000000][0.000000][0.000000], N:Supply0
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:10, Pos: [0.000000][0.000000][0.000000], N:B_Parachute
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:11, Pos: [0.000000][0.000000][0.000000], N:Supply40
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:12, Pos: [0.000000][0.000000][0.000000], N:Supply60
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:13, Pos: [0.000000][0.000000][0.000000], N:Supply20
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:14, Pos: [0.000000][0.000000][0.000000], N:Supply60
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:15, Pos: [0.000000][0.000000][0.000000], N:Supply40
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:16, Pos: [0.000000][0.000000][0.000000], N:Supply100
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:17, Pos: [0.000000][0.000000][0.000000], N:Supply40
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:18, Pos: [0.000000][0.000000][0.000000], N:Supply0
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:19, Pos: [0.000000][0.000000][0.000000], N:B_Parachute
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:20, Pos: [0.000000][0.000000][0.000000], N:Supply20
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:21, Pos: [0.000000][0.000000][0.000000], N:Supply60
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:22, Pos: [0.000000][0.000000][0.000000], N:Supply120
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:23, Pos: [0.000000][0.000000][0.000000], N:CUP_B_GER_Pack_Flecktarn
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:24, Pos: [0.000000][0.000000][0.000000], N:Supply40
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:25, Pos: [0.000000][0.000000][0.000000], N:Supply120
> Loc:0, ForDel:0, Dstr:0, DmgDstr:0, DmgDead:0, Sim:1, Inv: 0, AnyPl:0, Out: 0, NetID:8:26, Pos: [0.000000][0.000000][0.000000], N:CUP_B_US_IIID_OCP

Full #logEntities for this other server -> https://drive.google.com/file/d/1WMxA_szHwrva4LJkSq7CAywPFzKuLEUs/view?usp=sharing

Full #logEntities for our server before i deleted -> https://drive.google.com/file/d/1mUN4meDhHFcepSfaKajnNNg97_D-PwZm/view?usp=sharing
Full #logEntities for our server after i deleted -> https://drive.google.com/file/d/1rjQFGGN5ZkcK8_6_0_HCMg64qaHEUJxq/view?usp=sharing

That's what i used to delete the entities (executed on the server)
https://sqfbin.com/zabacimusayugididile

[whole cloud]

Do you run windows server?
Can you get me a vTune capture of when that server fps is really bad with the many entities? So I can see why they pull performance down so much

A couple entities that don't do anything at the edge of the map shouldn't totally kill performance, maybe i can do something about that

But i need (optimally) a vTune hotspots capture to see what's wrong

[silk pewter]

oh btw I remember a text saying somewhere "don't place stuff @ [0,0,0]"

I assumed it was because many many many scripts use this position, but is there a performance/engine reason to this?

[whole cloud]

There is a RPT warning for when you move stuff to 0,0,0 in diag binary.
That's because when that happens, it's usually a mistake.
Like if some script fails and passed nil or something like that, it may silently fall back to 0,0,0, and that warning is there for you to know that something may be wrong

Would make sense if that warning would print script calltrace.... I think it doesn't. Someone put that on my to-do list or I'll forget

[silk pewter]

diag only, not retail?

[whole cloud]

Dunno

I think diag

[kindred radish]

We run a windows server yea.
VTune Profiler ?

https://www.intel.com/content/www/us/en/develop/documentation/vtune-help/top/introduction/whats-new.html

[whole cloud]

Yes

Hotspots capture, software, attach to process paused, wait a few seconds, start recording, wait about 2 minutes, stop recording.

[kindred radish]

Will do 🙂

[crisp wagon]

iirc wiki on it has messages below it suggesting you should spawn objects at 0,0,0 first before teleporting it over cuz performance wise it was faster in execution.

[whole cloud]

I'll only look at it in a couple weeks though

[kindred radish]

Yea no hury, we have this problem since ages, so a few weeks more don't matter.

[whole cloud]

But would he good to have it ready then, and maybe repro too so I can dive right into it

[kindred radish]

Does this look right ? (We have a AMD processor)

https://gyazo.com/7c255eb85c3786099588ddc3bdb71e0c
https://gyazo.com/ff6f233097d3bdead2aacdbaa0d792b6

https://gyazo.com/164abfc3227e3e7853ec38950da0402b

[whole cloud]

In first Screenshot, midle Panel, you can select native only (not managed) that gets rid of that warning on second shot.
Besides that all good, also that CPU utilization is pretty good..........
Oof, i might know where the bottleneck is

[kindred radish]

It was just a test run, right now the server is running fine.
Low amount of stray entities (< 500) 20 players online and server running around 65 - 90 FPS (90 FPS is the limit)

Will do it again if we have a high amount of stray entities and low server FPS

[whole cloud]

If all threads are loaded down when the fps is that low, it's the entity error calculation.
I'd have to do some big brain stuff to get that much better

[kindred radish]

Guess it would be the best to send you the whole result files then ?

https://gyazo.com/352cbb36623274c13afdbb4fab6fab9d

[crisp wagon]

I'd have to do some big brain stuff to get that much better :BlobFearSweat:

We're rooting for you dedmen.

[tawdry gazelle]

Yes, this shall be true, the performance gain should be caused by lock pages.

[tawdry gazelle]

I assume you use the CMA dll in my zip, not dwarden provided one? Then this shall be true. All dlls in my zip should have the same performance gain. The dlls compiled by the same version of MSVC should all have the same performance.

[tawdry gazelle]

By the way, are you reading the fps from the YAAB itself? The author says the FPS in YAAB can not be accurate, you better use external tools (FrameView for example) to measure the fps.

[whole cloud]

Yes as zip/7z archive.
Shouldn't hurt to have multiple.
But there might be problems that make them unusable, I'll find that out when I try to open them.
Also note down exactly which build you were running when you recorded it
And if you happen to have a crash .mdmp from that exact same Arma build, that makes it easier for me

[kindred radish]

VTune Version: 2022.0.0
Arma Build: 148592

[tawdry gazelle]

@empty gobletSo we come to an conclusion about the memory allocator. The performance gain was not caused by different tbbmalloc version, it was caused by the compiler or compiling parameters. All those dlls compiled by me has the same performance gain over both old CMA and default. When large pages is not enabled, all my dlls have no performance gain than default, that means default might not support large pages at all. Besides, the huge pages might give more performance gain on low-end pc (10%+) than high-end (6%+), I assume on high-end pc, memory allocation should not be the bottleneck. 🤯

[whole cloud]

Whether OS even gives you huge pages can also depend on how long PC has been running
But that's probably irrelevant for your testing

[empty goblet]

ye huge pages need all the bells, enabled in OS (restart of OS needed), lotsa of memory (i can't help you there, i don't know what's less than 32GB for last 15 years or so) and ofc allocator/its interface capable to use it

the compiler's switches make sense, that's what they for

the performance effect for weaker CPUs could be linked to the L2(L3) cache as those are usually smaller so freeing it helps more than at high end

but generally 3 to 10 % free performance is better than nothing

and for the newer(newest) allocators ... less old bugs (plus some new ones 🤣 )

[autumn timber]

it was caused by the compiler or compiling parameters
Tbh that sounds awfully suspicious to me.
What about you just tell each other what compiler you're using? Any latest VS version is free nowadays, so it's not like you can't afford to test the other guy's build...

[tawdry gazelle]

I am aware of that, so I reboot my newly installed OS each time I run a new testing.

[tawdry gazelle]

Agreed, even upgrade from 10900K to 11900K wont give that much... 🤣

I've installed a new cleanly OS to do a compare. The final result is 6%-8% performance gain on the new OS. When I switched back to a daily use OS (with many background processes), performance gain is at about 10%... So I may assume newly compiled version may do better allocation on a more fragmented memory?

[tawdry gazelle]

I'm actually using MSVC v143 (VS2022) with the following parameters enabled. <RuntimeLibrary>MultiThreaded</RuntimeLibrary> <InlineFunctionExpansion>AnySuitable</InlineFunctionExpansion> <IntrinsicFunctions>true</IntrinsicFunctions> <FavorSizeOrSpeed>Speed</FavorSizeOrSpeed> <EnableEnhancedInstructionSet>AdvancedVectorExtensions2</EnableEnhancedInstructionSet>

Please notice that different options on EnableEnhancedInstructionSet does not gives any difference, and the most noticeable different is caused by <RuntimeLibrary>MultiThreaded</RuntimeLibrary> (default is MultiThreadedDll), may be 3% more on my pc.

Leopard20 should have replicated the same performance gain with the same parameters, but a different compiler, MSVC v142 (VS2019).

I'm still wondering if someone know how to use g++ for a test. (I'm interested in the /O5 optimization, currently MSVC's default max is /O2.)

[inland dew]

I do 10 runs per dll, write down min and avg of each run, for eacg dll.
Then I do the average of min FPS of all runs and the average of avg, due to how YAAB works.
It's vrry accurate in the end

[inland dew]

No, FPS with Intel 2021 5 was same as with CMA from 2016, not with the one you've provided

So lock pages in memory is the only thing that boosts FPS

No matter the version/age of Intel mallocs

[tawdry gazelle]

what about 90th% and 99th% fps? are they improved?

min and max does not reflect the actual performance at all. as it's one time value, not an average. See: https://images.nvidia.cn/content/geforce/technologies/frameview/frameview-user-guide-1-2-web.pdf and search 99th.

[tawdry gazelle]

Yes, I agreed.

[inland dew]

@tawdry gazelle large pages in Arma launcher has no effect on FPS aka isn't working

[tawdry gazelle]

Yes, I'm writing a new Arma CMA implement, to automatically ultilize large pages once the OS has it enabled.

Arma launcher's huge page is totally useless.

No matter what, you always need to manually set the lock page privilege.

[inland dew]

On Intel 2021 5?

[tawdry gazelle]

Yes. But it should not bring more performance boost than 2016 CMA.

[inland dew]

Most prpbably lock pages can't work only for a specific process, but only for the whole system/user and that's probably why -largePages in Arma launcher doesn't bring anything

[inland dew]

Would be great - no need for cma.ini then

[tawdry gazelle]

As I can tell from the 2016 CMA code, it does not implement enableLargePage function, so Launcher's -hugePages does not take effects. It reads from the cma.ini, so you need to configure it there.

As for a certain process, the process itself need to acquire a special privilege token from the OS, which requires the program to run as an Administrator account and assgined lock page privilege at the same time.

arma_x64.exe wont lift their privilege to Administrator by default. So even the account is an Administrator, you need to configure it in Properties of the exe file to force it run as Admin. Once it runs as Admin, a UAC window would pop up.

[inland dew]

But I don't use CMA to play - only for benchmarking.
Since after many hours of uninterrupted gameplay, textures/objects start to flicker heavily and unplayable FPS.
And if you continue playing, the game will simply crash.
And that's with 8 GB vRAM, 32 GB RAM and unlilited paging file.

People with less vRAM, only 8 or 16 GB RAM and manually set paging file to only few GB or just slightly more, will experience textures/objects flickering and unplayable FPS much sooner.

With default Intel malloc it starts to happen much later

[tawdry gazelle]

I haven't notice that before... I will look into about it. 🤔

[inland dew]

To notice it, you have play something like Warlords for many many hours

[tawdry gazelle]

I can use script to change my camera among players from time to time and record when it crashed. 😎

[mellow lark]

Anything with a lot of garbage xd, like Antistasi, Warlords, VIndicta, ...

[empty goblet]

low 1% fps shall be concern next to avg fps and frametimes ... if you have too many low spikes/ hitchup then it means stuttering

too high frametime is problem too e.g. i was showcasing someone, i can run Arma 3 DX11 via dxvlk into Vulkan ... the fps in Vulkan end is higher than DX11 but the frametime is 2-3 times higher

[empty goblet]

could be some bugs within old/obsolete code ... there were fixes in the allocators since (both tbbv4 (2020u3 is last) and ofc then intelone 2021.5 last)

[restive pilot]

But how can FPS be higher if frame time is also high?

[empty goblet]

processing time vs render and buffers (vulkan pipeline is way better than DX11 (threaded, better caching, less operations)

the FPS wasn't that much higher it was like 3 to 10% max but it wasn't lower and it was on cost of the frametimes

now mind me, that was on Windows, on Linux the loss of frametimes was way lower

also , the Windows driver for DX11 is way slower than Vulkan driver (raw performance)

toVulkan translation layer is mature, i checked the visual output (lossless video compare of same camera scenes) and pixel wise it's 1:1 so no visual degradation

[inland dew]

Then we need Intel 2021 5 AVX to be accepted by BE, so I can test it in Warlords, to see if there is still flickering of textures and unplayable FPS after many hours of uninterrupted session

[tawdry gazelle]

I guess that's impossible... 🤣

[inland dew]

Well, CMA was whitelisted for BattlEye, while not having anything to do with Bohemia, so...

[mellow lark]

Did you also test this on AMD?

When I tested DXVK, it provided substantial gains

[empty goblet]

it's been long time before recent major gains ... but i bet the frametime is still worse than direct

[tawdry gazelle]

I haven't notice any gains using DXVK... how did you managed to do that?

[kindred radish]

Found another weird thing with the help of #logentities
Looks like the "Can't change owner from 0 to 2" error happens if a client accesses the inventory of a container spawned by the server and then something is trying to change the ownership of the supplyXXX entities inside (backpack,vest,uniform).
"AeoG_B_CargoNet_01_ammo_F" is a crate we only use for a "heli loot drop & crash script" 🤔

 3:34:51 Server: Object info 11:11 not found.
 3:34:51 Can't change owner from 0 to 2
 3:34:51 Server: Object info 11:12 not found.
 3:34:51 Can't change owner from 0 to 2
 3:34:51 Server: Object info 11:13 not found.

> Loc: 1, ForDel: 1, Dstr: 0, DmgDstr: 0, DmgDead: 0, Sim: 1, Inv: 0, AnyPl: 0, Out: 0, NetID: 11:11, Pos: [0.000000][0.000000][0.000000], N: Supply120, SupplyInfo: (W: 0, M: 0, I: 0, B: 0), OwnerN: AeoG_B_CargoNet_01_ammo_F, OwnerSupplyInfo: (W: 0, M: 0, I: 4, B: 2)
> Loc: 1, ForDel: 1, Dstr: 0, DmgDstr: 0, DmgDead: 0, Sim: 1, Inv: 0, AnyPl: 0, Out: 0, NetID: 11:12, Pos: [0.000000][0.000000][0.000000], N: Supply140, SupplyInfo: (W: 0, M: 0, I: 0, B: 0), OwnerN: AeoG_B_CargoNet_01_ammo_F, OwnerSupplyInfo: (W: 0, M: 0, I: 4, B: 2)
> Loc: 1, ForDel: 1, Dstr: 0, DmgDstr: 0, DmgDead: 0, Sim: 1, Inv: 0, AnyPl: 0, Out: 0, NetID: 11:13, Pos: [0.000000][0.000000][0.000000], N: Supply40, SupplyInfo: (W: 0, M: 0, I: 0, B: 0), OwnerN: AeoG_B_CargoNet_01_ammo_F, OwnerSupplyInfo: (W: 0, M: 0, I: 4, B: 2)

[tawdry gazelle]

Hi Dwarden, I got some question about the your CMA implement. I see MemFlushCache(size_t size) directly returns size without flushing any cache, would this be an issue? For example, game would get wrong info about those flushed memory cache at a long run, and may cause the flikering issue said by Groove_C? I would like to know in what situation would arma call this API? Would it be a better solution if I return 0 instead of size?

[tawdry gazelle]

And I have noticed most malloc (tbbmalloc, mi-malloc) dont have API to get total commited/reserved memory, which is required by CMA API MemTotalCommitted and MemTotalReserved. Though we can implement them by ourselves, I'm still not sure if I might miss something... When and how are these two API actually used in Arma? Can they be ommited?

[empty goblet]

it's not my implementation, that's the original CMA as it was released years ago

[tawdry gazelle]

Ah that's sad... 🥲

[inland dew]

if we could have Intel 2021 5 as default malloc of the game...
so it probably would eliminate textures flickering and unplayable FPS that the default Intel causes now, after many hours of uninterrupted gameplay, although it gives you some more time before it starts to happen vs. CMA.
and also inside this Intel 2021 5 everything configured so, that you only need to "lock pages in memory" in Windows, with PC restart, and start Arma as admin, it wouldn't required any .ini, like CMA.
And it would give expected FPS boost and no need to look for places where to get outdated CMA malloc.

[tawdry gazelle]

Yes, that's the case. But making a change on a published product is not that easy, especially it goes to some underlying codes like a memory allocator...

[tawdry gazelle]

I've managed to implement Microsoft's mi-malloc (https://github.com/microsoft/mimalloc), and got a huuuuugely startup time increases (15 minutes passed and it's still a black screen)... Though I expect performance degrade, but not like this one. Any senior programmers here? I really need help... 😭

[restive pilot]

after I finally "fixed" mimalloc it was giving me out of memory errors 🤣 I just gave up...
I really don't think any "magic" can be done with mallocs

[tawdry gazelle]

Good news is, I've just done with mimalloc and am about to post it 🤣

I think mimalloc is more documented than tbbmalloc, I've managed to utilize mimalloc's reserved large pages feature, though havent tested it yet.

https://1drv.ms/u/s!AmQv-K9BLIUolWS0f_N96bbtwJa8?e=OfQ9QY

Here's the file, both tbbmalloc and mimalloc, if you would like to test it.

Source code is here: https://github.com/GoldJohnKing/mimalloc/tree/Arma-3-v2.0.3

[restive pilot]

does it even work?

[restive pilot]

or is it this one?

[tawdry gazelle]

yes

[tawdry gazelle]

ofc it works 🤣

[restive pilot]

you said it didn't work

[tawdry gazelle]

yesterday and today is totally different for a programmer~ 😎

[restive pilot]

the source code is pretty much identical to what I had tho

[tawdry gazelle]

it just works for no more than 1 hour

[restive pilot]

it didn't work for me

[restive pilot]

yeah I don't think it's actually working. are you sure it's not falling back to default?

[tawdry gazelle]

my rpt shows the correct dll file name, and I am truly gain performance than the default one.

[restive pilot]

ok. what about tbb?

[tawdry gazelle]

And I've tried putting OutputDebugString in those CMA API, it works.

mimalloc seems to give another ~= 3% performance gain than tbb...

[tawdry gazelle]

tbb works too on my machine...

[restive pilot]

I know. I meant the performance

[tawdry gazelle]

             Avg FPS   90th %    95th %    99th %    RenderPresentLatency (ms)
tbbmalloc    65.207    52.586    49.441    45.021    2.512
mimalloc     68.146    54.858    51.613    45.695    2.694
Gain         4.51%     4.32%     4.39%     1.50%    -7.25%

yeah! another 3% performance gain lol I merry you a Christmas~ 🤪

[restive pilot]

it's worse than tbb (1-2%) for me

[tawdry gazelle]

how much available ram do you have? I utilized the ReservedHugePages API, it can lock large pages at startup to prevent allocation at runtime.

[restive pilot]

available was around 9

total 16

[tawdry gazelle]

if your system has run for a long time and memory becomes fragmented, it may decrease your performance. See this: https://github.com/GoldJohnKing/mimalloc/blob/Arma-3-v2.0.3/src/cma/cma_utils.cpp

you may want to eliminated those codes... here: https://github.com/GoldJohnKing/mimalloc/blob/2dcb4b03417215c47d881bce0fa18275981e4e8c/src/init.c#L525

[verbal mist]

Is the Mimalloc file available?

[restive pilot]

yes
#perf_prof_branch message

[verbal mist]

THX
How about a new record for Christmas 🙂

[hearty haven]

@inland dew you can contact https://www.battleye.com/contact/ about malloc dll's whitelist for arma 3
provide open code if possible(i linked armaholic archive where code was included)
, i just wrote back then about avx2 malloc mentioned there and they seems allowed it
dunno it was my request or someone else
https://i.imgur.com/iuPp8hO.png

@tawdry gazelle also

[tawdry gazelle]

Got that! Thanks~ Will do.

[inland dew]

not before you implement lock pages in memory check, when/if active in windows, then use it (Arma launched as admin), without having to have .ini to make it work.
And submit them only AVX version. no AVX2/FMA3 or AVX512

I will happily abandon default Intel, since it gives lower FPS and will abadon CMA, since textures flickering and very low FPS, same as default Intel, but much sooner

[tawdry gazelle]

yep. And the alloctors don't utilize AVX/AVX2 instruction at all... Though I send the AVX version, there should be no difference at all. 🤣

[tawdry gazelle]

I will concentrate more about the flickering issue recently and see if it reproduces with my memalloc.

[inland dew]

maybe it's not the case anymore, with Intel 2021 5. but impossible to know/check, since it can't be used online, without BattlEye whitelisting it

[tawdry gazelle]

My server dont use BE, and I have many servers with many different mods and modes and with many players, I am capable of checking it~

[inland dew]

before submitting it to BE, make sure lock pages in memory check and use of it works, without .ini

[hearty haven]

@inland dew could you give intel 2021 avx's link?

[inland dew]

should give more FPS. if not, then lock pages in memory check and its utilization doesn't work

@tawdry gazelle #perf_prof_branch message

[tawdry gazelle]

That's was old and not perfect optimized. the new link contains the final version of tbbmalloc v2021.5.0 and mimalloc v2.0.3, you may test and find which suits you.

Link here: https://1drv.ms/u/s!AmQv-K9BLIUolWS0f_N96bbtwJa8?e=OfQ9QY

[night mulch]

thanks for your work!! gonna test them now

[tawdry gazelle]

Hi, I've finally come to a final version of the memory allocators, posted on both BI's forum and Github. You should download it and replace your previous downloaded version.

https://forums.bohemia.net/forums/topic/237134-golds-cma-the-first-ever-port-of-microsofts-mimalloc-and-a-brand-new-version-of-intels-tbbmalloc/

If there are any further updates, I will post them directly on Github.

[inland dew]

coold you, please, make also a SSE version and post it for me to test vs. AVX?

[restive pilot]

I doubt it uses vectorization

neither does oneTBB

not sure about the old intel TBB. didn't look at its code

[tawdry gazelle]

Do you need a 32 bit SSE version? SSE is 32 bit only instruction set. MSVC can not compile a 64 bit program to SSE.

[inland dew]

no 32 bit for me. let it be then

[inland dew]

so you managed to exclude the dependency on .ini for it to use lock pages in memory, when enabled in Windows?
so there is only the dll?

[restive pilot]

it doesn't use the ini

it just enables it by default, and there's no way to disable it

[inland dew]

nice

[restive pilot]

also @verbal mist tested it yesterday and said it was worse than TBB version

[inland dew]

now need only to submit it to BattlEye, so we can finally thoroughly test in online, in long game sessions, with a lot of garbage, players, AI , armor/air and destruction

[restive pilot]

120 vs 130 FPS

[inland dew]

it's not - it's same

tested it and exactly same FPS

[restive pilot]

I mean the xtbb one, not default

is that what you mean too?

[inland dew]

Intel 2021 5, without lock pages in memory, performs like default Intel and with lock pages in memory, like CMA

[tawdry gazelle]

Yes, there shouldn't be noticeable differences.

[tawdry gazelle]

Do not give the privilege then huge page will be disabled. 🤣

What about the mimalloc? Does it gives similar performance boost as legacy CMA and 2021.5?

[inland dew]

when do you think you will submit it to BattlEye?

[tawdry gazelle]

Yes, there's only a dll, without need to care about other things.

[tawdry gazelle]

I've done it already, but still not get a reply yet.

[inland dew]

ok. let us know, once there is a response from them

hope textures flickering and unplayable FPS will be things of the past with Intel 2021 5

[tawdry gazelle]

though I recommend mimalloc more than tbb... 🤣

[verbal mist]

Yeah will test again in a few days - I did the recommended settings in BIOS to RAM of all the settings that are not being covered by MSI Canon Ball. After that, lots of stability issues.
Both MALLOC atm at 119-121 instead of 130.

[tawdry gazelle]

wired...

[verbal mist]

yeah after a bios reset to auto to those 3-4 RAM values and the old config can test again.

I

[verbal mist]

Since my PC is unstable here some findings of a great member of my Community :
I5-11600K: 2021_avx 96,5fps, Mimalloc 97,4fps, TBBmalloc 97,8fps.

[inland dew]

it's same FPS. 1-2 FPS variation is within margin of errors in Arma

[empty goblet]

and system allocator results?

[verbal mist]

by system allocator results you mean no Malloc active - by windows?

[empty goblet]

yes, no allocator dll, wrong allocator (failing to load, no interface etc.) or forced via -malloc=system

you can verify if afterward by looking at the .rpt on the allocator report line

[verbal mist]

yeah within margin of error

[verbal mist]

Final results:
I5-11600K: 2021_avx 96,5fps, Mimalloc 97,4fps, TBBmalloc 97,8fps. Intel TBB Default 85,8fps, Windows system allocator 80,7fps.
All tests with cma.ini with uselargepages=1 in the arma folder.

[empty goblet]

what version of OS?

was -hugepages used as the A3 startup param ?

[inland dew]

it never gives anything. no FPS boost, nothing, never

[empty goblet]

not even on the system allocator ?

[inland dew]

no

[empty goblet]

weird ... seems like it got broken at some point then

[inland dew]

as if it simply doesn't work or do anything

[restive pilot]

it does work

[inland dew]

then it's been like this for years

[empty goblet]

it's supposed to do something, if the allocator interface and function(s) operate with it

[restive pilot]

it calls EnableHugePages() of the malloc

[inland dew]

I've reported it many years ago and many times

[empty goblet]

ofc CMA enables is it via ini. so it doesn't matter

[inland dew]

it doesn't. otherwise -hugePages would have helped to boost FPS when using CMA or Intel 2021 5, without having to use the .ini

[restive pilot]

they don't implement that function

they use the ini as Dwarden said

I literally just tested it and it gets called when you launch with -hugePages

[empty goblet]

aka https://community.bistudio.com/wiki/Arma_3:_Custom_Memory_Allocator#EnableHugePages.28.29

[inland dew]

with mallocs part of Arma itself, -hugePages brings nothing, not even 1 or less than 1 FPS boost

no matter Intel or Windows

so it's just a meme

useless

for many years

[empty goblet]

the real question is if the 2017u5 (the last official TBBv4 in A3) was compiled with EnableHugePages() at all

[inland dew]

still, CMA and 2021 5 are, but -hugePages doesn't bring anything with them

[empty goblet]

let's leave this q to dedmen for january ...

[inland dew]

I told this many many times during many years, but no reaction to this not working...

[empty goblet]

TBBmalloc vs Intel TBB Default , which is what version ?

[whole cloud]

Arma has multiple internal allocators.
The pool allocator used for all of Scripting, doesn't respect large pages flag

But these don't allocate that often

[whole cloud]

Well if the allocators don't react to being told to use large pages then.... Well.

[inland dew]

same "response" as all the previous times...

[tawdry gazelle]

Yes, I am pretty sure default arma
allocator does not utilize huge pages at all, but it does gives 10% more fps then system allocator

[hearty haven]

if i remember correctly there was some allocator which was totally not compatible with battleye because it wasn't defragmenting or encrypting memory,
but gave best fps
(back in time when only x32 bit arma was available)

[empty goblet]

probably that was fred41's one ... afaik he never released the source (tho some claimed they got it) ... also he had some quite detailed debug variant on (de)allocation operations (again, no sources)

there were some private variants of allocators for servers, where the authors claimed 20 to 100% improvements but w/o sources / way to validate it ... nobody except them can tell if it was true

[tawdry gazelle]

I guess my malloc would give better performance on server too, but I dont know how to test it...

especially mimalloc's reserved huge pages feature, but it's strange and seems not behave the same as on client...

[whole cloud]

Unlock fps, place down 100 AI in vehicles.
Check fps counter. Bam easy benchmark :D

[rain moth]

All, move to that.. house 12 o clock

[tawdry gazelle]

Shall I make them fight? or just idle?

I cannot come up with an idea to reproduce a similar fight...🥲

[whole cloud]

Just idle will put enough load on the server, unit simulation, AI thinking, vehicle simulation,...

[rain moth]

Could do path traversal with ai agent?

[hushed seal]

Doesn't need to be similar enough if you do 10 runs each, either you have a significantly different result at the end or its so bunched up its not worth the hassle (to use the different malloc).

[tawdry gazelle]

Good idea, I will give a try.

[verbal mist]

Intel TBB 4 allocator - 64 bit (tbb4malloc_bi_x64) (Default)
tbbmalloc was in the last zip file with mimalloc

[agile cove]

Regarding the -hugepage option,
What is the difference between HugePages and LargePages?
The optional cma.ini file contains „UseLargePages = 1“ which is reported in cma.txt by „Using large pages“
If you choose cma.ini with „UseLargePages = 0“ then cma.txt reports „Using normal pages“
On the other hand the launcher option „-HugePages“ is reported only in .rpt file.
As an non programmer both sounds similar by the name, but I doubt they do the same.

[restive pilot]

they're the same

cma only uses the ini

it doesn't care about launcher option

the launcher option doesn't seem to work with vanilla mallocs either

[gritty wasp]

Would be nice to have line in RPT that will show if arma use(permissions granted) HugePages or not.

[whole cloud]

There is, or atleast its supposed to be

[feral harness]

but it shows Page size already:

PhysMem: 16 GiB, VirtMem : 8192 GiB, AvailPhys : 10 GiB, AvailVirt : 8192 GiB, AvailPage : 24 GiB, PageSize : 4.0 KiB/2.0 MiB/HasLockMemory

[whole cloud]

the last one is if it has the lock memory priviledge

[gritty wasp]

oh. my bad.

[verbal mist]

But the CMA.ini does always work, right ?
Just Did some testing with TBB4 Intel (default) and got up to 10 fps more with it.

[restive pilot]

I'm 100% sure it doesn't use it at all

CMA.ini does always work, right
only with cma malloc

[restive pilot]

because I did an exports dump using dumpbin and it didn't export EnableHugePages

no malloc uses it as of now (except maybe the newer ones that Gold John King made)

[verbal mist]

and what are your findings on the "tbbmalloc_2021_AVX2_hugePages.dll" ?

[restive pilot]

I made a malloc for myself that does, but it's just based on mimalloc

[restive pilot]

idk. where is it?

[verbal mist]

https://github.com/GoldJohnKing/oneTBB/releases
Just a simple repost

[restive pilot]

it does export it, yes:
https://github.com/GoldJohnKing/oneTBB/blob/eafa81713992261ae5879c493d91e0ad90c34772/src/tbbmalloc/cma/cma_api.h#L24

[verbal mist]

It also shows its effect in my last benchmark:
https://www.computerbase.de/forum/attachments/20211229150937_1-jpg.1164706/

[tawdry gazelle]

strange... on my pc mimalloc is always much better than tbb... 🤔

and AVX wont get any more performance on my machine, even worse than IA32……

[restive pilot]

as I told you it doesn't use AVX

just because you enable it doesn't mean it'll be used

[tawdry gazelle]

however mimalloc seems uses avx at some part, compiled dll is different in size.

I have totally dropped tbb.

[agile cove]

i found also also this malloc https://onedrive.live.com/?authkey=!AEjM3Zq26DBWTeY&cid=28852C41AFF82F64&id=28852C41AFF82F64!2784&parId=root&action=locate And I am getting very good results. I noticed the file is much bigger than any other mallocs so I am wondering what is different in this file.

[rain moth]

included bitcoin miner, trojan, virus?

[hushed seal]

Anything if it boosts performance

[golden wedge]

Does the profiling branch has anything to do with nearestBuilding? I have an script that works with profiling version but not with the stable. (the script gets the closest building to a position and disables all doors, using the bis_disabled_Door_X variable, and adds the options to open that door with a code inserted using a dialog) Did not see it on the pach notes.
EDIT: The scripts gets executed on module init, if it matters.

[whole cloud]

What does not work on stable mean?
Doesn't find building while it should?

[golden wedge]

is not blocking the doors

[whole cloud]

Profiling might do stuff with nearestBuilding yeah.
But your door blocking is a completely separate thing.

[golden wedge]

// _logic is the module
_building = nearestBuilding _logic;
//Lock all doors of the building
_class = configFile >> "CfgVehicles" >> typeOf _building;
_num = getNumber (_class >> "numberofdoors");
for "_i" from 1 to _num do {
    _var = format ["bis_disabled_Door_%1", _i];
    _anim = format ["Door_%1_rot", _i];
    _building animate[_anim, 0];
    _building setVariable[_var,1,true];
};

[whole cloud]

Is nearestBuilding functioning differently, returning different output?

[golden wedge]

Let me debug it on both versions to see it.

nearestBuilding in the profiling version is taking into account editor placed objects (something that I like :P) and the stable version not. will try do more testing.

[whole cloud]

Oh that's bad

[rain moth]

as stated on the wiki, you can use nearestObject instead to fetch the editor placed object

[restive pilot]

are you calling this in init?

[golden wedge]

Yep thats the fix im using with ["Building"] filter

[restive pilot]

The scripts gets executed on module init
yeah, I guess something's different about when it executes then

[rain moth]

https://community.bistudio.com/wiki/nearestBuilding

This command doesn't return any house or building placed in editor (with createVehicle)
sounds like a breaking change was made in profiling 😛

[golden wedge]

to be honest here on the module definition, not sure if its the init.

class GVAR(mod_doorBlock): Module_F
    {
        scope = 2;
        displayName = "Door Lock with Code";
        category = "fsg_eden_module_category";
        function = "fsg_eden_fnc_mod_doorBlock";
        functionPriority = 1;
        isGlobal = 1;
        isTriggerActivated = 0;
        isDisposable = 0;
        is3DEN = 0;

[restive pilot]

it is

[golden wedge]

"fsg_eden_fnc_mod_doorBlock" is the script that gets executed with nearestbuilding

[tawdry gazelle]

Ah it was an early version posted by me with many debug function which may cause slightly performance overhead. As the new version has been published, you can delete that one and switch to the new ones. 🤣

I have deleted that link to avoid any future confusions... 😅

[verbal mist]

preliminary findings:
-the old TBB file performs very well and stable, highest minimun fps - I dont know why it is 😀
-MiMalloc has drops=lower min FPS and produced some flickering of textures in 25 COOP mission with the Performance Build. @tawdry gazelle You might perhaps look into that issue.
-TBBMALLOC prodced the highest YAAB value so far.

[restive pilot]

The problem with Gold John's mimalloc is that it doesn't return amount of reserved and commited memory to the game (I noticed that the game calls them frequently, so it definitely needs them)
The free cache request doesn't work either
I guess issues at long run is not a surprise

[vagrant zodiac]

Is that with large/huge pages enables? (If that can even be done with that malloc?) Without, I've found the CMA better on all my benchmarks but not sure I've ever tried large pages with it

[verbal mist]

yes it is enabled in tbbmalloc 2021.5

[vagrant zodiac]

Sorry - one more - is that the 'tbb4malloc_bi_x64' that (I think) is also packaged with the CMA?

[restive pilot]

That's BI's default malloc

[tawdry gazelle]

Nope. If you run some tests, you can find out MemTotalCommitted() and MemFlushCache(size_t size) does not affect game behavior at all. Always return a large value, or always return a small value, or return a random value each time, Arma 3 behaves all the same. And if you know how to do memory analysis, you may also find out it has similar memory layout as BI's default memory allocator (ofc we should use the same tbb version as BI). Besides, the original CMA (old tbb) does not do MemFlushCache(size_t size) either. You can see in it's source code, it just return the incoming size. And once we add actual flush action in MemFlushCache(size_t size), performance degrade would be quite noticeable for both tbb and mimalloc.

The only problem of mimalloc might be the MemFlushCacheAll() is not taking effect. This API is called when a new scenario starts/ends. For example, it will execute once when you start a scenario, and execute once more after scenario has finished initialization and is about to start. This issue has been fixed in 2021-12-30 version, you can find it on my Github Release page. Besides, mimalloc v2.0.3 has a runtime memory optimization thread, which might cause some performance degrade. You may also try v1.7.3 to see if it runs better.

[tawdry gazelle]

The earlier version of mimalloc would: ```

1. Lock huge pages by default, which may cause memory consumption issue and would produce flickering of textures if you do not have enough free memory during gameplay. >16GB free memory is recommended if you use "LockPages" version.
2. Use AVX instruction set, which may cause performance overhead.
3. Does not flush memory when MemFlushCacheAll() is called, which may gives worse fps during gameplay.

These issues should be fixed in the most recent released version, you can find it here: https://github.com/GoldJohnKing/mimalloc/releases.

[tawdry gazelle]

Hi Dwarden, just though since the memory allocator topic does not have anything to do with perf prof branch, do I need to move to Hardware vs Arma channel? 😅

[restive pilot]

Then why would the game bother to call them at all? It DOES call them based on a simple test I did
It does something. Maybe in the long run.

Also CMA is not perfect either. Its performance degrades more noticeably than BI TBB after some time

[tawdry gazelle]

Tested for 12 hours continous running on my 32GB PC, randomly switching camera among spawned AIs, did not notice any leak... total working set are always 16GB, the same as BI default.

but you remind me... I may create a thread and flush the memory once in a while...

[empty goblet]

the sources for the last TBBv4 isn't in the CMA, they got the 2017u3, BI last on BIKI is 2017u2
our last published with game is 2017u5 , and intel published more v4, (listing last each year) 2017u6 > 2018u6 > 2019u9 > 2020u3 (that's last of the legacy, intelone is different branch)

[restive pilot]

#perf_prof_branch message
I meant this: tbb4malloc_bi_x64
I guess I didn't understand his question properly

[stark falcon]

32-bit exe profiling crashes when loading any map in editor (no mods)

Exception code: C0000005 ACCESS_VIOLATION at 00C40B18

[whole cloud]

exception codes are useless, need crash .mdmp's

[inland seal]

downloaded and zipped it. Comes with 204perf and 206perf, which one should i use?

204 have up to v17, and 206 to v06

[restive pilot]

the newest ofc

[inland seal]

and that is v17?

[restive pilot]

no, latest one in v2.06

v06

[inland seal]

oh!

[autumn timber]

Use this:
https://community.bistudio.com/wiki/Arma_3:_Steam_Branches#Arma_3

[inland seal]

aah now i see. 2.06

[autumn timber]

...instead of downloading it manually

[inland seal]

didnt know what 206 stood for

[autumn timber]

@inland seal just remember that Dedmen said:

I'm not sure if they are on there already, if not then in next two weeks

[whole cloud]

Its in

• Tweaked: Potential performance optimization when attaching static objects

[inland seal]

are you allowed to play online and using this?

[rain moth]

yes, with the performance binary

[inland seal]

so they working on a updated engine?

[whole cloud]

The game is constantly being updated and improved

[inland seal]

cool

[mellow lark]

Yes it's being ported to creation kit :) hype!

[inland seal]

cool!

[tawdry gazelle]

I have added a dedicated thread to execute mi_collect every 5 minutes, as you care about the memory collect things, hope you can run some tests on the stability st a long run. See: https://github.com/GoldJohnKing/mimalloc/commit/58069c3b87f59d2cc6ebe7958ae58b8a978a8685

I still cannot reproduce any instability on my PC at a 8 hours long run on my heavily modded server with 8-16 players and about 250 AI with both 12000 terrain & object view distance + 3.125 terrain grid. I can only reproduce a crash when all Physical + Virtual memory are all allocated by programs on my PC. For example, Arma 3 allocates 20GB pages in total but only 8GB active in physical memory, then I run a memory page allocating program to allocate about 25GB pages to drain all my Physical + Virtual memory, then Arma 3 finally crashed with a heap corruption. However if I increase my virtual memory (pagefile.sys), I will need to allocate more pages to make Arma crash. This happens when I use BI default tbb, too. And the memory consumption are same between mimalloc and BI default.

[plain trout]

have you tried that on Livonia? I felt the differences of the allocator even in the editor from just loading trees. For me I could only get it stable with the system allocator

[tawdry gazelle]

That's strange... how many free memory do you have when you launch the game?

[plain trout]

I have 32GB, and did the tests after restarting. When I am MCC (heavily modded) I only get the game not to crash with the system allocator

[nocturne obsidian]

@tawdry gazelle nice work. Unfortunately I am late to the party. Could you give me a link for an AVX2 malloc? I can test it on a 5800x

[inland dew]

All of them only for offline - not compatible with BattlEye.
Intel 2021 5 malloc has same performance as CMA

[plain trout]

were can I download that allocator? Found it. Doesn't help with Ryzen 🙂

[fickle geyser]

Is this "normal"? Never seen such logs.

[hushed seal]

Means I can still run them on any server not using battleeye, right?

[autumn timber]

You can only connect to servers not requiring BattlEye to be running on the client.
You can use it on any server, if you host your own server

Servers can always use any dll they want and BattlEye will not prevent them from doing so.
Only clients have that restriction

[hushed seal]

sweet! all the servers I play on have BE disabled anyway 😄

[whole cloud]

float32 to float16 conversion overflow/clamping, probably some object faaar out of bounds

[tawdry gazelle]

You may find the newest files and performance comparison on @verbal mist 's post: https://forums.bohemia.net/forums/topic/237257-arma-3-performance-improvement

[tawdry gazelle]

(Ah sorry replied to the wrong guy... 🤣 )

[hushed seal]

Was looking for it anyway, thanks 😄

[thin wyvern]

Don't waste your time with AVX512 because Intel disable that instruction in last microcode.

[tawdry gazelle]

I can easily enable AVX512 support on my 12700KF with latest MSI BIOS update. However I don't get any performance upgrade on any of the AVX instruction set on my 12700KF, but @verbal mist is sure he gets some... So I guess it actually depends on various types of hardware?

[night mulch]

so the mods "winter 2035" and "autumn livonia" that modify the existing terrain, seems to work fine on main branch, but doesnt work as intended on perf and dev branch, trees do not show up and when game starts it throws an error: addon 'a3_data_f_loadorder' requires addon a3_plants_F_bush.

does perf branch make changes to load order or something?

iirc the problem started happening to winter 2035 mod after this update

[whole cloud]

You tell me now about a problem that appeared 7 months ago? 🙃

[whole cloud]

no, it shouldn't.
And both of these pbo's are basegame pbo's?

[night mulch]

well the problem was with a mod not with the vanilla game, so i didnt tell you, but since it's been 7 months and the issue is still there i wanted to know what might be the change that causes it to work on main but not on perf/dev

[night mulch]

yes both a3_data_f_loadorder and a3_plants_F_bush are basegame

[whole cloud]

do these mods try to replace these pbo's?

or rather, the bush one?

[night mulch]

I'm looking at the configs rn but still havent figured all of it out what it does,, but i know at least it replaces A3_Map_Data

[whole cloud]

does it have a CfgPatches a3_plants_F_bush entry?

[night mulch]

it has a new cfgpatches entry for plants but doesnt seem to replace them A3_Plants_Winter

that's the mod's plants pbo's cfgpatches entry

[whole cloud]

Okey then I know what's wrong.
I didn't consider the possibility of the "same" pbo loading twice in the pbo lookup improvements.
Luckily I kept that profiling branch exclusive for 7 months 😁

Next time when an issue appears please tell me right away. Even if its mods, profiling branch shouldn't break anything, including mods

[night mulch]

oh thank god, so it will be fixed?

[whole cloud]

yeah, maybe in two weeks

I won't get that done till next weeks planned update

[fickle geyser]

Are the improvements disabled with filepatching on?

[night mulch]

@queen owl tagging you since it's related to your mod :D

[whole cloud]

no

[fickle geyser]

As I'm kinda using similar workflow for my dev stuff and it works fine.

Maybe I'm misunderstanding.

I'm having 2 pbos with same prefix and the one loaded last "wins".

[whole cloud]

are you loading a pbo that replaces a existing pbo by using same CfgPatches name? 😄
I'm pretty sure the issue happens at pbo load at game start

thats a different thing

[fickle geyser]

Same prefix and same CfgPatches

The issue is with different prefix but same CfgPatches?

[whole cloud]

I assume

[fickle geyser]

👍

[whole cloud]

Its been 7 months since I wrote that stuff, I'll find out what it is when I get to it

[fickle geyser]

So patching of CfgPatches

[whole cloud]

could also be same pbo prefix but with different case, my "is same" checks might be case sensitive when they shouldn't be.
Its been too long and I don't want to investigate that now

[night mulch]

does pbos with same prefix get loaded together? as in this case A3_Plants_Winter gets loaded together with A3_Plants_F?

[fickle geyser]

That's not a (pbo) prefix

And no.

[night mulch]

ah ok im wondering the mod works since it added new cfgpatches entry instead of replacing the vanilla's A3_Plants_F

[rain moth]

AFAIK the winter version replaces the prefix for map textures and objects?

[night mulch]

is it done thru config.cpp?

[spiral pond]

be careful here

same prefix = only first pbo loaded gets read by the engine. if same prefix is encountered, the pbo is skipped thereafter

cfgPatches load order = "last/later" loaded overwrites previous parameter values

load order

1. -mod=last;middle;first
2. A3 launcher: top = last, bottom = first

what both these mods do: they are repacks of A3 pbos with modified textures

as the base game data is loaded after modfolders, these are active as the A3 prefix is encountered in these modfolders and the A3 pbos are skipped

(can be seen easily in rpt log in the pbo loading section or via procmon)

[night mulch]

but the ones in winter 2035 are not exactly repack of the vanilla, like some of the mod's pbos are simply textures without any config at all. but they do have the same names for the textures as vanilla ones. Could it be that it's the texheader.bin that then replaces the old textures with new ones since they have same names?

[night mulch]

you're right, shows up in my rpt
17:35:28 Conflicting addon A3_Plants_Winter in 'a3\plants_f\bush\data\', previous definition in 'a3\plants_f\_polyplane\data\' 17:35:28 Conflicting addon A3_Plants_Winter in 'a3\plants_f\tree\data\', previous definition in 'a3\plants_f\_polyplane\data\'

[spiral pond]

a config is not required by the engine AFAIK

its only a convention by tools to expect one in the root folder

for data replacement only the prefix matters [unless its done via replacement config like usually for infantry/weapons/vehicles]

Conflicting addon
only says a cfgPatches name is used/found more than once - which is not ideal as it can confuse the load order [which isnt relevant for model/texture/rvmat containers - only for configs]

one can do partial prefix replacement/composition - BI did this in A2 due to pbo size limitation but having too much data in a folder

like
XXX
XXX\YYY

[fickle geyser]

Not true, last one overwrites.

Ah well, you're thinking about it in the opposite way, -mod=first;to;last

At least for me explaining it that way is more logical.

[night mulch]

by this do you mean prefix of pbo? prefix of cfgpatches entry? or prefix of texture itself?

[rain moth]

no, it's a flag inside the PBO header

which virtual filesystem directory the pbo represents

[autumn timber]

I always thought the order with the same CfgPatches was random because when I was listing open .pbo files to find loaded mods, I could swear that (at the time) sometimes it would pick one file and sometimes the other. That was a long time ago, though

[spiral pond]

I am going by how the engine works. -mod load order behavior is counter-intuitive but thats how it is.

[spiral pond]

not random 🙂 however depends on -mod order, filename and requiredAddons dependency resolution

[spiral pond]

pbo prefix

[fickle geyser]

It's intuitive to me, what's at the END is loaded as it "overwrites" previous mod. That's why I prefer to explain it the way I did. Internals don't matter for most users, it's the end effect they care about.

You read -mod param left to right, what you read last is the one with highest priority.

Correct me if I am wrong.

[spiral pond]

The distinction is important as prefix behaves the opposite to config overwriting.

Prefix: only 1st is loaded
Config: last loaded is overwrites

[whole cloud]

Well if that mod uses prefix to overwrite, then prefix would be "only last is loaded" ?

[violet elbow]

If two mods have pbos with the same prefix, the pbo from the mod that's further down the line in -mod is the one that applies.

[jagged minnow]

I wonder how much optimisation is still possible for the current version of the arma 3 engine regarding the AI ? Since a rewrite is too complex and cost inefficient

[whole cloud]

performance yes, but we won't change behaviour

[jagged minnow]

kk nice!

[spiral pond]

While to the average user the modline/mod launcher is the more obvious, its also more confusing, but in the end what matters how the engine processed the data. Here matters in what order the pbos get loaded, if a prefix is already defined, and only after that config load order comes into play.

[silk summit]

Guys, would it be worth it to make a request ticket for a new scripted command akin to setObjectTexture but for markers? SetMarkerTexture

There are very few mods on the workshop that add new markers and it seems that most content creators don't want to go trough all the trouble of mod making just for modifying the texture of the markers. It would be therefore great to be able to have these custom marker textures on the mission folder, specially considering how small they are, and setting them via this proposed "setMarkerTexture" command.

[fickle geyser]

IMHO it would make more sense to have description.ext support for CfgMarkers akin to CfgSFX

[silk summit]

That would work too 😉

[gritty wasp]

There was such request.
Rejected

[silk summit]

What about my _marker setMarkerTexture ["C:\Folder\Folder\Texture.paa"]; suggestion?

[gritty wasp]

1. this is wrong channel
2. say no to direct access to file system
3. that is obvious that implementation behind command would be same as for description.ext. Except config parsing. So would not be implemented. A3 EOL.

[fickle geyser]

A3 EOL
that does not matter as we're still getting new commands/features.

But yes this is not a channel for this.

[gritty wasp]

Maybe there are other limitations. Actually ticket was not totally rejected but “unlikely”. You can find it and ask there why.

[whole cloud]

Its technically possible but I don't think its worth the effort. But yes wrong channel 🙃

[patent hazel]

• Fixed: Possible game freeze when running inside Proton
  Do you think this is the freeze that people are talking about in protondb all the time? (https://www.protondb.com/app/107410) Apparently a workaround is to disable esync. If it's fixed, does this mean I could now turn esync on without freezes? It improves wine/proton performance quite a bit.

[whole cloud]

what is esync

[patent hazel]

a wine tweak that improves thread synchronization performance
https://github.com/zfigura/wine/blob/esync/README.esync
it's by default enabled in proton

[whole cloud]

Well best way is probably to just try it out

[hearty haven]

@whole cloud could you make some quick fix to allow more than 255 playercount?

https://i.imgur.com/XxybLt4.png

[whole cloud]

no

Also this is #perf_prof_branch and not #give_me_new_feature_please

[hearty haven]

poke dwarden or who?

we would love to test something like 340 players on some crazy single server(like 12900k or ultraCoreCountAMDEpyc and RamDrive) and provide you game data as you wish..

[whole cloud]

poke noone

[hearty haven]

im not getting why u being kinda rude honestly