Combat auto resolve results are wildly inconsistent even for the same combat with different RNG seed

[bleak raven]

I was testing some auto resolve battles to get a better feel for the strength of a build and a set of units. I ran the battle and got total defeat. Then I ended the turn, changed nothing else, and ran the battle again. I won without a single loss. In fact, I barely took any damage. My total mana changed by 7, and I had plenty to cast spells (was using cheats to move to wonders to test results, so I had a million mana). I did not complete a research in between turns or otherwise increase my casting points or anything like that.

The result of an auto resolved combat simply should not be this dependent on pure randomness.

This golden wonder has particularly substantial randomness in it due to the units it spawns every two turns being somewhat random - In the lost combat I rewatched and it spawned a lost wizard instead of a magebane on the first cycle, which is obviously worse for the attacker - though the fight was going much worse even before that. But since the randomness is so much more substantial in this fight than in others I decided to check against another fight, and went to a silver wonder to do the same thing. The results are less dramatic, but still clearly different despite there being functionally no changes to the armies.

I think that if the combat conditions are fundamentally identical, the result of an auto resolve should almost not vary at all. The auto resolve should not be impacted by whether the turn number is 15 or 16, it should only be impacted by the actual factors inside of the battle - the makeup of the units including enchantments and traits, the available spells and mana to cast them, and other things that are actually relevant to the combat. Sure, RNG bits like whether a % based status effect is applied should change with the seed, and sometimes that will have a big impact. But the decisions leading to where to move units and when to use those attacks and abilities should not change via some factor outside the battle like random seeding.

[green summit]

In order to work out what happened in the auto battle, the devs will need a saved game from before it started. This will help work out if there's an actual bug, if there's just a lot of variation in that particular battle, or if the tactical AI is making exceptionally bad decisions sometimes.

[calm zinc]

The tactical AI is bad pretty much all the time. Some units it just literally throws away for no good advantage.

[finite hull]

I kind of like that it varies, personally.

[bleak raven]

You don't need a save game for this, you can test it trivially. Start a game, initiate a combat, auto resolve and watch the combat. Reload to start of game, end turn, initiate the same combat, watch the combat. The AI will make different choices even though the situation is effectively the same.

And yes the tactical AI is usually making exceptionally bad decisions, watch it sometime.

[bleak raven]

Varies based on conditions changing yes, varies for no reason is bad.

Also the degree of variance here is WAY too high, win without losses and lose entire army should not both be options for the same combat

[green summit]

Ok, but you've apparently stumbled across an unusually extreme case here, and it might be useful to the devs to be able to see it.

[finite hull]

But is it? Meh.

Did you watch the replays? I bet it was due to different spell selection.

[gusty relic]

AoW4 is not like Total War where auto-resolve effectively compares stats and as such the result for the same battle will always be the same.

In AoW4 the game actually simulates the battle with all of its randomness in hit chances, crits/fumbles, etc. Even the starting positions of individual units can change from load to load. As such any slight variation can also cause units to move much differently in each scenario.

Overall, stats and auto-resolve are not a reliable measure for an army's effectiveness in battle in AoW4, the system used is not deterministic but stochastic.

The first case you show is indeed quite an extreme example of this, the second a bit less so with similar results. If you were to run each battle 1000 times and took an average that would give you your army's actual combat effectiveness for each encounter.

It would also here be important for you to show the actual balance of power before each battle and as such the chances for victory auto-resolve actually shows. If for example it shows an approximate 30% win chance then that could realistically be broken down to:
20% Win with heavy losses
9% Win with light losses
1% Win with no losses

Is the AI the best ever? I would never argue that. A lot of times it makes silly mistakes or sacrifices units for no apparent reason and these could indeed be improved especially so for higher difficulties, but moving from a stochastic to a deterministic system would be a very big change that has a lot of issues of its own. No system is 100% perfect, it is really a matter of design choice and personal preference.

[bleak raven]

Power estimate doesn't really matter as you said early on, so posting it doesn't mean anything to post it

[bleak raven]

Yes I did. Spells used were the same, at the same times. It was pretty much entirely positioning based, where the AI positioned so much worse in one of the fights on side A and better on side B, and vice versa in the other fight.

Stuff that's objectively bad too, like putting the dogs behind the hound masters so they basically don't exist for the fight

[green summit]

Ok, that is strange. I would have expected the AI's actions to be deterministic.

Sure, they might change based on random events earlier in the fight, but given two goes at the exact same setup, the AI should behave identically.

Do you mean the AI moved differently, or that the initial placement of units on the battlefield was different?

[bleak raven]

The AI moves differently. To be honest I didn't check but I would assume the spawn points of hounds around the hound masters should be consistent, and same for the leader. Regardless, if those additional spawns happen at random and affect the combat consistency then they should be standardized.

[green summit]

If you're willing to share the save file, I can check what the difference was.