▫️Sea of Thieves

[tropic helm]

Discussion of Sea of Thieves

Previous: https://discord.com/channels/1161035767917850784/1211660711609114654

[teal pebble]

https://www.neowin.net/news/sea-of-thieves-boasts-performance-boosts-with-directx-12-update-but-its-a-90gb-download/

[tropic helm]

modding this game is interesting to say the least. i feel like it's a DX11 render inside DX12, if that even is a thing. i can modify the last two shaders, but anything else doesn't take

it being an xbox game (on PC) just makes things harder

either way. it seems it uses the old Hable/ACES mix that Unreal made before for SDR, and then they have some custom stuff to make it "go to" HDR

i have mostly everything i need to modify it, just the game won't accept the replaced shaders

[tropic helm]

maybe the shaders aren't being dumped right. if i could get PIX to work, i could confirm that.

but i'd need the steam version for that

[last sparrow]

I've been waiting for one of the lead devs to get back from vacation before I "pester" them about this issue, but I'm hoping that appealing to the creative director, art director, and lead dev might get some traction since all three them are very passionate about the game, and that even in the best case scenario, using the provided sliders, the game is both too dark or bright depending on location and it doesn't match their intended art style.

The more I can provide about what appears to be wrong/outdated and any solutions the easier it might be to convince them to work on it. Or so I hope.

[tropic helm]

at this point i kinda have the decompiled source code and where that code exists in the unreal engine source code. but even modifying it, i can't seem to upload my changes to the game engine.

do you have the steam version or xbox?

[last sparrow]

Xbox

But technically both with an insider version of Steam, though I havent used it years.

[tropic helm]

would have to be the same

what's interesting is i don't have actually have access to the full output. only the UI layer. so maybe this is like a DX11 inside DX12

and only the UI is in DX12? it's weird.

maybe it's a memory protection thing.

i do have full output access. it's not a layer: edit... well...

yeah, i think layered in the end

[tropic helm]

i think it's the compilation process at this point. using reshade's replacer will cause the same issue of a broken output.

[tropic helm]

finally!

decompilation issues i had to work out by hand

[tropic helm]

unscaled, maybe even untonemapped

[tropic helm]

i'm guessing bloom

[tropic helm]

they use a 32x32x32 lut which just has luminance, it seems. maybe that's their tonemapper

[tropic helm]

calculating luminance with BT601 😦

[tropic helm]

seems to be untonemapped color without any exposure adjustment (pngs are a lie)

found peak nits. maybe that's enough now

Tone mapped to ACES at 800 peaks nits (user setting) w/ BT2020 processing

Vanilla / ACES HDR

Vanilla / ACES HDR

oh right, i set UI Paperwhite to 100

i don't think this is using their color grading though

it is. it seems they use a baked LUT for luminance. it's probably the ACES 1000 but they stretch it, i'm sure

they use a log based LUT which again, is probably ACES

[tropic helm]

@vale harbor i have to figure out the rest, but the tonemapper is essentially one file. a quick test of inputting ACES works as expected but without the added saturation from stretching

[vale harbor]

this is their version or yours?

how did you hook reshade?

[tropic helm]

It's theirs. I added the lines at the bottom for my ACES

[tropic helm]

Install Reshade against the launcher, not the SoTGame.exe

[vale harbor]

but how, you can't even find the executable.

u got it on steam?

[vale harbor]

r3.w = yFromBT601(r3.xyz); wow so they use Rec.601?

[tropic helm]

Yeah. Look for 0.114 in the Unreal engine and it's everywhere

Like they copy-pasted some old luminance formula from somewhere

https://github.com/search?q=repo%3AEpicGames%2FUnrealEngine 0.114&type=code

Bethesda used the same number for random for film grain generation.

[vale harbor]

oh no, @strange crag the BT.601 curse strikes again. We have to fix this... I'll make a PR soon, I have contacts with epic.

[strange crag]

lol

[tropic helm]

Cyberpunk also uses bt601 somewhere in their tonemapper

[vale harbor]

there's about 20 wrong luminance vectors across all kinds of code

most of them shouldn't even be there given that UE has configurable color primaries (???)

[last sparrow]

I just want to pop in to say I might only understand about 12% of everything you're discussing... but it's incredibly exciting and interesting seeing all this discovery work. :)

[vale harbor]

I hope you fixed that

btw, once you have ~finished with CP2077, I wanted to write down a recap of what we (you) did and inform them about it, maybe get some answers or fixes

[tropic helm]

They use it for hue shifting. I think YCbCr is based on BT601. I haven't seen a location use a hue angle other than 0 so I just ignored it.

[vale harbor]

YCbCr is relative to the color space you want to use I think.

[tropic helm]

They didn't even build a branch against the cbuffer, so it was doing sin/cos on 0.

[tropic helm]

r5.xyzw = cmp(asint(cb0[37].yyyy) == int4(2, 1, 3, 4));
r1.w = (int)r5.z | (int)r5.y;
r1.w = (int)r5.w | (int)r1.w;
r0.x = r0.x ? r1.w : 0;
r0.x = (int)r0.x | (int)r5.x;
if (r0.x != 0) {

I'm guessing this is what a switch statement with fallthroughs looks like in asm: 0 1 2 3 4 5

https://github.com/clshortfuse/renodx/tree/main/src/seaofthieves Added to github.

[strange crag]

YCbCr is different for every ITU standard. For BT.601 the luma vector is derived from something else. For BT.709 and BT.2020 it's basically the relative luminance vector rounded to 4 digits, while making sure all 3 values added together equal 1.

YCbCr is a difference signal btw.
Y holds white and green, Cb blue and Cr red.
If you pay attention while watching a video (not movie as they tend to avoid that issue) where Cb and Cr are subsampled, you will notice that every color that has just a very slight tint of green in to appear sharper and grey scale colors too obviously.
YCbCr was developed to have both color TV and black and white TV in the same signal. These days the model is outdated and ICtCp is a good step forward in perceptual compression.

[tropic helm]

if (r0.x != 0) {
    r6.rgb = saturate(r2.rgb);
    r6.rgb = pow(r6.rgb, 2.2f);  // 2.2 Gamma
    r7.rgb = mul(BT709_2_BT2020_MAT, r6.rgb);
    r6.rgb = r7.rgb * 600 / 10000.f;  // Stretch to 600 nits
    r6.rgb = pqFromLinear(r6.rgb);
    r3.rgb = min(r6.rgb, 1.f)  // Clamp (again?)

if using one of the options. seems like they tested against 600 nits and i believe it's stretched

i'd imagine it's like cyberpunk where more peak nits = more stretching = more saturation

[vale harbor]

is the tonemapping always based on the same paper white and peak brightness, and then stretched after tm?

[tropic helm]

this option probably isn't being used in runtime, but they may have just used 600 internally and though the game looked good. i'm still reworking the part that uses the user's peak nits value. and i'm seeing they do (2 * userPeakNits) / 10000.f.

i guess 600 * 2 is close to 1000 if they are using the Unreal ACES 1000 nit LUT

don't understand this yet:

      r5.rgb = mul(BT709_2_BT2020_MAT, r4.rgb);
      r4.rgb = r5.rgb / 0.18f;
      r4.rgb = pow(r4.rgb, cb0[38].y);
      r4.rgb *= cb0[37].w;
      r5.rgb = 0.18f * r4.rgb;
      r6.rgb = r4.rgb * 0.4518f + 0.3f;
      r6.rgb = r6.rgb * r5.rgb;
      r4.rgb = r4.rgb * 0.4482f + 0.59f;
      r4.rgb = r5.rgb * r4.rgb + 0.4f;
      r4.rgb = saturate(r6.rgb / r4.rgb);
      float newPeak = (2 * userPeakNits) / 10000.f;  // Maybe to stretch contrast?
      r4.rgb = r4.rgb * newPeak;
      r4.rgb = cb0[38].xxx * r4.rgb;

[vale harbor]

the game defaults to 1200 nits as peak brightness

[tropic helm]

does it peak to 2400 then?

[vale harbor]

i don't know

clipping is not the problem

[tropic helm]

i set it to 800 and it just goes to like 1500 for me, so i figured they're scaling 2x

[vale harbor]

changing the peak seems to linearly scale the image

not just highlights

@bleak siren

[tropic helm]

anytime it uses peaknits here, it does r0.x = 0.000199999995 * userPeakNits; which is 2x / 10000.f

https://github.com/EpicGames/UnrealEngine/blob/072300df18a94f18077ca20a14224b5d99fee872/Engine/Shaders/Private/PostProcessCombineLUTs.usf#L78C2-L92C2
might be this (not sure yet). without the AP1 stuff

i saw AP1_R2B2Y somewhere else. maybe it builds the LUT... every frame

i think that is close to that ^ Unreal code, which could be color contrast for cb0[38].y

haha: https://github.com/EpicGames/UnrealEngine/blob/072300df18a94f18077ca20a14224b5d99fee872/Engine/Plugins/Experimental/ColorCorrectRegions/Shaders/Private/ColorCorrectRegionsCommon.usf#L174
ColorAP1 = pow(ColorAP1 * (5.55555555), ColorContrast.xyz * ColorContrast.w) * 0.18;

5.55555 = (1 / 0.18). so it looks like even Unreal engine uses reverse engineered code

[vale harbor]

yes, that's definitely contrast, to pivot it around mid gray, which makes a lot of sense

[tropic helm]

yeah, scale down via 0.18, pow, then scale back up around 0.18

i don't get the 0.4518 and 0.3 yet

[tropic helm]

found it: https://github.com/EpicGames/UnrealEngine/blob/072300df18a94f18077ca20a14224b5d99fee872/Engine/Plugins/Experimental/ColorCorrectRegions/Shaders/Private/ColorCorrectRegionsShader.usf#L96

well, one of them. might be some optimization

MaterialFloat Luminance( MaterialFloat3 LinearColor ){
  return dot( LinearColor, MaterialFloat3( 0.3, 0.59, 0.11 ) );
}

https://github.com/EpicGames/UnrealEngine/blob/072300df18a94f18077ca20a14224b5d99fee872/Engine/Shaders/Private/Common.ush#L597
???

what's primaries are that?

RGB Luminance value = 0.3 R + 0.59 G + 0.11 B

https://www.scantips.com/lumin.html

@strange crag do you know where these numbers come from?

perceived luminance with gamma?

ugh, it's BT601 again, with even less precision: float3(0.299, 0.587, 0.114));

@vale harbor please get upset again^

[strange crag]

There is also a luma vector meant for in gamma values a dude made up in the 90s. It's also all over games

[tropic helm]

i'm starting to this think unreal engine ain't so great

wasn't there a green hue or something in HDR?, because they wrote 0.03 instead of 0.3 for the Red primary

i swear somebody said that

[strange crag]

These ones:
0.3086, 0.6094, 0.0820

[tropic helm]

From 1993 😦

To convert a color image into a black and white image, this matrix is used:
float mat[4][4] = {
rwgt, rwgt, rwgt, 0.0,
gwgt, gwgt, gwgt, 0.0,
bwgt, bwgt, bwgt, 0.0,
0.0, 0.0, 0.0, 1.0,
};
Where rwgt is 0.3086, gwgt is 0.6094, and bwgt is 0.0820. This is the luminance vector. Notice here that we do not use the standard NTSC weights of 0.299, 0.587, and 0.114. The NTSC weights are only applicable to RGB colors in a gamma 2.2 color space. For linear RGB colors the values above are better.

• https://www.opengl.org/archives/resources/code/samples/advanced/advanced97/notes/node140.html
• https://www.opengl.org/archives/resources/code/samples/advanced/advanced97/notes/node258.html#haeberli:matrices
• https://web.archive.org/web/19970605062709/http://www.sgi.com/grafica/matrix/index.html

maybe it's true? I can't BT601 's XYZ.

in 2.4? https://www.color.org/chardata/rgb/bt601.xalter

the pdf mentions gamma pre-corrected for the coefficients:

[strange crag]

@vale harbor and me have already been through this so have fun 🙂

[strange crag]

Yeah because it's for encoding YCbCr and not getting the relative luminance

[vale harbor]

unless there's other color spaces or color representations at play, BT.601 or similar luminance vectors should never be used.

[strange crag]

Ah it was meant for linear values

[tropic helm]

still doesn't match up with the 525-line XYZ though. i wish people would show their work. but maybe that wasn't a thing 31 years ago.

no vignette / vignette

[strange crag]

of course they don't

[tropic helm]

this is weird:

  float3 inputColor;
  inputColor.r = t1.Sample(s1_s, v3.zw).r;
  inputColor.g = t1.Sample(s1_s, v3.xy).g;
  inputColor.b = t1.Sample(s1_s, v0.xy).b;

red and green could be sampled by a different position. no delta either: testColor = abs(t1.Sample(s1_s, v0.xy).rgb - inputColor);

i wouldn't care too much except it's 3x sampling

[strange crag]

context?

[tropic helm]

  float3 inputColor;
  inputColor.r = t1.Sample(s1_s, v3.zw).r;
  inputColor.g = t1.Sample(s1_s, v3.xy).g;
  inputColor.b = t1.Sample(s1_s, v0.xy).b;

  const float3 bloomColor = t2.Sample(s2_s, v0.xy).xyz;

  r1.rgb = bloomColor.rgb;

  float2 texture0Coords = v0.zw * float2(0.5, -0.5) + float2(0.5, 0.5);
  const float3 texture0Color = t0.Sample(s0_s, texture0Coords).xyz;

  r2.xyz = texture0Color.rgb * cb1[0].xyz + cb0[33].xyz;

  r1.rgb = r2.xyz * bloomColor.rgb;

  inputColor = inputColor * cb0[32].xyz + r1.rgb;

  r1.rgb = v1.xxx * inputColor;
  float3 bloomedColor = r1.rgb;

  // half3 ColorLookupTable( half3 LinearColor )
  // LUTEncodedColor = LinToLog( LinearColor + LogToLin( 0 ) );

  r2.xy = cb0[34].z * v1.yz;

  // Vignette?
  float vignette = 1.f + dot(r2.xy, r2.xy);
  vignette = rcp(vignette);
  vignette = vignette * vignette;

  float3 vignettedColor = bloomedColor * vignette;

[strange crag]

where are those coords coming from? 😄

[tropic helm]

struct PS_INPUT {
  float4 v0 : TEXCOORD0;
  float3 v1 : TEXCOORD1;
  float4 v2 : TEXCOORD2;
  float4 v3 : TEXCOORD3;
};

struct PS_OUTPUT {
  float4 o0 : SV_Target0;
  float4 o1 : SV_Target1;
};

[strange crag]

the coordinates not the textures 🙃

ah sorry oops

[tropic helm]

same texture, one coord for red, another for green, and another (v0) for blue

like, it should all be v0

[strange crag]

do you know to what textures the samplers are bound?

[tropic helm]

😅

[strange crag]

time to get a frame capture and search for the shader

btw you can compile the shader with debug info so it's easier to debug

[tropic helm]

can't. it's Xbox Game pass, no PIX

[strange crag]

oh

game is on steam too 👀

[tropic helm]

$$$

i mean, it is the same t0 texture, just sampled weirdly, with the same sampler

maybe they were doing some upscaling

[strange crag]

I mean how do you think I got a capture of starfield?

[tropic helm]

using your magic of course

if you try the main exe with PIX

[strange crag]

I mean for starfield it was bad because it isn't really supported

ah

there is a workaround

[tropic helm]

i tried for a bit but didn't really need it. (symlinks, changing owner, acl command)

it is the same texture, but i'm not sure why the v3.xyzw even exists. i used v0.xy and the delta is the same

[strange crag]

what it did for me was moving the exe to another folder and in the original folder hardlink to it
in the other folder I symlinked everything from the original folder and then just start the exe in the other folder

[tropic helm]

can't even copy the exe

oh, rename the folder... hmm

[strange crag]

how did you install it?

yeah not that

[tropic helm]

installed it from xbox app

though i can rename the folder. let me try that

[strange crag]

still have the same setup:

[arch@Flowering Content]$ find . -links +1
./bink2w64.dll
./DefaultGlobalGraphicsSettings.json
./gamelaunchhelper.exe
./High.ini
./layout_ff5f9afb-a689-44d8-85cd-06cf9139122e.xml
./Low.ini
./Medium.ini
./MicrosoftGame.Config
./resources.pri
./ssce5564.dll
./Starfield.exe
./Starfield.ini
./Starfield_it.ini
./Starfield_pl.ini
./Starfield_ptbr.ini
./Starfield_zhhans.ini
./Ultra.ini
./XCurl.dll

with linux's find you can determine hardlinks

[tropic helm]

i can make a symlink, but won't work with PIX. hardlink is denied:

C:\XboxGames\Sea of Thieves\Content\Athena\Binaries\WinGDK>mklink SoTGame.exe ..\WinGDK.2\SoTGame.exe /H
Access is denied.

[strange crag]

running as admin?

[tropic helm]

same, yeah

like, i can't read the exe contents

[strange crag]

let me install it 🙂

[tropic helm]

bestest of viruses

[strange crag]

I actually own the game on the xbox app

100GB omg

[tropic helm]

haha, yeah. i had to uninstall stuff

[strange crag]

xbox app does not have a timer how long it will take smh

[tropic helm]

also slower than my connection. i think it was 200mbps

[strange crag]

getting 30MB/s here

which is almost 100% saturated

[tropic helm]

yeah, so ~240mbps

i have 2500 so i really felt it

[strange crag]

15%

[strange crag]

recently got a 4TB SSD 🙂

[tropic helm]

you're doing windows in linux, right? so i'd imagine the ACL nonsense they do to block access isn't emulated

[strange crag]

I have WSL

also have arch on another SSD

also arch in WSL and not Ubuntu 😄

[tropic helm]

oh, so your linux commans were in WSL. let me try reading from WSL

can't sudo cp 😕

so i think the game isn't even use that LUT. it's an SDR lut. they have a custom HDR tonemapper i think

r0.rgb = lerp(bloomedColor, cb0[40].rgb, cb0[40].w);

      r1.rgb = mul(BT709_2_BT2020_MAT, r0.rgb);

      r0.yzw = r1.rgb / 0.18f;
      r0.yzw = pow(r0.yzw, colorContrast) * colorContrastGain;
      r1.rgb = 0.18f * r0.yzw;

      r5.rgb = r0.yzw * 0.4518f + 0.03f;
      r5.rgb = r5.rgb * r1.rgb;

      r0.yzw = r0.yzw * 0.4374f + 0.59f;
      r0.yzw = r1.rgb * r0.yzw + 0.4f;
      r0.yzw = saturate(r5.rgb / r0.yzw);

      float newPeak = (2 * userPeakNits) / 10000.f;  // Maybe to stretch contrast?
      r0.rgb = r0.yzw * newPeak;

      r4.rgb = cb0[38].xxx * r0.rgb;

that is the tonemap

bloomedColor never gets inputted to the LUT which is an SDR lut

though... i'm confused about the vignette...

the SDR lutted + grain color is in output1, and HDR in output0. maybe they composite...

yeah, i think i'm done now

just about their average nits which is 4x exposure

might have to double their color saturation for this

now it's just a matter of playing with the contrast / saturation / color gain / exposure

now it looks like HDR because of that contrast. paperwhite probably need to get pumped up: 1.5x color contrast, 2x exposure

[strange crag]

@tropic helm I can just move the file around

[tropic helm]

@strange crag but can you read it?

(unless that's what you mean by "just")

[strange crag]

I don't copy the file I move it (CTRL+X)

[vale harbor]

can u all tell me how the fk you got reshade working 😄

[tropic helm]

derp. yeah, i moved it outside C:\XboxGames

😅

[tropic helm]

install against the launcher folder, not the exe

@vale harbor

dammit this works lol

[vale harbor]

got it

I didn't know i could access the files, I thought they were behind encrypted folders

[tropic helm]

move the exe to a temp folder outside xboxgames, make copy, move back, paste copy as SoTGame2.exe and launch with Pix

[bleak siren]

The curse of 601

[tropic helm]

everything is even named

[vale harbor]

that's google's fault as well, because if you google sRGB luminance, it often gives u bt601 values

[strange crag]

I did too

λ ls -R "I:\XboxGamesProxy\Sea of Thieves"
'I:\XboxGamesProxy\Sea of Thieves':
total 168K
drwxr-xr-x 1 Lily 197121    0 2024-02-28 20:14 Athena/
lrwxrwxrwx 1 Lily 197121   42 2024-02-28 20:13 Engine -> '/i/XboxGames/Sea of Thieves/Content/Engine'/
lrwxrwxrwx 1 Lily 197121   45 2024-02-28 20:13 Resources -> '/i/XboxGames/Sea of Thieves/Content/Resources'/
-rw-r--r-- 1 Lily 197121  292 2024-02-28 20:12 _link.bat
-rw-r--r-- 3 Lily 197121 3.2K 2024-02-28 19:56 appxmanifest.xml
-rwxr-xr-x 3 Lily 197121  98K 2024-02-17 09:07 gamelaunchhelper.exe*
-rw-r--r-- 3 Lily 197121  47K 2024-02-28 18:58 layout_55026684-c83b-4a77-a2c0-8f86e6520e5b.xml
-rw-r--r-- 3 Lily 197121 2.0K 2024-02-28 18:58 MicrosoftGame.config
-rw-r--r-- 2 Lily 197121 1.9K 2024-02-28 18:58 resources.pri
-rw-r--r-- 2 Lily 197121   32 2024-02-28 18:58 UE4CommandLine.txt

'I:\XboxGamesProxy\Sea of Thieves/Athena':
total 1.0K
drwxr-xr-x 1 Lily 197121   0 2024-02-28 20:09 Binaries/
lrwxrwxrwx 1 Lily 197121  49 2024-02-28 20:09 Config -> '/i/XboxGames/Sea of Thieves/Content/Athena/Config'/
lrwxrwxrwx 1 Lily 197121  50 2024-02-28 20:09 Content -> '/i/XboxGames/Sea of Thieves/Content/Athena/Content'/
-rw-r--r-- 1 Lily 197121 152 2024-02-28 20:09 _link.bat

'I:\XboxGamesProxy\Sea of Thieves/Athena/Binaries':
total 0
drwxr-xr-x 1 Lily 197121 0 2024-02-28 20:09 WinGDK/

'I:\XboxGamesProxy\Sea of Thieves/Athena/Binaries/WinGDK':
total 145M
-rwxr-xr-x 1 Lily 197121 3.2M 2024-02-28 19:56 Party.dll*
-rwxr-xr-x 1 Lily 197121 1.0M 2024-02-28 19:56 PartyXboxLive.dll*
-rwxr-xr-x 1 Lily 197121 1.5M 2024-02-28 19:56 PlayFabMultiplayerGDK.dll*
-rwxr-xr-x 1 Lily 197121 139M 2024-02-28 19:56 SoTGame.exe*
-rw-r--r-- 1 Lily 197121   12 2024-02-28 18:58 Version.txt
-rwxr-xr-x 1 Lily 197121 190K 2024-02-28 19:56 XCurl.dll*

[tropic helm]

for now the copy is fine

yeah, well that clarifies things:

BlackDummy.

that explains

[strange crag]

if you install the game through the windows store they still are afaik

[vale harbor]

but if you install through the xbox game app it's different/

?

[strange crag]

yeah

[vale harbor]

seems like I installed it through the xbox app then 😄

i got 1 day left of trial. RIP SoT.

[bleak siren]

I’m downloading it now - I will use another gamertag to be safe

[vale harbor]

buying two subscriptions ?

[bleak siren]

Interested in you getting pix working - I’ve only ever used renderdoc for prying

[bleak siren]

Nah, I guess that I can use another profile on the same machine - I think it is per machine rather than per account

[tropic helm]

i essentially have two values that i need to tweak to replicate their funky contrast/saturation. i settled on this.

const float newContrast = colorContrast * 1.5f;
const float newGain = colorContrastGain * 2.0f;

pushing to git...

[bleak siren]

It is on xbox - I don’t know if you can do a similar things

[tropic helm]

https://github.com/clshortfuse/renodx/blob/main/src/shaders/sot/tonemapper_0xE41360A3.ps_5_0.hlsl

oof, pretty dark. let me actually see what game options there are

Vanilla / ACES 800

const float newContrast = colorContrast * 1.5f;
const float newGain = colorContrastGain * 4.0f;

probably black floor being 0.0001 😕. whatever that's HDR

@quartz wraith

@zealous shale

[quartz wraith]

thanks! I'm going to test in in a sec

[tropic helm]

there are no real options. it's just ACES to peak nits. may be a bit dark, but i don't know what user options there are yet

i wasn't able to inject buffers because all the slots are used.

[zealous shale]

Sorry, last multiplayer game I was into was Quake III 🤣

[tropic helm]

sorry, i thought you had played it

[zealous shale]

Nah it's cool

maybe next time

[tropic helm]

it's on game pass. i didn't buy it 🙂

btw, not my fault if you get banned 😅

[zealous shale]

I wonder if it's using the same tonemapper as Gears 5
Since Gears 5 tonemapper is dog shit
And you can't use Lilium's on the legal copy, since anticheat blocks reshade
I miss the days when SP and MP were on separate exe's

[tropic helm]

this is the tonemapper:

r5.rgb = r0.yzw * 0.4518f + 0.03f;
r5.rgb = r5.rgb * r1.rgb;
r0.yzw = r0.yzw * 0.4374f + 0.59f;
r0.yzw = r1.rgb * r0.yzw + 0.4f;
r0.yzw = saturate(r5.rgb / r0.yzw);

float newPeak = (2 * userPeakNits) / 10000.f;  // Maybe to stretch contrast?
r0.rgb = r0.yzw * newPeak;

it's fast i guess.

some weird BT601 numbers

[quartz wraith]

it cant load the addon

do I need a special version of reshade?

[tropic helm]

#1204850050530807838 message

[quartz wraith]

ah that one

okok

[tropic helm]

yeah, this is the new Reshade API.

[zealous shale]

If it's clipping highlights, oversaturating the colors and crushing shadow detail, then yes 🙂

[quartz wraith]

should I use the default settings?

in-game

[tropic helm]

that's how i configured it, but i'm testing the in-game sliders now

[bleak siren]

I interviewed the tech lead at gears 5 - I can tell you how it was described “We used machine learning to train an inverse tone mapper for colour space conversion. We blended this with a Reinhard HDR buffer and then did a lot of tuning of the resulting image to get the optimal visual results.”

[quartz wraith]

umm

it looks very weird

lol

or it's just an event going on not sure

but it's very dark too

[zealous shale]

If by optimal the meant "it looks like AutoHDR" then I guess it does
But tons of folks say it's the best HDR they have seen, so what do I know

[bleak siren]

Often it’s really hard to mentally separate “good graphics” from “good HDR” whatever that truly is

[tropic helm]

hdr / sdr

how SDR should i get it?

i'm going to try to match HDR 400

[bleak siren]

It’s super dark - prior to the update that added the extra hdr controls - all you could do is increase the “peak nits” up to 1500+ or so to be able to see in dark areas if i wasn’t in a dark room.

[zealous shale]

You need to set in game brightness slider above 2600 nits to start recovering highlights, it's bizarre to me and goes against the idea of expanded dynamic range
Heck, you can easily compress it even further vs SDR if you set PW in the game above 100
But enough about Gears 5

[tropic helm]

did the contrast controls ever work?

i balanced to the intro screen, so i'll do actually gameplay now

[bleak siren]

Yes 😂

Or

If you prefer

[tropic helm]

this is probably more sensible

[quartz wraith]

I'm goin to try the intro too

[tropic helm]

if you set color contrast to 0... uh

[quartz wraith]

cuz the sky is green atm lol

[tropic helm]

stuff might move with peaknits... maybe

why is SDR so bright!

      const float newContrast = colorContrast * 1.0f;
      const float newGain = colorContrastGain * 8.0f;

400 nits / 800 nits

10K nits. seems stable

@quartz wraith

[quartz wraith]

HDR brightness 60 + 35 color contrast kinda matches SDR

oh new version

[tropic helm]

please don't tell me HDR Brightness is just this lerp: r0.rgb = lerp(bloomedColor, cb0[40].rgb, cb0[40].w);

it's not. but now i have to find it to make it better

[quartz wraith]

so I should just use 10k nits now?

50/50 with 800 nits looks good now

[tropic helm]

nah, use your normal nits

[quartz wraith]

night and day difference lol

[tropic helm]

from the last version or the vanilla game?

[quartz wraith]

both

it looks normal now at the default settings

ah I cant upload jxrs again

😄

[tropic helm]

can't find this HDR Brightness thing 😕

i want to make it into paperwhite

[quartz wraith]

At default settings:
https://gofile.io/d/R2Fqx8

the tavern

[tropic helm]

reduce resolution?

how do i get there?

[quartz wraith]

start normal game mode

private session

[tropic helm]

meh, i have to be in-game for the HDR sliders to work. all that loading...

[quartz wraith]

yea the game takes ages to load

[tropic helm]

time to draw all the variables onto the screen

[quartz wraith]

@last sparrow

[tropic helm]

got it, the scale is 0 = 0; 25 = 0.0679; 50 = 1.x, 75 = 5.17; 80 = 6.419; 100 = 16

i wonder if there's another slider i can use for UI paperwhite

[tropic helm]

pow(x, 0.25) basically (since it's pow(x/50, 4)). so now we have paperwhite, which i'm doing on a linear scale

50 will be 200 and 100 is 400.

rip 203, but math is easier

actually works out well because peak nits is capped at 400 in the UI anyway

SDR vs HDR at 400 paperwhite. hmmmm

maybe that should be the new default?

Game's default UI Paperwhite

Half of Game's default UI Paperwhite (100 nits)

[quartz wraith]

yea 400 is pretty close to SDR

[tropic helm]

okay, so i think that's as much as i can do right now. no buffers to tweak stuff, unforunately

[quartz wraith]

still HDR is basically fixed now 🥳

🙏

[tropic helm]

i don't know SDR uses ACES, but if it does, it'll look the same

hmm, failed to connect to server 😅. edit: it's fine

[quartz wraith]

😄

[tropic helm]

so the game actually has a buffer value for UI Paperwhite and it just sends 200 nits to the shader. they could have made a slider for that...

[vale harbor]

@tropic helm can we have a recap of what you did/changed?
Sorry, no energy to read through everything now, but I could help and test

[tropic helm]

#if RENODX_SOT_ACES_TONEMAPPER
      r1.rgb *= 8.f;  // Scale up close to SDR;

      float hdrBrightness = pow(finalGain, 0.25f);  // 0-2
      float userPaperWhite = 200.f * hdrBrightness;

      float3 ap0Color = mul(BT2020_2_AP0_MAT, r1.rgb);
      float3 tonemappedColor = aces_odt(
        aces_rrt(ap0Color),
        0.0001f,  // minY
        48.f * (userPeakNits / userPaperWhite),
        AP1_2_BT2020_MAT
      );
      r4.rgb = tonemappedColor.rgb * userPeakNits / 10000.f;
#else

      r5.rgb = r0.yzw * 0.4518f + 0.03f;
      r5.rgb = r5.rgb * r1.rgb;
      r0.yzw = r0.yzw * 0.4374f + 0.59f;
      r0.yzw = r1.rgb * r0.yzw + 0.4f;
      r0.yzw = saturate(r5.rgb / r0.yzw);

      float newPeak = (2 * userPeakNits) / 10000.f;  // Maybe to stretch contrast?
      r0.rgb = r0.yzw * newPeak;

      r4.rgb = finalGain * r0.rgb;
#endif

that's it. use ACES hdr

editing with new commit...

i just realized the color contrast is the same. so i can clean up more

[vale harbor]

this was the whole tonemapper?

the new unreal ACES tonemapper is like 2000 files

[tropic helm]

well, i have the aces stuff in another file

but it's the same ACES as reference

[vale harbor]

why is the color BT.2020 to begin with? Where's the BT.709 to BT.2020 conversion?

[tropic helm]

if i want it be "perfect" unreal ACES, it's 2x paperwhite

[vale harbor]

When I first saw it I thought the game was interpreting 709 as 2020.

[tropic helm]

https://github.com/clshortfuse/renodx/blob/a013e6d9b60b395bf4fe185aa951d40ec8b4cd11/src/shaders/sot/tonemapper_0xE41360A3.ps_5_0.hlsl#L193

see it on the github, better

if i want to match how Unreal does HDR, i would do base peak nits at 108 instead of 48, then almost it's straight up 1:1 SDR (minus highlights of course)

[vale harbor]

maybe that'd be best, as long as SDR in this game uses the same ACES as UE does now, but this is from ~2018

      float3 ap0Color = mul(BT2020_2_AP0_MAT, r1.rgb);
      float3 tonemappedColor = aces_odt(
        aces_rrt(ap0Color),
        0.0001f,  // minY
        48.f * (userPeakNits / userPaperWhite),
        AP1_2_BT2020_MAT
      );
      r4.rgb = tonemappedColor.rgb * userPeakNits / 10000.f;

this converts from 2020 to AP0 and then from AP1 to 2020 back?

[tropic helm]

yes. aces uses ap0 as input. and the last parameter i put converts AP1 back to 2020

[vale harbor]

so internally it coverts to AP1

[tropic helm]

yeah, aces_rrt is AP0 => AP1

aces_odt is AP1 => (based on matrix)

(it's wrapped)

float3 ap0Color = mul(BT2020_2_AP0_MAT, r1.rgb);
float3 ap1Color = aces_rrt(ap0Color);
float3 tonemappedColor = aces_odt(
  ap1Color,
  0.0001f,  // minY
  48.f * (userPeakNits / userPaperWhite),
  AP1_2_BT2020_MAT
);
r4.rgb = tonemappedColor.rgb * userPeakNits / 10000.f;

fxc dump went from 152 lines to 547 lines

[last sparrow]

It's beautiful! I love the highlights and detail. :D Nearly 6 years later and finally fixed HDR. That's amazing.

[vale harbor]

Where was the color shift from?

[tropic helm]

Likely:

r5.rgb = r0.yzw * 0.4518f + 0.03f;

Another section of the code uses

r6.rgb = r4.rgb * 0.4518f + 0.3f;.

That's probably the BT601 coefficient for red (0.299)

[vale harbor]

Why would they multiply rgb by red luminance anyway?

[tropic helm]

It's to get luminance (in bt601). The green primary is there too (0.59). the 0.114 for blue probably got optimized away.

Maybe it's in an older source code of the unreal engine.

[vale harbor]

Why would 0.114 get optimized away, it's a relevant number.

This looks like something else.

Maybe it's just a random offset of 0.3, eyeballed

[tropic helm]

It's 0.3 and 0.59 lines above but 0.03 and 0.59 here

[bleak siren]

Was just looking and saw there was a Dolby vision plugin for UE - but it’s by request only. Boo.

A game in DV would be interesting to peer into - see what the DV tonemapper is doing

[vale harbor]

I think the code would be hidden away in a pre compiled dll.

We'd have to reverse engineer it.

[tropic helm]

Their ACES tonemapper is just one for 1000 and one for 2000. There's no "to user nits" code. I'm guessing they just have DV know it's a 1000 nit grade and have the TV tonemap the rest.

[vale harbor]

DV has its own tonemapper on top of aces I'd think.

Maybe they'd do aces at 10k nits and then DV to your target nits and color range.

[vale harbor]

I think that for games it's the game that needs to do the DV tonemapping. Lilium and others might know more.

dogelition

[tropic helm]

ACES only has a sample for 1K and 2K and that's as far as Unreal got (copy paste).

[vale harbor]

Did you try to only correct this and see if it fixed the color shift?

[tropic helm]

I can try again. But the peak nits being double is kinda such a mess you don't know what's the game and what's Windows, or my TV handling the overage. The main screen is 1580 nits

I didn't really care about fixing it since it was already wonky.

And since SDR is probably ACES, then ACES on HDR too makes sense. They probably just didn't know how to do it and Unreal engine was no help (1K or 2K only)

Kinda the same problem with Cyberpunk, but at least they tried to use the full ACES code. I wonder how Larian did it. Maybe that's the one I should be referencing.

[vale harbor]

this might have been before unreal had their own HDR. Not sure.

[vale harbor]

we shall get some Larian devs here 🙂

[tropic helm]

Google'd the dev:

One week ago he left.

[vale harbor]

I'd give it a try, I think there's something worthy in their HDR implementation

[vale harbor]

he clearly left so he could join our discord full time

and mod games with us!

[vale harbor]

like colors look off, but the rest looks gr8

I wonder how it would look with DICE 🙂

I could try, but I will not start anything new until I'm done with Luma

[tropic helm]

https://www.linkedin.com/posts/jean-michel-gilbert-a755657_today-marked-my-last-day-at-larian-studios-activity-7164501971217952768-mmjl

I never said it but, once that work was completed, I started to miss our HDR and color grading calls 🙂

There will be plenty more HDR here

[bleak siren]

Larian were using lots of ACES stuff I think, I saw someone from there posting on the ACES forum a while back

Above and beyond what I had read about in gaming before I think

Just found them, it is him. I bet he has some good insights and advice

[tropic helm]

i figured out buffer injection in D3D11 (and likely D3D10/9)

so i can make sliders now.

unreal and unity use all 14 buffers, i can just use up a buffer i know is not in use at the time of draw. (like buffer 14)

[quartz wraith]

The game looks amazing in HDR now! Thanks again!

[vale harbor]

Damn. Gonna have to buy it to try it now, my pass trial ended yesterday

[quartz wraith]

It's a lot of fun with a crew

[slim jungle]

amazing that this thread exists right when I want to play it after a few years of dropping it. Question though since a lot of this is over my head as I'm still new to the HDR world. I just need to install reshade w/ addons and use ShortFuse's addon64 file with SoT right?

[tropic helm]

Hi. You need a pre-release version of Reshade you can grab here: #1204850050530807838 message

[last sparrow]

Should we be concerned about possible anti-cheat violations by using it? Rare doesn't have much anti cheat yet, but they have done rounds of bans and they are working on a proper anti cheat system in the somewhat near future.

Oh and does it work for both DX11 and DX12?

[slim jungle]

Wow nice work, completely fixed it. Note: having rtxhdr enabled globally did some weird stuff and made the image washed out so had to disable it for this game.

[quartz wraith]

You have to be reported by other players to get banned atm. I think it’s safe to use for now.

[vale harbor]

they might be jealous of us having better HDR and report us 😄

[quartz wraith]

True 🤣

[tropic helm]

I haven't tested DX11, but I think it'll work in theory. If not I know how to mod DX11 games now.

[quartz wraith]

With the new diving system high seas is actually fun now, you can avoid other players. Smuggling stuff in a solo sloop without getting caught is fun 😄

[vale harbor]

wouldn't the shaders be different?

[tropic helm]

It's a SM5 shader in DX12 right now. So I think DX11 is the same shader. I split with cbuffer injection but I'm not injecting any buffers into sea of thieves at the moment.

[tropic helm]

works in DX11. no need to change anything

[last sparrow]

https://youtu.be/fjNoopIn-IY?si=43Oep39n1T0oy2wq

So looks like Rare is adding Easy Anti Cheat with next week's update. Does that affect Reshade?

[tropic helm]

probably breaks it

probably worth filing an issue saying the HDR can't be fixed now

well, after it happens

[last sparrow]

Well then even more reason to ask Rare to fix HDR since you already solved it. :)

I'll try to show the examples of their current issue and how it should look.

With what you learned and fixed, is it something that Rare could apply similarly? Or does Reshade bypass the issue entirely?

[tropic helm]

i only changed this:

#if RENODX_SOT_ACES_TONEMAPPER
      r1.rgb *= 8.f;  // Scale up close to SDR;

      float hdrBrightness = pow(finalGain, 0.25f);  // 0-2
      float userPaperWhite = 200.f * hdrBrightness;

      float3 ap0Color = mul(BT2020_2_AP0_MAT, r1.rgb);
      float3 tonemappedColor = aces_odt(
        aces_rrt(ap0Color),
        0.0001f,  // minY
        48.f * (userPeakNits / userPaperWhite),
        AP1_2_BT2020_MAT
      );
      r4.rgb = tonemappedColor.rgb * userPeakNits / 10000.f;
#else

      r5.rgb = r0.yzw * 0.4518f + 0.03f;
      r5.rgb = r5.rgb * r1.rgb;
      r0.yzw = r0.yzw * 0.4374f + 0.59f;
      r0.yzw = r1.rgb * r0.yzw + 0.4f;
      r0.yzw = saturate(r5.rgb / r0.yzw);

      float newPeak = (2 * userPeakNits) / 10000.f;  // Maybe to stretch contrast?
      r0.rgb = r0.yzw * newPeak;

      r4.rgb = finalGain * r0.rgb;
#endif

They use Unreal and can use Unreal's ACES function and apply the same fix

[quartz wraith]

It will take 3 to 6 years for Rare to fix it

[tropic helm]

It might be worth taking comparison shots now for when the game ultimately breaks to file the issue with the devs.

[last sparrow]

I've unfortunately been busy lately, but I went through the normal game without reshade installed, looking for as many [bad] HDR examples I could find and got about 400 (~800 with JXR + PNG) screen shots of SDR, HDR 50/50, HDR-60/40, and HDR-80/40 for the HDR brightness/contrast sliders.

When I did install reshade with the addon I couldn't take screen shots and I wasn't 100% sure the addon was working correctly or not despite Reshade showing that it was in fact active. The tone mapping seemed better than normal but it still appeared darker and color more exaggerated than some of the examples above, so I didn't know what to think and lack of sleep wasn't helping :)

In any case, the servers are shutting down soon for this update with Easy Anti Cheat, so I had to call it quits and reset everything just in case.

Here's the folder of the 400 screen shots for posterity: https://1drv.ms/f/s!ApgMd8bkJ9mcgZOHKIWXuy6eyJ3gabw?e=pBOUBE

[quartz wraith]

Cant check the screenshots atm but you need the special fork version of reshade, posted by ShortFuse

and the game looked "perfect" with the addon

I will test reshade with the new anti-cheat

The game is almost unplayable now without the HDR fix imo 😄

[zealous shale]

Easy Anti Cheat doesn't like reshade at all, I doubt add on will work any more unless the game will also have offline mode like Halo MCC

[quartz wraith]

thats a shame

[tropic helm]

they said there will be a solo mode i think

[quartz wraith]

like offline?

safer seas is okayish, but it's a very limited game mode atm

[tropic helm]

Alongside the update that is introducing anti-cheat, solo Xbox players will be able to jump into the game's Safer Seas mode without a Game Pass Core or Game Pass Ultimate subscription. For those that haven't heard of the Safe Seas mode, it's essentially a private server that you can play with just your friends, meaning you won't run into other players online. It was a great addition that arrived last year for those that just want a quieter pirate experience.

would be interesting if it works with the mod at least

[quartz wraith]

nothing new then unfortunately

PC version is subscription-free

[last sparrow]

Yeah I used the one that was linked to and at first couldn't get the add on to show, so I tried the add on release version of Reshade and eventually went back to the linked version, but with all my back and forth maybe it was using the wrong build of Reshade.

[last sparrow]

That's what I thought from the examples shown before so when it seemed better but still had issues I wasn't sure what was the cause exactly. I tried disabling the add on and restarting but it felt like I was gaslighting myself to tell if it was better or not.

[quartz wraith]

yup the game doesnt even start with reshade

[tropic helm]

i assume the new update launched?

how about an official reshade? sometimes anti-cheat allows those

[quartz wraith]

it did

[quartz wraith]

hmm leme check

game launches but reshade is not working

[tropic helm]

i don't think addons will work but maybe fxs

[quartz wraith]

I mean reshade is not hooking

[winged lake]

Can you guys summarize me of what are you discussing about? Will turning on HDR without tweaks work well on SoT? I have AW3423DWF

[quartz wraith]

The fix is no longer works.

We can’t use reshade since the anti-cheat update

I don’t recommend turning on HDR without the tweaks

[slim jungle]

It's a shame because I can't even get RTX HDR to work now. Bummer because the fix made SoT look amazing in HDR

[quartz wraith]

It is yea 😦

[pulsar sigil]

RTX HDR somehow randomly works from time to time. Can't find the reason why it refuses to do it's job

[last sparrow]

https://youtu.be/ko0hcXzyvS8?si=rSplX4jlxDpaslJy at 5:02 Rare has now publically announced their work to improve HDR coming to Insider Builds. I'd recommend (ask) those who have the best knowledge on HDR to test it out when it's available to make sure Rare is implementing it correctly.

Also reminder that Insider Builds/Program require an NDA so nothing can be discussed except for the specific details that are publically announced like this in the video. So if you do join Insiders put all discussions on the Sea of Thieves Insider Program forums.

[tropic helm]

is there a discord where i can chat with devs

doing it through their website

rip

progress

[tropic helm]

can't access the forums yet. i'll try it later

[last sparrow]

There is an Official SoT Discord but they forbid any NDA disclosure and I'm don't know if the devs use it to communicate directly besides server announcements/community members.

So at the moment it's just the Insider Forums that I'm aware of, which i don't know why they are broken for you.

I did want to get a list of anyone willing to talk to devs if they are interested in speaking directly.

[tropic helm]

maybe i have to log in with the steam build https://www.seaofthieves.com/community/forums/category/90/insider-programme-hub is just a 404 for me

[last sparrow]

@tropic helm Two months later, it's finally a good time to check out the latest Insider Build and give them feedback.

[karmic berry]

Did Rare fixed anything?

[deep socket]

I just reinstalled the game after years, tried HDR, but when I set the appropriate peak brightness (500 nits in my case), average picture level was quite dim. Didn't try messing around with HDR brightness because people say it just shifts everything.

[tropic helm]

I fixed the HDR a while ago but the game got Easy Anti Cheat and blocks it.

[deep socket]

yeah I know

[quartz wraith]

▫️Sea of Thieves