Unity provides an API to compile your custom variants for shaders by reserving a keyword. Namely, the API is defined at Data/PluginAPI/IUnityShaderCompilerAccess.h. There is also a documentation for it, although it is not really extensive https://docs.unity3d.com/2022.3/Documentation/Manual/LowLevelNativePluginShaderCompilerAccess.html. But I couldn't understand how it works exactly.
I managed to compile my own plugin and catch the kUnityShaderCompilerExtEventPluginConfigure event inside the exported UnityShaderCompilerExtEvent function, in which I specified the reserved keyword, platform and supported shader programs. So far so good. But then, kUnityShaderCompilerExtEventCreateCustomSourceVariant event is never called. Instead, only kUnityShaderCompilerExtEventCreateCustomBinaryVariant is called. And I am also confused about what the plugin can do with the provided argument when compile binary variant is called:
struct UnityShaderCompilerExtCustomBinaryVariantParams
{
void** outputBinaryShader; // output of the plugin generated binary shader (platform dependent)
const unsigned char* inputByteCode; // the shader byteCode (platform dependent)
unsigned int inputByteCodeSize; // shader bytecode size
unsigned int programTypeMask; // a mask of UnityShaderCompilerExtGPUProgram values
UnityShaderCompilerExtCompilerPlatform platform; // compiler platform
};
It gets an input bytecode and outputs a bytecode? I thought it would get the source code of the shader and output the compiled bytecode. And why is the output bytecode of type void**? how do I specify the output size? I might be misunderstanding the purpose of this API, but if someone could clarify what exactly it does and is used for, I would appreciate it.