Self modifyable executeable and intrinsics?

[west drift]

so recently my friend introduced me to intrinsics (AVX2 etc), and honestly, I've been busting my head nonstop how I could automate it at runtime. That is, to check if CPU has an instruction and if it does, it uses it, but if it doesn't it uses regular methods.

Ofc, since the point of it is optimization, I want to avoid ifs as much as possible.

My first idea was to write DLLs with and without intrinsics, and then in an init function I load appropriate DLL, which my friend said is a bad idea as it would be function pointers which would also have performance overhead.

Today I was talking to my friend and I asked him if it would be possible to at runtime replace a function with another function without the use of function pointers, and he mentioned self-modifyable code, but said he knows nothing about it.

So I was wondering, how would I do it?

eg. I have operator+ overload for __m256, and it calls _m256_add_ps, but in init function I want to replace it with a regular function that adds them sequentialy in case cpu doesn't support AVX2

ofc none of those info is know at compile time, which is the source of my problems

[tepid grotto]

When your question is answered use !solved to mark the question as resolved.

Remember to ask specific questions, provide necessary details, and reduce your question to its simplest form. For more information use !howto ask.

[daring reef]

@west drift https://mohitmv.github.io/blog/CTwik-General-Purpose-Hot-Patcher-For-Cpp/ you may be interested in this library

[west drift]

if I read that correctly, it also uses self modyfiable code, but that's not completelly what I'm after. That's basically a debug tool, like the hot patch in visual studio where you can just add changes at runtime as long as they are not in a global scope

[daring reef]

Yes, self modifying code is definitely only suitable for debug/testing because it is a big security risk

You would probably just have to branch as early as possible into code that contains whatever vectorization is necessary - i believe this is how most applications do this but I'm not sure since i've not done vectorization before

[dusk remnant]

Checkout stockfish makefile

https://github.com/official-stockfish/Stockfish/blob/master/src/Makefile

But that isn’t runtime though you can set these during compilation and then in your actual code check if it is defined

And here there are switches what to use

https://github.com/official-stockfish/Stockfish/blob/master/src/simd.h

Though it is much more than avx

Though modern compilers have auto vectorization (if you pass the correct compilation flag)

So my guess this doing all this manually adds a 5% speedup

Maybe more if certain hot code paths don’t get auto vectorized

[latent topaz]

the thing is: in order to really take advantage of these SIMD instructions, you're going to want to use them for bulk operations. the overhead of an indirect function call or an if should then generally not really matter.

especially if you consider that the if would be 100% predictable, so it's gonna be essentially free on a modern CPU

another option would be to just provide multiple .exe files and run the appropriate one based on what machine you're on

self-modifying code would also be possible ofc, but that's gonna be insanely complex to pull off and probably not gonna scale well to larger projects.

[west drift]

yes, they have vectorization, but that then requires you to have that instruction set , it's not checking if you have the instruction set, it just compiles assuming you do have it because you told it to

[west drift]

I'll have to do some benchmarking, thank you for the answer

[dusk remnant]

Well if you compile with march native it gets all the instruction sets accessible and uses these when you have O3 though march native will not get you builds that are compatible with other systems

Unless it has the same instruction set

Otherwise I think you can see what march flags native activates and then use this to define stuff

And then in your code use different instruction similar to what stockfish does

[west drift]

seems like the discussion is pretty much at a standstill so I'll close it now, thank you all for your answers, I'll try to think of more ways, but these suggestion are also great

!solved

[tepid grotto]

Thank you and let us know if you have any more questions!

[SOLVED] Self modifyable executeable and intrinsics?

[tepid grotto]

@west drift

This question thread is being automatically marked as solved.