Even if you met all dlc requirements, I am not sure if the launcher or the game can make you leave your current server and join another one
#arma3_tools
vague shard
@elfin oxide see above: https://discordapp.com/channels/105462288051380224/105464579600977920/584329737556590602
elfin oxide
Oh okay, did not know about that. Cool
cinder meteor
Can someone clarify for me, if I am building a .pbo with pboproject, when does the preprocessor run? When arma loads the addon?
I have heard somewhere that it has its own preprocessor but I am not sure
https://github.com/acemod/arma-project-template/blob/e6a69d03837a278ec2d0fc92c65192ec55044dc4/addons/main/script_component.hpp#L2
This thing confuses me, so it's including it from P drive, right? Meaning that it runs at .pbo assembly time?
glossy inlet
configs are binarized
binarized configs have no preprocess stuff so that needs to be done before they are binarized and packed
SQF files are not touched
"This thing confuses me, so it's including it from P drive, right?" for config.cpp/bin yes.
cinder meteor
cool, thanks!
so it means I can't really have dependencies for include files from base arma for instance?
because I must update arma contents in the P drive when arma updates?
or rebuild my addon that depends on an include from another addon
I don't get it. I do some plain define like #define stuff 3, then I do some stuffValue = stuff; inside cfgMagazines.hpp which is included into config.cpp, but inside the .pbo it is still not preprocessed, so it doesn't do stuffValue = 3;, it still remains stuffValue = stuff; 🤷
scenic canopy
you should always update your P drive after every patch
files change and are removed
and some files required by buldozer are updated quite often
glossy inlet
"so it means I can't really have dependencies for include files from base arma for instance?" you can. Put it on your pdrive (or DokanPbo hint hint)
"because I must update arma contents in the P drive when arma updates" you have to do that anyway
"or rebuild my addon that depends on an include from another addon" that too
@cinder meteor what do you get inside the pbo?
it still remains stuffValue = stuff; can't be Might be stuffValue = "stuff";
cinder meteor
it still remains stuffValue = stuff; can't be Might be stuffValue = "stuff";
no, that's what I've got
inside the .pbo I have exactly same files as source files, with all includes, etc, like this:
#define myStuffValue 3
class CfgMagazines
{
class CA_Magazine;
class vin_document_0: CA_Magazine
{
mass=myStuffValue;
...
so my friend has set up the addon based on ace addon template, he builds it allright, but when I build it myself, arma errors with include file "z\...\ ... \script_component.hpp" not found
glossy inlet
Do you have P:\z\...\...\script_component.hpp ?
cinder meteor
Yes I do, but I don't understand:
mikero's tools would have produced an error if I didn't have it?
you have said previously that it's supposed to preprocess it at pbo packing time, why would arma try to include it?
glossy inlet
Yes if mikeros tools did actually preprocess it would've errored
Did you manually disable binarization for pboProject?
Afaik you can only disable it with a command line switch, so I don't understand why it seems to not binarize the config
cinder meteor
as I understand, build.py makes unbinirized config
make.py, instead, makes config.bin
glossy inlet
ah yes.
correct, so you do actually set the special command line parameters to explicitly disable binarization
If you disable it, then it won't be preprocessed at build time
cinder meteor
will arma preprocess it then? I don't understand, how is preprocessing of addon configs supposed to work originally in arma, without mikero's tools? or are they the official tools and the official way it's supposed to work?
glossy inlet
Official arma tools also binarize configs at build time
If you give Arma a text file, then it has to load the text file
and loading the text file includes preprocessing
cinder meteor
oh ok, so it will preprocess it anyway
dawn palm
my tools it will NOT process sqX >files< It does process sqX statements embedded (eg) as eval or exec statements inside a paramfile (config. cpp/mission.sqm/etc), but not, (dot) sqf or sqs.
if your 'friend' is working ok, the answer is reasonably obvious, you dont' have the same pbos he has, loaded, at game time.
cinder meteor
Hi! The problem was that I didn't un-exclude .hpp from the .pbo in pboproject settings
glossy inlet
I just got arma3.ovh domain. If anyone needs a subdomain for any of your projects feel free to contact me.
tfar and sqf subdomains are already taken 😄
If you just need a http redirect I can host that myself
would also host simple redirects to workshop if anyone just wants a short link to their workshop mod
nocturne basin
would need to create some info page for SQF-VM first
or get started on working at it again
or ... stuff
obsidian sluice
glossy inlet
Might aswell just make a redirect to the discord invite. That's what I do for tfar for now
nocturne basin
🤔 🤷 😄
nocturne basin
karmic niche
Just buy Morocco's Ar.ma domain 😛
But you may end up like that relatively unknown arma3.com domain: https://web.archive.org/web/20110210111702/http://arma3.com/
glossy inlet
I just needed it for short link in a upcoming tfar update. Might aswell give subdomains away for free if anyone wants them
neon flax
awesome.arma3.ovh to https://github.com/veteran29/awesome-arma3 ? :D
(I really need to start updating this list again, PRs welcome ^^)
glossy inlet
done.
Tools/Mods stuff. Maybe a section for server management stuff?
TADST and others. extDB/IDB. rcon tools.
karmic niche
tbh I'm not really convinced that advertising the OVH company each time we'll be mentioning arma community stuff is the best idea 😛
glossy inlet
https://awesome.arma3.io ¯_(ツ)_/¯
With full SSL support
Uh. Even shows the Github embed
I also support full delegation if you need.
So redirect *.something.arma3.io to your dns server
neon flax
awesome 
glossy inlet
Added: RVExtensionRegisterCallback and "ExtensionCallback" mission event handler
OwO
native kiln
neon flax
So does this allow extensions to fire multiple registered handlers etc?
glossy inlet
I assume the EH get's something like param ["_eventName", "_stringdata"]
Intercept is still superior, but this is great! Truely async things
native kiln
What a really random nice thing to see though
Is this something that has been on their Todo list for years now?
glossy inlet
I have seen a old scripting todo list from Dwarden, that wasn't on there
Also heard no rumors or anything about it, no idea where that's coming from all of the sudden
nocturne basin
@glossy inlet where RVExtensionRegisterCallback
need to update biki and sqf-vm then accordingly
glossy inlet
dev branch.2 minutes before I posted. no docs yet. I'll see tomorrow
nocturne basin
well ... the moment i get some intel, i can implement it into SQF-VM
so one can fiddle around with it
and write docs
on biki
scenic canopy
cool stuff
karmic niche
This is relevant to my interests! o_o
scenic canopy
string data would be serialized sqf-ish data structures I assume?
karmic niche
I don't see any other way tbh
nocturne basin
gonna be very wonky i think
i mean ... unless i missunderstood something, the interface could be some way to literally call back to the game from an extension
karmic niche
Because if they could, they would have already implemented that in the regular callExtension
native kiln
i can imagine it only works with a string
nocturne basin
soooo .... RVExtensionRegisterCallback should pass some sort of method pointer that one can call any time to "give control" to the game
probably will be delayed and just "push data to queue" but still
glossy inlet
I assume same string stuff as callExtension. Can't think of anything else
karmic niche
or string that gets automatically parsed to code <dreams>
[in a faster way, without hitting SQF's functions]
glossy inlet
They are probably doing some scheduling. When you callback they just push it to a queue to be processed next frame
Otherwise you'd have to be in mainthread and stuff
native kiln
polling: be gone
glossy inlet
what did I make InterceptDB for now (╯°□°）╯︵ ┻━┻
nocturne basin
┬─┬ ノ( ゜-゜ノ)
wind elm
(╯°□°）╯︵ ┻━┻
native kiln
tfw they finally add the good stuff https://s3.eu-central-1.amazonaws.com/trski-storage-public/ShareX/2019/06/2019-06-04_Zx0kQs5RjIBWCjcu9Ei.png
nocturne basin
chill
interceptDB still is miles ahead from literally everything callExtension could realize
native kiln
Dscha flipping tables, no surprise
nocturne basin
@karmic niche chances are you can register something like "callbackcode" and then just in there just take the string passed and straigth compile + call it 🤷
karmic niche
That's what I'm doing with Pythia already
But it would be good if you'd receive code without having to actually call/compile it on the SQF side
Saves a few 0.0001ms
Otherwise... well... not much gain from that 😄
scenic canopy
ideally BI would offer some header with sqf data structures? 😄
nocturne basin
you dream too much
nocturne basin
though ... it would be enough if they just created a copy of their data and made that public
scenic canopy
it's almost like an external developers wet dream
there's a lot of different data types though
nocturne basin
not even needed to create something new, grab some open source project that funny enough is a custom implementation of sqf and copy-cat the stuff into and from internal data structures into that
scenic canopy
if you want the custom types such as location etc
nocturne basin
shameless self advertising
scenic canopy
it wouldn't be too hard to interpret the string into something more sane to work with either so...
nocturne basin
yup ... made some trivial code to do that years ago
should rewrite that stuff at some point though 🤔
smoky halo
ExtensionCallback has one param _this which is a char const * you pass from extension converted to RString
The signature of the callback function is int(*callback)(char const *param);
scenic canopy
@glossy inlet there's no intercept.arma3.io !!
smoky halo
Returned int is the number from -1 to 99
When you call Arma the argument is added to array and the remaining size returned. Array is 100 long so 1st call gets you 99 left when 100 args entered return is -1
Array is cleared after simulation which is next frame after callExtension call
Can handle shit load of extensions calling at the same time
The EH is really a function to construct own dispatch of received data to function
nocturne basin
literally understood nothing but some artificial limit and fancy wonder bra
prototyping required, or the actual source
void RVExtensionRegisterCallback(???, int(*callback)(char const *param), ???);
``````sqf
addMissionEventHandler ["ExtensionCallback", { params ["_value", ???]; ... }]```???Array is cleared on mission end and on removeAllMissionEventHandlers "ExtensionCallback", which you probably want to do anyway before adding EH in case you have something written in array in between, otherwise EH will fire as soon as you added it with all data currently in array
Not on PC will tell you later @nocturne basin
In 5 hours
smoky halo
Traffic is a bitch
glossy inlet
@scenic canopy fixed
I understand M242 like this
int(*callback)(char const * _this) RVExtensionRegisterCallback();
Might also have a ** as argument and expects you to fill it, but I think return makes more sense.
Though return would limit to one callback per extension
addMissionEventHandler ["ExtensionCallback", { params ["_this"]; ... }]
It's ironic that on one side, they tell you that they are not planning to add any extension support to DayZ at all and, on the other hand, they tease you with extending extension support for Arma 😦
smoky halo
No RVExtensionRegisterCallback is called by the engine before everything with function address, the argument is function pointer you need to assign to your own variable
vague shard
@karmic niche its done on personal initiative by one guy
no relation to BI plans/prios/tasks
smoky halo
What kju said
karmic niche
I can guess that it's the case. Still sad 😦
vague shard
"just" need the same to happen there too 😉
karmic niche
the one guy doing something on personal initiative > we still love you!
vague shard
well this happens often with BI
a lot of community modding stuff was done that way
helps if you have good amount of ex community guys in the company
smoky halo
Can’t complain, something is better than nothing
glossy inlet
"its done on personal initiative by one guy" same as my aWWACG and flashlight fix and others on my list.
But who is that one guy that gave us this jewel?
vague shard
not for me to tell
nocturne basin
better send him some better motivation
this implementation, even though appreciated, feels like there is a hell lotta room for improvement
and the literally horrifying nature of enscript having no extension support because blah ... yeah ... enough said about that already
vague shard
@nocturne basin make a BIF thread and list them
nocturne basin
mhh?
glossy inlet
So like this then
int(*EngineCallback)(char const * _this);
void RVExtensionRegisterCallback(int(*callback)(char const * _this) {
EngineCallback = callback;
};
EngineCallback("My stuff I wanna tell you")
addMissionEventHandler ["ExtensionCallback", { params ["_this"]; ... }]
Not having seperate name in the handler makes it a whole lot harder to parse the result.
And you have to take precautions for compatability between multiple extensions as other peoples format might look similar to yours
make a BIF thread and list them
Guess you can post problems/suggestions in dev-branch feedback thread.
nocturne basin
personally, i think it might be even worse that the -1 may only be received by attemting to write to the callback
a separate check wether or not one can write would be good
as that avoids having to add a separate buffer
glossy inlet
separate buffer? I assume Arma is copying the string you give it
And if you want to fire a callback, you will do it anyway.
So if it returns -1, push it onto a queue for next frame. So building the callback string that you're trying to send also won't be a waste
nocturne basin
int(*EngineCallback)(char const * _this);
int(*EngineCallbackCanAdd)(void);
void RVExtensionRegisterCallback(int(*callback)(char const * _this, int(*canadd)(void) {
EngineCallback = callback;
EngineCallbackCanAdd = canadd;
};
if (EngineCallbackCanAdd("stuff")) then
{
//... get result
EngineCallbackAdd("stuff");
}
though .... wait ... concurrent problems still arise as there is no way to lock
so forget that ...
glossy inlet
If callback returned -1 push onto a queue that a thread checks every second or so, and the thread then tries to re-do the callback
nocturne basin
end solution thus requires a buffer any way 🤷
something like:
#include <stdbool.h>
// May require mutex if more then one thread uses theese
char* message_buffer;
bool can_write(void)
{
if (message_buffer == NULL)
{
return true;
}
else
{
if (write(message_buffer) != -1)
{
free(message_buffer);
message_buffer = NULL;
return true;
}
return false;
}
}
bool write(const char* message)
{
int res = EngineCallback(message);
if (res == -1)
{
// Copy string into message_buffer
[...]
return false;
}
return true;
}
there is a threadless solution @glossy inlet
smoky halo
It is every script simulation not every second @glossy inlet
You don’t need mutex @nocturne basin
nocturne basin
if you use more then one thread in your very own extension to use theese two specific methods, you do
smoky halo
Can abuse the shit out of function pointer
nocturne basin
the EngineCallback, i do not care about with the mutex 😛
smoky halo
No you don’t trust me not how this was done
nocturne basin
it is about concurrent access to the message_buffer in the code i provided above as sorta example
smoky halo
Ah ok
nocturne basin
how one could solve the "problem" with concurrent access, -1 and tasks that need to be repeated
glossy inlet
"It is every script simulation not every second" I know, But can't see when script cycle is done in extension
nocturne basin
aka:
- call
can_write()
2.1 if not, resume in your fancy thread loop that takes us to sleep for a moment
2.2 if yes, start receiving the result - call
write(...)and continue in your fancy thread loop
smoky halo
When multiple extensions call back you can just while (callback("whatever") < 0);
smoky halo
Guaranteed to get it through this frame or another
Nah array is cleared every simulation
You will have to wait 1 arma frame at most
glossy inlet
Bombarding a arma function which probably has to acquire/release a mutex internally with millions of calls and constantly burning your cpu at 100% is not a good idea.
So... add a sleep 😄
karmic niche
Quick question: what does "every simulation" really mean?
glossy inlet
once per frame. Like eachFrame and scheduler
smoky halo
I’d not bother with sleep you have buffer 100 long per frame, you will need 100 extensions or threads to write at once before you have to wait
nocturne basin
or 100 messages in a few ms
smoky halo
Yeah but why?
nocturne basin
in any way, if you do need to wait, wait a few ms simply because nobody likes to brew coffee with their CPUs
smoky halo
We are talking like mega extreme edge cases here, not worth it
nocturne basin
we do not
if you receive 1000 rows from a DB as example
you cannot just add all of them separate to the game
as after 100, you hit the dead-end
smoky halo
100 rows you pack in an array
nocturne basin
and then you want to idle at 100% CPU until the game says "i am done"?
smoky halo
And send it as string array
nocturne basin
why would i? if i am able to process them properly, i will send them to the client properly
and running at 100% CPU (which is the enqueue stuff callback method in a while loop)
will bring your FPS down to a few ms
though.... modern CPUs more likely will spread the task ...
so you just loose a few FPS with a multi-core-monster CPU
still enough to impact the game meaningfull
smoky halo
Ok maybe ask to increase the limit then? Don’t forget you will be running 100 scripts in unscheduled per frame
nocturne basin
that is why you do while (callback("whatever") < 0) { sleep(10); }
instead of just letting it run like rambo
smoky halo
Anyway if you have good suggestions you better come up with them fast
nocturne basin
callback("whatever", "function")
smoky halo
It is miracle dev updated so soon
nocturne basin
- a bloody higher limit ... like seriously ... 100?
adding the extension that send the message to the EH also would be nice to have
smoky halo
Can have it inside "[""watever"", ""function""]" then parseSimpleArray which you would probably need to do anyway
nocturne basin
but that is ugly AF
- some needless additional call
for no other reason then not to touch a simple method
passing an additional string is literally a trivial task
glossy inlet
Can have it inside "[""watever"", ""function""]" then parseSimpleArray which you would probably need to do anyway
Let's say we have 5 extensions with 5 eventhandlers ingame. Let's assume the extension returns a big value.
Now we run parseSimpleArray 5 times, for each callback and parse the WHOOOOLE chunk of data just to find out we don't want it 😄
smoky halo
What limit, 1000? @nocturne basin
nocturne basin
make it open end 😏
glossy inlet
set it in mod config. Makes everyone happy
nocturne basin
why? fear the out-of-memory bug?
100 is only a valid limit if it actually is executing the whole in a single frame
then i would go for some even smaller limit
smoky halo
Because you can crash arma yes and BI doesn’t like crashes
nocturne basin
but if that stuff is scheduled in some way, no
then i cannot see any reason why this should be limited in any way
smoky halo
It executes the whole in a single frame
nocturne basin
as i also can create some extension and just malloc in a loop
karmic niche
if you have good suggestions you better come up with them fast
Do we really need to have this conversation during [EU] work hours? 😛
nocturne basin
as it is BI work hours too @karmic niche
yes
scenic canopy
@glossy inlet so some kind identifier or whatever for the callback would be preferred then?
karmic niche
True, but I'd rather not get fired 😦
glossy inlet
Either give it extension name (then you don't need to store any string that you don't already have) or give ability to specify name when you execute the callback in extension.
I think easiest is to just provide the name of the extension which fired the event, in the MissionEH
smoky halo
Name of extension
Automatically?
Don’t think you can
Having 2 const char * instead of one? One for name one for everything else?
The advantage you can dispatch without parsing result first
And ignore what is not yours
glossy inlet
"Don’t think you can" why not? BI already logs name of extension if you take too long to return, so they have it.
OOOH wait
Understood now
yeah that would require different callback pointers per extension
Yes 2 strings. One for "function" or "name" or "event" and the other for "param"
nocturne basin
nah @glossy inlet
just add some fancy "void*" to the register thing and expect the user to pass it everytime
and be done with it
glossy inlet
just use std::pair ¯_(ツ)_/¯
nocturne basin
nobody needs to now what is inside of the void*
smoky halo
No can’t use std in RV
glossy inlet
I don't see why not
smoky halo
Dunno, rulez
nocturne basin
cuz c not c++
glossy inlet
Arma is C++
nocturne basin
but the interfaces are not
glossy inlet
We don't care about the interface
nocturne basin
we do
glossy inlet
No
nocturne basin
yes
elfin oxide
No
wind elm
🍿
glossy inlet
You don't know what we are talking about it seems
That has nothing to do with the interface (the pair part)
nocturne basin
giving the callback some std::pair stuff
unless i missed something
nocturne basin
then i indeed missed something 🤔
smoky halo
Ok my battery is low, so laterz
native kiln
@wind elm exactly me, all this C talk https://en.meming.world/images/en/thumb/1/17/Monkey_Puppet.jpg/300px-Monkey_Puppet.jpg
scenic canopy
just use setTimeout with callback function and duration like in javascript!!
elfin oxide
native kiln
id just like to tell you theres a special place in gulag for all of you 😄
glossy inlet
scenic canopy
👍
that doc link is ooooold
postgres is on 11, soon 12
seems like only libpq is offically supported now?
libpqxx is moved to http://pqxx.org/development/libpqxx/
smoky halo
#include <thread>
#include <string>
#include <functional>
#include <chrono>
extern "C" __declspec (dllexport) void __stdcall RVExtension(char *output, int outputSize, const char *function);
extern "C" __declspec (dllexport) void __stdcall RVExtensionRegisterCallback(int(*callbackProc)(char const *));
int(*callbackPtr)(char const *) = nullptr;
// or
// std::function<int(char const *)> callbackFnc;
void work(std::string fnc)
{
using namespace std::chrono_literals;
std::this_thread::sleep_for(2s);
callbackPtr(fnc.c_str());
// or
// callbackFnc(fnc.c_str());
}
void __stdcall RVExtensionRegisterCallback(int(*callbackProc)(char const *))
{
callbackPtr = callbackProc;
// or
// callbackFnc = callbackProc;
}
void __stdcall RVExtension(char *output, int outputSize, const char *function)
{
if (!callbackPtr)
return;
// or
// if (!callbackFnc)
// return;
std::thread t(work, function);
t.detach();
}
fallen stone
cpp
smoky halo
thanks
smoky halo
Here is an idea how to compose callback data with other modders in mind. So you want to deal only with your own data and quickly and safely reject all other data. You can compose your data as ["myextname", ["my","data",123]] for example. Then in "ExtensionCallback" EH you do
addMissionEventHandler ["ExtensionCallback",
{
if (_this select [2, 9] isEqualTo "myextname") then
{
parseSimpleArray _this call TAG_fnc_myFunction;
}
}];
this substring is quite fast, if the data is not yours, you quickly ignore it, if yours you pass it further, call or spawn whatever
vague shard
no BIF thread yet, right?
nocturne basin
still ugly AF @smoky halo
- additionally you still need to double check for eg. return "code" stuff
so in the end, this is still not ideal
at best you get 3 strings in _this
- being the extension
- the actual method/blah of that specific extension
- finally the actual data
not to mention that the select method involves literally counting a string
smoky halo
What do you mean “code” @nocturne basin
Ugly or not it was an adhoc addition, so we might not get any additional changes at all, so brace yourself for that
nocturne basin
you got the extension: fancy magic stuff where you have 2 methods that call back to the game
i want this in _this:
["fancyMagicStuff", "magicMethod1", "i am a rainbow horse <3"]
instead of some custom way like this (as that is what it will probably end up be in all extensions):
"fancyMagicStuff:magicMethod1:i am no majestic rainbow horse :("
so brace yourself for that this
the exact wrong attitude
"something is better then nothing, right?" no, something is just something
the current solutions are partially even better, since they allow the SQF code to control when to pull/push + provide more informations
that thing is just some half-hearted implementation
where i even wonder why one sat down to fight for this only to literally not think about it for a minute and start implementing it straigth
fallen stone
Is it yet known exactly how it's implemented?
smoky halo
You don’t seem to understand that the game has no more programmer support apart from crash fixes. You cannot expect any new features added period. Now tell me about wrong attitude for being grateful when shit added. @nocturne basin
nocturne basin
that is the whole point
you somehow fought through the painfull way of adding a feature and then in the end just implemented it without a single proper thought besides "that would be neat"
no actual planning, no nothing
do not get me wrong, i appreciate that thing
but i will not praise something that is not implemented as one would expect it
fallen stone
So we do know exactly how it's implemented? @nocturne basin
nocturne basin
more or less @fallen stone
theoretically it is in "drafting" still
but we all know that means "just wait till next update"
read above conversation from yesterday - today
for more intel 😉
smoky halo
You think it had no planning involved? 😂
native kiln
Remembers they introduced an idle kick without making sure the headless client is not affected
glossy inlet
You can compose your data as ["myextname", ["my","data",123]] for example. @smoky halo I already said why that's shit. "if the data is not yours, you quickly ignore it" yeah.. I'll just "quickly" parseSimpleArray that HUGE array that my extension might return.
I'd recommend to ask the community for opinions if you're gonna pitch it to BI.
For aWWACG I asked people ACE people and in #arma3_scripting to gather all the feedback. And got a couple things that I never thought about but that make alot of sense.
smoky halo
Huh?
Why you want parseSimpleArray?
Just for the record improvements were suggested and rejected
Now what?
smoky halo
Exactly
nocturne basin
and which is why you should stop cheering and only complain instead
never praise, always just critizise
dawn palm
Enoch Origami is now available for subscribers. It contains all usable assets for map making from A irport buildings to Z ebra crossings.
It contains nothing useful for thieves.
It is only usable by map makers who own the dlc. and similarly, users of their (eventually) published map.
NONE of these origami.pbos are requiredaddons for the end user.
Enjoy
smoky halo
from next dev the function pointer exported will be
int(*callbackProc)(char const *name, char const *function, char const *data);
@nocturne basin ^^^
nocturne basin
👌 👌 👌
scenic canopy
smoky halo
does what you want it to do
the idea you pass your extension name with name
sqf function you wanna call or some method id you used in extension as function
data is whatever
glossy inlet
Ah so it will call a SQF function? instead of the eventhandler now?
smoky halo
so if (_this select 0 == "myexten") then {hint "back off, this shit is mine"}
no
it will call EH with 3 args
glossy inlet
Okey! perfect
smoky halo
instead of 1
glossy inlet
Think 2 args would've been enough. But more is almost always better
smoky halo
3 is 👌
glossy inlet
Now you have one as buffer in case people still want to complain 😄
glossy inlet
More efficient too if you directly split extension/function
smoky halo
if (_this select 0 == "myexten") then {parseSimpleArray (_this select 2) call (missionnamespace getvariable [_this select 1, {}])}
glossy inlet
I thoughta bout using TAG'ed names as function, but then you'd always need to check all your func names
vague shard
as predicted BI extended the existing data packs to DZ SA
lets see if Arma mods are following suite or not
neon flax
vague shard
i am posting in regards to tools/porting - not IP discussion
glossy inlet
What does porting to DayZ SA have to do with the Arma tools?
DayZ discord has it's own tools channel
vague shard
arma community is here
glossy inlet
Don't understand, arma community doesn't really make DayZ mods does it?
vague shard
this change of license allows various mods built on BI data to port to DZ SA now
the porting path is from RV4 to "RV3-Enfusion" mix
prisma dragon
for the callextension developers, who somehow missed recent 1.94 dev branch entries:
Added: RVExtensionRegisterCallback and "ExtensionCallback" mission event handler
Fixed: The RVExtensionVersion function definition not matching the Windows one (the "int outputSize" function arg was missing)
Changed: RVExtensionRegisterCallback provides three arguments now: extension name, function name, and data
you can now do more CRAZY stuff with the callExtension callback and it's mission event-handler 
fiery thunder
trying to wrap my head around making use of RVExtensionRegisterCallback...
Is that partciular DLL function called when addMissionEventHandler ["ExtensionCallback", _myCode]; is called?
I'm assuming _myCode is wrapped up in a ptr and that's what you're storing on the DLL to pass your callback data through
glossy inlet
"Is that partciular DLL function called when " no
"is wrapped up in a ptr and that's what you're storing on the DLL to pass your callback data through" no
your dll gets a function pointer when it's loaded.
You can then at any time call that function, arma will put it into a queue, and push the data you passed to the eventhandler on the next simulation cycle
fiery thunder
ahh, okay, so it's a one-time thing when the DLL is loaded
glossy inlet
ye
fiery thunder
A bit less useful that I thought, but I suppose that's nice for an async notification that something else has finished
glossy inlet
no more need to poll
or you could now implement efficient timers.
Instead of checking ingame if X seconds have elapsed, do it in a seperate thread in an extension with 0 performance impact in the game itself
fiery thunder
Yeah, the polling that I'm currently doing would be helped a fair bit
Do you reckon the callback has the same string length limit that the regular callExtension return does?
glossy inlet
no
Very different approaches
One is a buffer that might or might not be filled, the other is just a string that you wanna copy
fiery thunder
I'd be interested to see how a crapton of data sent through the callback deals with multiple EHs all taking their turn with it
glossy inlet
what you mean?
fiery thunder
the function name will help skip ones that doesn't need it - but just the overhead (if any) of the data string on the engine side if it decides to copy it
glossy inlet
The strings are passed by reference, so calling multiple EH's isn't expensive
fiery thunder
okay that's fine
cinder meteor
So if that thing is a queue, I wonder if it's thread safe already :3
fiery thunder
are there in-place string editing script commands?
glossy inlet
And you have 3 parameters you pass. So you can already filter at the first, and not actually process too much data in the EH's
It HAS to be threadsafe
So I think we don't need to ask that question
there is no stringReplace command if you mean that
Tho we might get that in 1.96
fiery thunder
okay, just wondering if reference passing the data string means it can be tweaked by another EH in the queue
glossy inlet
no
there are no commands to edit a string by ref
Hell that would break LOOOOTS of stuff
Did you know, you can modify the input argument into your Extension (callExtension) and at the same time modify the internal string?
So you can actually put output data into your input string
fiery thunder
cool, I've been caught out by arrays beng passed by ref before (and now I depend on it) so just wanted to check
glossy inlet
everything is passed by ref in script 😉
The only difference is that some things have commands that modify them directly instead of copying
fiery thunder
I think only arrays are
karmic niche
@glossy inlet input is actually a const char *
So while you technically can, isn't it the realm of UB?
fiery thunder
well, when you params[]; something passed in it becomes a local variable - unless it's an array, in which case it seems to stay a ref
private _arr = ["abc", "def"];
private _int = 999;
private _str = "hello";
systemChat format ["%1 -- %2 -- %3", _arr, _int, _str];
[_arr, _int, _str] call {
params ["_arr", "_int", "_str"];
_arr pushBack "ghi";
_int = 111;
_str = "goodbye";
};
systemChat format ["%1 -- %2 -- %3", _arr, _int, _str];
"input is actually a const char *" I know
"So while you technically can, isn't it the realm of UB?" I'm sure it's undefined, but we know quite well what's going on
"unless it's an array" arrays also "become a local variable" if you wanna call it that
A reference to the value, is registered as a new variable name
but the reference points to the exact same thing
= sign doesn't modify any value
it always copies a reference
in your case on _int.
the = copies a reference to 111 into _int.
It changes the reference that _int points to. It doesn't modify the underlying value
_var = _var + 1
the + operator creates a new value of type number. And the = operator changes the reference that _var points to, to the new value
_int = 111
_otherint = _int
_yetanotherint = _int
how many instances of the value 111 is in memory now? Just one.
karmic niche
I was talking about potential compiler optimization somewhere in there. That and the fact that even if something works now it may stop working in the next build (although it's not much different from the intercept situation after all :D )
glossy inlet
If you tell the compiler "this pointer is a non-const char* now" then it won't optimize after that
Because it can see what you're doing
It might optimize on Arma engine side before your stuff is called, but what would it optimize? Instead of passing directly just create a full copy? not optimization.
The only optimization it would reasonably do is to pass it directly, which it is already told to do anyway
If it knows that it's a small string, it might store it just in a register instead of passing a pointer, but this is a dll boundary, it cannot do that.
karmic niche
I was thinking something on the arma side after calling callExtension. I've heard of dark magic stuff done by optimizing compilers when they know that "X is guaranteed to be Y from now on" so I won't ever push my luck in this regard :D
glossy inlet
Well, You modify the input string, and as I said everything is passed by reference
if you pass something that comes directly from the compiled script, you manipulate the string in the compiled script, and everytime it executes it will now write your new string
Directly modifying the compiled instructions basically
You definitely have to know what you're doing if you're doing it
smoky halo
Do you reckon the callback has the same string length limit that the regular callExtension return does?
No more limits apart from the actual game string limit of 10000000 https://community.bistudio.com/wiki/String
You are passing a pointer that is copied from into game string
wind coral
after starting a bercon session from a seperate application, is there a way to get the console handle in cpp?
nocturne basin
bercon?
i do know what bacon is
but bercon?
jokes aside though, https://docs.microsoft.com/en-us/windows/console/consoles
wind coral
I mean, if you can find some way to get me a handle on bacon I would reward generously.
Hmm, okay with AttachConsole it seems possible, how would I pickup the PID of the newly created window?
nocturne basin
unless you run some new application, you do not need to get any PID
there is one exclusive console window available for every process (at least i did not yet found a way to run multiple for a single process)
glossy inlet
If it's your own console, then it's your own PID isn't it?
You can search for window by title
nocturne basin
https://github.com/X39/X39_OpenGL_Application/blob/master/src/startPoint.cpp#L255-L274 this is how to create a simple console window that is capable of doing basic IO on windows using the WIN32 API (ignore the rest of the whole repo though)
you could also get more specific but, this tells you all you need to know
as there is this fancy single important line you need (for a window handle to the console): https://docs.microsoft.com/en-us/windows/console/getconsolewindow
wind coral
std::system("BERcon.exe -host 127.0.0.1 -port 2307 -pw @Collister432");
Burn me at the stake for using system, but I wanted to see if it worked. This is what I got so far, and I want to write input to the BERcon.exe window
wind coral
Understood
nocturne basin
(you already noticed you do not get any PID from it 😛 )
wind coral
Yep
nocturne basin
nocturne basin
though ... why do yo u want the window?
wind coral
So I can give it input automatically instead of having to manually type commands in when needed through the tool
nocturne basin
you may then want to check out this: https://docs.microsoft.com/en-us/windows/desktop/procthread/creating-a-child-process-with-redirected-input-and-output
wind coral
Hmmm, okay. Makes my head spin a bit. But thanks for the reads. I will read well.
glossy inlet
I do that with boost::process in my arma server tool to grab and filter RPT output.
It would basically be the same for input
smoky halo
Looks like you are trying to hack be console
Easier to write own client, the protocol is public
native kiln
@wind coral You can try conPTY
Or use a library to connect to rcon
(in the latter case if there's no C++ library, write the implementation yourself or apparently it's also not that hard to use a C# library in C++)
glossy inlet
It's a fun task implementing it yourself. Good learning exercise 😄
nocturne basin
Just to note something for those who have some application with intensive logging going on: Make sure you actually do logging in a separate thread (the output related stuff)!
Currently at work (using NLog (C#) for logging) and everything is logged to console
now added a separate application that is allowing me to filter everything, added a queue and a thread that writes to the output. Main application now is pretty much as fast as it would be without logging at all
karmic niche
async spdlog logger ❤
elfin oxide
Yep spdlog is still my go to solution for c++/CLR logging
smoky halo
If only Arma could write rpt in a separate thread
But then you wouldn't know your game had errors because it would not freeze until it puked into the file
nocturne basin
sees scripters who spawn angry while {true} do { diag_log something; } loops
smoky halo
diag_log and then read and parse the file
the difference between passing data to a worker thread and exiting and waiting until write is finished is huge
uuuuuuuuuuge
rancid relic
I heard that one programmer was working on it but back then, he was assigned to other task without completing it
and some of the designers in company really regret that it didn't get high enough priority
cinder meteor
I'd guess that file IO is still faster than the speed of sqf, no?
Especially if it's buffered and writes in huge chunks anyway?
glossy inlet
No. A slow HDD is not faster than sqf
nocturne basin
*IO Access is pretty much never faster than anything
unless you use a ramdisk, you will always be slower due to sync-IO (and even with ramdisks, there still is the overhead of that filesystem stuff)
cinder meteor
Well idk, I thought that hdd also should have some buffer, no?
nocturne basin
haha
no
though ... kinda ... but that depends on how you access
your harddrive will not have one
and even that is not fully true
but the buffer mostly still would not matter
as the transfer speed to that buffer in your HD is slow
cinder meteor
I think so yeah you are right, a second thread is better
smoky halo
and some of the designers in company really regret that it didn't get high enough priority
Soon to be said about not upgrading to the latest PhysX
rancid relic
well, latest update brought issues than it fixed
I guess upgrade to new version could be done but would probably take +2/3 months to do
smoky halo
Moving off to a separate thread in something as single thread dominated as Arma makes a lot of sense. I have seen however cases where the lock on the log buffer starts to feature as a performance issue in heavily multithreaded programs. The policy around how much you hold in RAM and what you do when it fills can matter a lot in bad scenarios where you might get a log message storm, throwing them away can be just as problematic as getting all threads stuck waiting on the log buffer to clear. Point is that it isn't universally better, there are scenarios where it ends up worse.
nocturne basin
one could create a buffered-buffered logger
method init
start ActualBuffer
for 0 to 10
start ActualBuffersBuffer
method GetActualBuffer
Return ActualBuffersBuffer according to "round robin"
method write
buffer := GetActualBuffer
buffer writeActual
method writeActual
write to ActualBuffer```or a buffered-buffered-buffered buffer
possibilities are endless 🔥 
sacred flume
on my todo list is intercepting the filewrites to log and redirect them to another process
using easyhook
smoky halo
BattlEye might not like it
knotty belfry
i read the whole thread regarding the implementation concerning RVExtensionRegisterCallback, I am so disappointed, I find this implementation weak 😦
glossy inlet
what would you prefer?
knotty belfry
I would have preferred that there is no link with a call extension (cause finaly it s only a defering result function not really an handler function).
knotty belfry
yes, but i don't understand why there is an handler
it could be something like that
"myextension" callbackextension [_mysqfcallbackfunction,[params]];
not really sure, it can works in my case. I have to test the current implementation
cause when we currently ask a dll a result, we have to insure the result is given in the right order
glossy inlet
What you propose is what we already have. But with the first parameter checked in engine instead of in SQF
I don't see the need
knotty belfry
i don't know, for me the design is simplier
smoky halo
You are talking about callExtensionAsync which has nothing to do with ExtensionCallback. Sorry you unreasonably expected something else.
jolly widget
Updated Rtm Import Addon to work with Blender 2.8 for anyone that needs it.
coarse knoll
Can I ask what an rtm file actually is
Is it the precompilatuin model format
Cause arma uses binarized asset packs doesn't it
pbo
glad mesa
in simple form, it is an animation file.
glossy inlet
rtm file is a animation
sly skiff
thanks @jolly widget
vague shard
@native kiln gratz to the API release 👍
ref: https://ryantt.github.io/wsc/docs
can you fetch the item history somehow? i recall one can do somewhat
if so, you could offer version support as paid feature
native kiln
@vague shard thanks
do you mean the file history or the history of stuff like the description?
atm I don't keep track of changes like the description of an item changing, but I do keep track of when a new version is uploaded
you can look at the file list of any version of an item and download any version of it too aswell as specific files of a specific version
vague shard
@plush shard didnt you mention you can go back to older versions if one knows the id?
native kiln
if youre referring to the files, yes
when you get an item with https://ryantt.github.io/wsc/docs#tag/Items/paths/~1items~1{id}/get you get a version list at the bottom of the json
each version also has the manifestId, you can use that to download that specific version from steam
vague shard
like as author you can revert versions "now" on workshop
native kiln
oh?
native kiln
hmm, well all i know is that im logging the important ids, so i know all necessary information about an item per uploaded version
i think he probably meant the manifest ID then
scenic canopy
yes, it's different manifests for each version
native kiln
hm, but that second number looks too short to the be the manifest id
ah, looks like a unix timestamp
scenic canopy
the id is probably just the change version
native kiln
23.05.2019
scenic canopy
each change contains metadata of what happened, including new manifest id if there are file changes
scenic canopy
sorta
native kiln
when i checked it out, it seemed like for all the appids i tested, depot id was just the appid
scenic canopy
but they belong to an app
native kiln
im wondering how the workshop items actually tie into the whole system in the background
as far as i was concerned until a few days ago, a depot has a version history primarily identified by its manifest id
so, new manifest id comes up, its a new version of that depot, however that doesnt really work for workshop items, as its essentially all manifests of them in that "pseudo" depot
and then a depot also has a build version 
hm i guess each depot version has a build id and manifest id then
plush shard
@vague shard with the manifestid one can access old depot versions - and the workshop of an app counts somehow as one depot - i.e. download_depot 107410 107410 7418351574074791705 downloads you the IFA3_AIO_LITE version from 20.04.2017
it's possible that you have to search a little bit where it has placed the files
glossy inlet
I think something like this is doable right?:
Grab all hashes of all files of all versions of an item, then bundle the hashes together and search for reuploads of them, to find reuploads of old versions of your files too
native kiln
that would work yes
but afaik steam doesnt provide a history of the manifest ids of a workshop item
so atleast i can only index all versions starting from now where im regularly checking if theres a new manifest id
but thats a good idea, ill add that to the query api endpoint
@plush shard you seem to know a lot about the inner workings, would you mind if i pm you a few questions?
plush shard
that's what I am doing on my local ws grabber - but sadly I haven't updated the index regularly
@native kiln sure, but I don't think I know more than the last times
scenic canopy
@glossy inlet hashes are included in the manifest even so that takes away the need to download files, unless you want inspection of course
karmic niche
i can only index all versions starting from now
Remember about making backups @native kiln 😉
native kiln
i cant wait till i dont have enough storage lol
then i will need to migrate all my data to a bigger cluster
vague shard
do you compress the data?
glossy inlet
What stuff takes all that storage? You are not storing binary files for a long time. And text data and hashes shouldn't be that big 🤔
native kiln
i mean its not going to happen anytime soon, i just store some text, but still, eventually i might need to migrate everything
because right now its just on a 200gb SSD
😬
i restructured the output a bit to make it more readable and logically grouped, also theres regex support for filtering what files to scan for now
also @vague shard now you can add the param allVersions=true to take files of all versions of your workshop item into consideration
ive been indexing the a3 workshop since yesterday ~21 oclock now, and now im at 8964/10917 items, gimme a few days and everything will be there 😅
can 2 steamcmd versions run in parallel with different logins?
ive actually forgot i added support for that in one worker instance, so in theory it could be downloading and indexing two things at the same time, but i never bothered checking if 2 steamcmd installations in different directories with different logins somehow hinder each other
vague shard
awesome 👍
native kiln
does anyone have a spare account with arma 3 on it? 😅
damnit, i should have bought more of em back then
scenic canopy
if you pull down all manifests and hashes first you'll know how many unique files you have to do deep inspection of. you could just divide each of those unique files to workers
native kiln
hmm, I don't know how much of a benefit that would bring, and would essentially require a total rewrite of the indexing worker
Pbo files are still the vast majority, so a few bisigns here or there won't be a lot of additional bytes to download per item
scenic canopy
you'd take away a lot of reuploads of cba, ace, island data etc
maybe I should do an estimate of how much that would save 
native kiln
I mean it sounds great on paper
scenic canopy
and if you fetch the data from CDN directly without steamcmd you could download as much as you want in parallel
and ignore all bisign files etc
native kiln
As far as I could see, that c# implementation was really hurting my eyes and SteamKit didn't seem like the most user friendly experience
scenic canopy
use some other library then?
native kiln
Is there any other? :D
scenic canopy
a lot
native kiln
hmm, thinking about it now it would maybe not even require a massive rewrite
scenic canopy
our downloader for CI and Arma worlds metadata is < 100 lines of code
native kiln
But I would need a reliable library for that first, I only really know SteamKit tbh
(c# lib that is)
One could also write the indexer in any other language, but I'm not proficient enough in others :P
scenic canopy
just split it into smaller applications each doing it's part of the pipeline 😛
makes it easier to scale out as well
native kiln
so many ideas, and in the end im just sitting on all of em 
ill just opensource it and someone else helps me out pull this all of 
scenic canopy
perhaps also use some storage which allows you to scale it to multiple machines as well if you need to store lots of metadata
native kiln
luckily im already doing that
scenic canopy
do you source workshop ids using steamworks?
native kiln
nope, all via the web api atm
native kiln
the first
scenic canopy
did some quick stats on the most recent 400 non scenario items
Total files: 82222, Size: 1.31 TB
Unique files: 42383, Size: 335.25 GB
I hope I got it right 😄
I'll need to pull down more manifests to see if the uniqueness increase or decrease
scenic canopy
Number of items processed: 10496
Total files: 1291654, Size: 17.52 TB
Unique files: 426970, Size: 1.28 TB
I'll need to verify those numbers properly after some sleep 😛
my favourite is https://steamcommunity.com/sharedfiles/filedetails/?id=1632768908 which seems to contain a Tom Clancy's Ghost Recon Wildlands 2019.01.22 - 19.28.32.03.mp4, 52.36 GB
smoky halo
How? It says File Size 368.159 MB
plush shard
@smoky halo the filesize shown on the ws pages (and returned by steam's api) is bogus - I guess it never gets updated - the *.manifest files contain names/sha1/size of the content- i.e. one can sum them up to get the real total filesize
karmic niche
When you guys said that it is actually possible to access old versions only if you know the ID, I immediately thought of parasitic file storage: upload your private stuff on the workshop, then overwrite it immediately with something else - a hello world sqf script or sth. You have access to your own stuff for free, while no one else does (and doesn't even know that there is something to download)
plush shard
@scenic canopy yes, the uniqueness is quite low
limited to PBO files of ws entries with mod tag:
datetime(:t, :d) | COUNT(DISTINCT sha1) | COUNT(sha1) | CAST(COUNT(DISTINCT sha1) AS REAL)/COUNT(sha1)
2019-06-24 10:01:57 | 88605 | 729778 | 0.12141363537952637
👍
plush shard
and 242 ws entries (with mod tag) which contain at least one ebo
subtle smelt
Does anyone know of a nodeJS module for STEAMWORKSquery?
scenic canopy
which part of steamworks?
karmic niche
and probably violates the EULA
And it's probably why I called it "parasitic"? 😛
Btw. if anyone is interested in that topic in general, I suggest them to read chapter 16 of "Silence on the wire" (book from 2005) on Parasitic storage.
It's completely unpractical (as opposed to what I've suggested above), but offers an interesting point of view, nevertheless
And yes, I know it's not legal. And no, it's not something that I'm planning to do. I really can afford that 2TB disk, stop looking at me as if I was a serial killer... 🤷
dawn palm
mikero tools update: (subscribers)
+new linux tarball (credit killswitch)
+corrected depew object exports for arma2->3 wrp conversions
karmic niche
Could anyone tell me how are the DLC and Mod hashes computed? The ones available through Server Browser Protocol: https://community.bistudio.com/wiki/Arma_3_ServerBrowserProtocol2
glossy inlet
Are they the same as displayed in RPT?
karmic niche
No idea, would probably have to check that
plush shard
they are modhashs, the Arma 3 Tools have a ModHashCalculator
it's pretty much all the pbo sha1 hashs (the last 20 bytes of a pbo) of a mod sorted, concatenated and hashed again
karmic niche
Thanks!
dawn palm
subscriber tools update:
MoveFolder:
+added DZ as an 'official' folder (credit nirozz)
+created a dual gui/cmdline interface. (credit @dusty imp )
PboProject:
+gives location of the offending temp\source folder if it is detected from a previous crash
+added -W turns off warnings are errors (credit @scenic canopy). -W does not clash with -W=
+minor addition to syntax to allow +/-'Warnings' or +/-W
+autodeleting png files after conversion to paa is now optional in setup-> (credit @sly skiff )
Enjoy
vague shard
can one make a console to work in the main menu and its submenus?
if so, would you need extent each dialog, or could you dynamically create it in any context with ctrlCreate and such?
finally what limitations does one have outside a running mission?
glossy inlet
you can make any UI in main menu.
No idea about creating dialogs.
running scripts outside of mission? Well. there is no player, and createVehicle and such doesn't work, just the obvious things. missionNamespace is available
vague shard
well its mainly to check GUI related stuff, and try to dynamically create GUIs/GUI elements this way as a test for the menus
plush shard
you can by doing some shenanigans with the initDisplay function - I used it to add a displayEventhandler to pretty much every BIS display to get a global hotkey for my console
smoky halo
Main menu is mission AFAIK
vague shard
even with -skipintro?
smoky halo
I don’t know what it does never used it
glossy inlet
skipIntro doesn't matter
-world=empty does
if you don't use -world=empty there is actually a mission running in background
plush shard
I ran into some issues with -skipIntro some time ago - somehow the cfgFunctions were not available in the main menu
vague shard
is there some limitations with sleep/uiSleep/waitUntil/while/spawn in the main menu?
elfin oxide
I'd say that depends on if your function is called or spawned.
glossy inlet
I think scheduled scripts might not be running
unless you have a mission in background
vague shard
hm kinda stuck now
preStart executes in missionNamespace
onload in main menu seems execute in none of the known namespaces
seems to work only if explicitly definined it: uiNamespace setVariable ["DEV_CurrentDisplayIdd",DEV_CurrentDisplayIdd];
vague shard
class RscDisplayMain: RscStandardDisplay
onLoad = "_this call DEV_fnc_onLoadRscDisplayMain;";
starts with
DEV_CurrentDisplayIdd = 0;
uiNamespace setVariable ["DEV_CurrentDisplayIdd",DEV_CurrentDisplayIdd];
i can get with uiNamespace do { systemChat str DEV_CurrentDisplayIdd }
glossy inlet
I think str currentNamespace miiight return the name? probably doesn't
vague shard
however other variables i cant with none of them
currentNamespace only returns namespace for all
glossy inlet
maybe currentNamespace isEqualTo can be used to compare them
vague shard
i did the comparison to detect the type
yes this
my console is missionnamespace according to:
systemChat str [
currentNamespace isEqualTo uiNamespace,
currentNamespace isEqualTo missionNamespace
]; ```what about parsingNamespace?
there are 5 global namespaces, out of which 4 can be accessed via script
But I think from last time I checked, the "default" namespace never had any variables in it
vague shard
uiNamespace setVariable ["DEV_ConsoleActive",DEV_ConsoleActive];
missionNamespace setVariable ["DEV_ConsoleActive",DEV_ConsoleActive];```like if i do this its defined in uiNamespace (only)
glossy inlet
Should definitely run in uiNamespace 🤔
vague shard
it seems like i have to do all with "uiNamespace setVariable" or with uiNamespace do { code }
glossy inlet
all UI eventhandlers run in uiNamespace
and it definitely already exists on Main display load
can you give me a config with a onLoad script? I could check what namespace is the current there
vague shard
preStart
//init.sqf
DEV_fnc_onLoadRscDisplayMain = compile preprocessFileLineNumbers "\DevCon\s\onLoadRscDisplayMain.sqf";```
```cpp
class RscDisplayMain: RscStandardDisplay
onLoad = "_this call DEV_fnc_onLoadRscDisplayMain;";```
```sqf
DEV_CurrentDisplayIdd = 0;
uiNamespace setVariable ["DEV_CurrentDisplayIdd",DEV_CurrentDisplayIdd];
DEV_ConsoleActive = true;
uiNamespace setVariable ["DEV_ConsoleActive",DEV_ConsoleActive];
missionNamespace setVariable ["DEV_ConsoleActive",DEV_ConsoleActive];```i can see the code getting execute via diag_log; my execution also works from the main menu in missionNamespace
glossy inlet
<CfgPatches>
class RscStandardDisplay;
class RscDisplayMain: RscStandardDisplay {
onLoad = "ade_dumpCallstack";
};
Just that?
vague shard
however the code inside onLoad seems not by default uiNamespace but have to explicitly set it
yes
smoky halo
all UI eventhandlers run in uiNamespace
You sure about that @glossy inlet ?
glossy inlet
yeah....
vague shard
like the watcher fields dont work as the context of variables checking the fields seems off
code execution works (and is in missionNS as said)
glossy inlet
Okey actually no. they are all missionNamespace 🤦 I'll better not go into why I thought otherwise as that would end with throwing chairs at people.
missionNamespace get's cleared at mission start, so maybe the mission loads after your onLoad and clears it
smoky halo
missionnamespace setVariable ["test", "missionnamespace"]; uinamespace setVariable ["test", "uinamespace"]; finddisplay 46 displayaddeventhandler ["mouseholding", {hintsilent str test}]
would you like to retract that statement @glossy inlet ?
glossy inlet
I just did
smoky halo
ah ok, ninja
glossy inlet
Who would've thought that "uiNamespace" returns missionNamespace 🤦 (not the script command)
smoky halo
BI sometimes uses with uinamespace do {...} in those EHs
vague shard
is uiNamespace active by default then anywhere, or only explicitly?
glossy inlet
Dang I spent 15 minutes being confused why my stuff doesn't load because I have 2 main.pbo without pboprefix 🤦
https://s.sqf.ovh/devenv_2019-07-01_18-42-30.png varspace is nullpointer. Even though missionNamespace is valid at that point and has 0x70 variables in it
so.. arma is trying to tell me that the uinamespace function returns either missionnamespace or a nullpointer even when missionnamespace is already valid and ready
Seems like we are walking through maximum inconsistent weirdness territory now
https://s.sqf.ovh/devenv_2019-07-01_18-47-52.png there even is still a script error from a script in my profileNamespace in the queue 🙃
vague shard
if i add with uiNamespace do {...} at the end of the prestart sqf file, it seems to convert the whole file/script to uiNamespace
glossy inlet
It changes the VM's currentNamespace
vague shard
but even for previous code of the file/script?
glossy inlet
nope
It only happens when the script line actually runs
the script doesn't know early on what might happen later down
vague shard
i tried this, but it made the main menu no longer know the above in missionNamespace
glossy inlet
actually it sets the current Namespace in the gamestate... 🤦
doesn't change anything here tho.
yeah I'd say just use explicit namespaces
Have you checked if missionNamespace get's cleared when the main menu background mission loads? it get's cleared on mission load, and that might happen after preStart?
But if that's the case it would also break without your "with" there
smoky halo
if i add with uiNamespace do {...} at the end of the prestart sqf file, it seems to convert the whole file/script to uiNamespace
you have to be careful with it, it is ok if everything runs in unscheduled but in scheduled after suspension you may end up in missionNamespace, some stupid bug that we have to live with
vague shard
is there any reason to use uiNamespace beyond storing things longer than the current mission session?
smoky halo
uiNamespace is not serialized
glossy inlet
parsingNamespace also isn't.
Noone uses parsing. stop bullying parsing
native kiln
theres a parsingNamespace??
glossy inlet
:u
native kiln
oh wow, guess i never used it because i dont do much with configs lol
glossy inlet
the config eval namespace should be it
- Replace all
uiNamespacewithparsingNamespace - ???
- Profit
native kiln
3000 items of 11000 items on the workshop done
28.6 GB data already
my 250 GB ssd is already sweating
scenic canopy
the largest workshop item is like 30GB or something?
scenic canopy
my postgres db with all active manifests (including filenames, hashes and size per file) takes up 281 MB
but deep pbo file info would of course take up a lot more
I just threw all manifests as jsonb blobs at it 😄
native kiln
some pbos just straight up contain 8000 files lol
i think once I shape the data into something more efficient I will see the storage drop by quite a bit
scenic canopy
do you need to keep more than filename, hash and size per file inside pbos?
native kiln
just an id per file
right now i have full name and name, but yea thats duplicate data, ill drop the filename field, that will save some data
karmic niche
Nothing to see here, move along, citizen!
native kiln
its so weird seeing so many items pass by where i read "No downloads required" lol
plush shard
my sqlite db has 790MB with 10641 active manifests + 26911 historical manifest data
native kiln
yeah that whole deep indexing really blows up the amount of data
4918 mb mod resulting in 205064 file entries 😅
glossy inlet
do you not de-duplicate file entries?
native kiln
will need to change the data structure for that
however it got a bit more complicated with elasticsearch now
ill look at mongodb when I have the time, but I'm not too familiar with it yet so I chose ES for the task for now
scenic canopy
postgres with jsonb 🤷
https://steamcommunity.com/sharedfiles/filedetails/?id=422777537 contains only "Arma3Tools.exe"
glossy inlet
mariadb all the way
plush shard
some stats:
datetime(:t, :d) | COUNT(DISTINCT sha1) | COUNT(sha1) | CAST(COUNT(DISTINCT sha1) AS REAL)/COUNT(sha1)
2015-07-01 00:00:00 | 60 | 67 | 0.8955223880597015
2016-07-01 00:00:00 | 5814 | 16568 | 0.35091743119266056
2017-07-01 00:00:00 | 36007 | 215926 | 0.1667562035141669
2018-07-01 00:00:00 | 69263 | 565018 | 0.12258547515300397
2019-07-01 00:00:00 | 86307 | 681764 | 0.12659365997617944
only sha1 of pbos
native kiln
@glossy inlet tell me more about that mysterious place
glossy inlet
It's in my backyard
scenic canopy
mariadb is the open mysql development
native kiln
scenic canopy
or rather, the actual continuation of mysql development by the original devs
native kiln
@scenic canopy how fast is postgres when accessing the fields inside the json?
scenic canopy
jsonb fields allows for indexing data which is not supported by json fields
glossy inlet
mariadb also has builtin json support.
I'm using it to store some 40 million text messages :u no problemos so far
scenic canopy
but not index support for it unless you add a virtual column
here's some interesting benchmarks for using json in mongo, mysql and postgres, even if slightly outdated, https://erthalion.info/2017/12/21/advanced-json-benchmarks/
next release of postgres will add concurrent reindexing which is awesome
hosted postgres databases are currently free at scaleway while in beta, https://www.scaleway.com/en/betas/#database
karmic niche
Oh! That's interesting! 🤔
Thanks for sharing that, didn't notice it and I'm using scaleway machines already
scenic canopy
we use a couple of online.net and scaleway stuff
even if hetzner's current lineup is very tempting
oh, hetzner have free setup during july
karmic niche
I simply love their cheapest servers (scaleway)
scenic canopy
time to order a 9900k machine then 😄
anyone made any tools that reads/writes tv4l files?
I see T_D made a reply about the format sometime before releasing the .net libraries (https://github.com/Braini01/bis-file-formats) but I see no mentions of it there
we have an older project with corrupt coordinates for some objects, somehow they've ended up in some weird non float value
I guess just writing 0 as coordinates would make them show up inside terrain builder so we can remove/fix them
plush shard
should be the ALB stuff
if it doesn't work - I have some python script which can rudimentary parse these files - but I think I've never finished it and it's not fit for production
but it would be a start
scenic canopy
got it!
objectInfo = {ObjectInfo} "NaN;NaN;0;0;0;NaN;1;821763"
ID = {int} 821763
Pitch = {float} 0
RelativeElevation = {float} 1
Roll = {float} 0
Scale = {float} NaN
X = {double} NaN
Y = {double} NaN
Yaw = {float} 0
scenic canopy
after fixing scale, x and y they showed up in terrain builder and the wrp worked again!
thanks for the help @plush shard !
vague shard
managed to get the console working now fully - cant say i understand it fully yet
it seems preStart is too early; the main mission runs after that and clears out the namespaces (at least the mission one - to be expected i guess)
i am running now preStart, preInit and postInit via cfgFunctions
foggy vortex
what can be done in pboproject in order to build a pbo regardless of a missing file? It has a beef with me not having RHS sources and trying to build a retexture config (one of the lines uses RHS texture in attributes along with mine)
scenic canopy
-S disables strip
-Gshould disable external references check
previously -X was used
foggy vortex
but these are for makepbo?
foggy vortex
nope, command params are bad, check syntax
scenic canopy
pboproject -P -G +Mod=P:\@mod P:\addon_folder?
+G complained about missing stuff, -G ignored that
foggy vortex
hmm... maybe I have an old version or smth
foggy vortex
btw, there's no option to define inline or per-pbo exception, for example in pboprefix?
scenic canopy
@foggy vortex if it's an old version try with -X instead
native kiln
got everything indexed now, 60 gb 😅
@scenic canopy i also used steams integrated way of getting all workshop items now, works much better 👍
karmic niche
scenic canopy
integrated way as in steamworks?
scenic canopy
that's not using steam, it's a reverse engineered steam client
native kiln
ehh, works good enough for me 😄
i may look into steamworks at a later date, but for now steamkit does the job pretty well
scenic canopy
steamkit allows for single file download, steamworks only allows for an entire item
need to improve the performance of the node library, file downloads caps out at 4-5 MB/s with 100% CPU usage, probably poor crypto implementation
foggy vortex
@glossy inlet can I use dokanpbo in a way so my Arma 3 Projects folder is also on P drive? Looks like it only maps pbo files, so adding this folder to starting line doesnt do much
glossy inlet
yes, you can combine it with a writable directory. It will symlink everything in the writable directory onto the pdrive, and then afterwards link in the pbo's
if you have file path conflicts it will try to resolve it, or throw errors in console if it can't
The... oh.. the parameter is not documented 😄
DokanPbo.exe -f "F:\Steam\SteamApps\common\Arma 3\@CBA_A3\addons" -o P: -w "F:\Steam\SteamApps\common\Arma 3\a3\dokan"
At the end is writable directory. -w parameter
That way you can also write stuff to the drive and have it not be readonly
I'd suggest making a copy of your stuff and trying around a bit
foggy vortex
👍 thanks
takes forever compared to "real" deal but now it all builds nicely. good one @scenic canopy
scenic canopy
Yeah, I need to do some benchmarking
It should in theory be somewhat performant since all metadata is kept in memory
foggy vortex
is there a reason for lack of release builds?
I do have a rider license, so it was not a problem, but setting up monodevelop can be a bitch 😄
glossy inlet
"reason" lack of time or lazyness 😄
scenic canopy
release builds are offered from CI
I can add the link in the readme
I probably forgot to push the wip gui version as well
and my desktop hasn't been reachable since sunday, the remote software crashed or something
foggy vortex
given how many times in last 5 years I deleted github reminders about dangerous dependencies in my projects, I should keep quiet 😄
(shut up github, these are plugins for long dead framework)
dawn palm
btw, there's no option to define inline or per-pbo exception, for example in pboprefix?
pboProject / makepbo <inline.lst
dawn palm
...and pboProject is never, ever going to take your word for it about missing files. It has to see it for itself (and actually runs checks on that file to see if it's contents are valid. Modding is littered with corpses that think they had correct\file\references with no sanity check.
foggy vortex
Yeah, with dokanpbo I'm all set and good to go :+1: , no need for weird solutions for now
vague shard
Since Arma 3 v1.95 it is possible to call the game directly from the extension via function pointer provided when extension is called for the first time. The function pointer passed over to RVExtensionRegisterCallback method is of the following signature ```
<https://community.bistudio.com/wiki?title=callExtension&curid=11718&diff=124873&oldid=123977>i am quite sure i wrote some example down about how to use callExtension relyable (aka: no need to worry about pushing too much)
scenic canopy
there's a couple of examples in history of this channel (and dev channel?)
smoky halo
Yeah I remember, you put brakes on it 😃
keen owl
I just compiled Boost.Asio using Boost.Beast for web sockets. Since it needs OpenSSL for secure web sockets even the optimized release build is 3mb in size... Does anyone have any recommendation for a networking library for extensions? I just would rather not implement my own framing on top of TCP if I can avoid it (especially since I have to do it the server side also).
karmic niche
Wasn't there a networking library that has been opensourced by oculus a few years ago?
keen owl
Maybe you are thinking of RakNet?
scenic canopy
microsoft offers one as well
https://github.com/microsoft/cpprestsdk, including websocket, https://github.com/Microsoft/cpprestsdk/wiki/Web-Socket
but it uses openssl and parts of boost as well
keen owl
Yeah, that gives about the same, 3.5mb in release build...... The price of SSL. (only 1mb release in Boost.Beast if no SSL)
smoky halo
Steamworks API has some socket utils. What do you need it for?
keen owl
Communication from Arma client and custom server
smoky halo
I’d look in Steamworks API as well as you could do some crazy shit with the shit they give you
karmic niche
But what exactly is your issue? Are you complaining about the size? (because you're bringing that 3.5MB up again)
I don't think that in times when everyone has a TFAR installation that is 150MB in size, and/or 10GB of RHS or CUP, that people will complain about a library being 4MB
I'd focus on features and ease of use, instead, when selecting the library you want to use
keen owl
Yeah, I was just curios whether there was smaller library.. For just HTTP one can dynamically link with WinHTTP for only ~50kB.....
@smoky halo When are those libraries available? I thought they were only if you are making a game for Steam platform... Can you use them from an (Arma) extension or otherwise?
smoky halo
Not sure about the commercial aspect of it
How do you launch the game? Via Steamworks API or steam command?
I mean how do you connect to a server?
scenic canopy
for our launcher? get paths to arma install and workshop folders from steamworks api, and then launching the arma binary as a process with needed arguments
glossy inlet
I think you can disable SSL on boost?
If you're hardcore you can go for ZeroMQ which I use in Intercept Network.
There is also a VERY nice library from valve for such things, it's on their github, don't know the name right now.
@karmic niche new TFAR is only 75MB 😉
This is the valve things https://github.com/ValveSoftware/GameNetworkingSockets
Just like ZeroMQ that might be overkill for you
In one of my projects I use json over plain TCP, and to wait till I've received all data, I just wait till I find a \n at the end of a json object
That way you don't really need to worry about splitting and combining and such
If you do custom things then HTTP is too much unnecessary bulk
scenic canopy
cpprestsdk allows for disabling openssl at least
keen owl
Thanks, that GameNetworkingSockets library seems like it fits the need. For a brief moment I was actually considered implementing my own replacement for SSL by using AES with Curve DH on top of TCP but it already has this made.
glossy inlet
Ah didn't know you actually want/need encryption
keen owl
I am having trouble shutting down a thread spawned by my extension dll.. When closing the game I get an access violation when my extension is used. This is my DllMain where I use a mutex and a condition variable to signal the worker thread to shutdown.
case DLL_PROCESS_DETACH:
{
{
std::lock_guard<std::mutex> lock(mtx);
workerStop = true;
}
cv.notify_all();
break;
}```
When creating the worker thread I use this to increment the DLLs reference count to keep the memory alive after Arma removes its own reference.
```cpp
BOOL IncrementDLLReferenceCount(HINSTANCE hinst)
{
HMODULE hmod;
return GetModuleHandleEx(
GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS,
reinterpret_cast<LPCTSTR>(hinst),
&hmod
);
}```
Finally this is the structure of the spawned thread
```cpp
while (true) {
std::unique_lock<std::mutex> lock(mtx);
cv.wait_for(lock, std::chrono::milliseconds(1000), []() { return workerStop; });
if (workerStop) {
break;
}
// .........
}
FreeLibraryAndExitThread(gInstance, 0);```
The refcount increase and worker termination is based on this https://devblogs.microsoft.com/oldnewthing/20131105-00/?p=2733 . But I still get the exception when closing the game. Does anyone know how to fix it, or an alternative approach?"When creating the worker thread I use this to increment the DLLs reference count" just make a dllmain, and block till thread is gone
how do you spawn the thread? Windows specific stuff or normal STL?
keen owl
STL.
So what I gather is, I should signal the worker thread in THREAD_DETACH, and wait there until the thread has terminated before returning from THREAD_DETACH?
glossy inlet
THREAD_DETACH is called when any thread exits, you want DLL_PROCESS_DETACH which fires when your dll will be unloaded
I usually never care about this stuff, my threads are usually owned by a class. And I just exit it in it's destructor
And at the start of the chain there is a singleton class, which will get it's destructor called on dll detach and then destruct all the other necessary stuff.
keen owl
When is your destructor called?
glossy inlet
The C++ runtime calls it automatically, when it destructs static objects on detach
keen owl
Hmm I guess that works because the static objects are destructed after the last THREAD_DETACH but before the PROCESS_DETACH. According to @scenic canopy link, you can't even rely on the address space in PROCESS_DETACH.
Ahh, I see, if thread is .detach()ed you can safely just destruct it (for some reason I gotten to the reverse understanding). So your destructor chain is safe, and no waiting needed (unless the thread does important stuff, like serialize to a DB or something). Do I understand correct?
keen owl
Okay I simplified the issue down to this small code fragment. Only thing DllMain does now is return TRUE. The other extension functions just echo the input back.
struct State {
int (*callGame) (char const* name, char const* function, char const* data) = nullptr;
std::thread workerThread;
};
static State gState{};
void workerProcess() {
int numSlotsAvailable = 99;
while (true) {
// Simulate background stuff...
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
auto x = std::to_string(numSlotsAvailable);
numSlotsAvailable = gState.callGame("broken-extension", "worker", x.c_str());
}
}
void __stdcall RVExtensionRegisterCallback(int(*callbackProc)(char const* name, char const* function, char const* data))
{
bool first = gState.callGame == nullptr;
gState.callGame = callbackProc;
if (first) {
gState.workerThread = std::thread(workerProcess);
gState.workerThread.detach();
}
}```Still access violation on game exit
glossy inlet
what causes the actual access violation?
if you detach the thread it will never exit it
and you need a destructor in State that tells the thread it should exit, and then "join()"s the thread to wait till it exits
keen owl
Yeah, I am looking at your code right now: https://github.com/michail-nikolaev/task-force-arma-3-radio/blob/4cafbc4764d9d40088c846aa0ced39a0affa2caa/ts/src/CommandProcessor.cpp#L42
But I am not sure about the join.. According to <thread> when a thread has been detached it is no longer joinable.
glossy inlet
Correct
you shouldn't detach
detach means "go away I don't care anymore" but you do care
keen owl
So if I understand ~TFAR -> ~CommandProcessor -> stopThread (gracefully signal worker thread to quit, and the join on it)
I am currently doing ~gState -> ~workerThread (ungracefully immediately terminate thread - but should still be permitted)???
glossy inlet
struct State {
~State(){
shouldRun = false;
workerThread.join();
}
int (*callGame) (char const* name, char const* function, char const* data) = nullptr;
std::thread workerThread;
bool shouldRun = true;
};
static State gState{};
void workerProcess() {
int numSlotsAvailable = 99;
while (gState.shouldRun) {
// Simulate background stuff...
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
auto x = std::to_string(numSlotsAvailable);
numSlotsAvailable = gState.callGame("broken-extension", "worker", x.c_str());
}
}
"ungracefully immediately terminate thread" not on detached threads
And on non-detached threads, all it does is a join, not a terminate
keen owl
Then this documentation is useless, or I lost just my reading ability... http://www.cplusplus.com/reference/thread/thread/~thread/ .
glossy inlet
Ah.. apparently it does terminate https://en.cppreference.com/w/cpp/thread/thread/~thread
but std::terminate kills your application
keen owl
^^ yeah...
glossy inlet
I posted code snippet above.
tell thread to exit, wait till it's gone, then return from destructor
keen owl
But I probably want a graceful termination still - though my curiosity about violent thread termination will be unsated... I will try your code now.
glossy inlet
my code is graceful termination
keen owl
Yeah, I know. I explained poorly that I am still curious why my "ungraceful" does not work. Since according to docs I should be able to just ~thread any non-joinable thread.
I guess I should also guard the boolean with a lock (your snippet)?
glossy inlet
not needed no
if there is nothing joinable, the thread destructor simply does nothing, as the thread object doesn't have any thread to take care of
I guess c++ STL doesn't offer forced termination of a thread.
Windows specific functions could probably do it with the native handle
keen owl
Personally I blame the online docs.... I mean from thread::detach it says After a call to this function, the thread object becomes non-joinable and can be destroyed safely. the word 'destroyed' is emphasized and links directly to the destructor..
keen owl
So after calling x.detach() , x becomes non-joinable and x can be destroyed safely (using ~x() ) ? Anyway I am booting up with your snippet now.
glossy inlet
It's like your grandma sitting in your car and knitting some socks. Before you destroy your car in a garbage press, you first have to take out your grandma.
But if you first detach your grandma from the car, then you don't need to worry about her anymore when you put it into the garbage press
detach is, removing grandma from you car
and letting her sit down on a park bench to happily knit some socks on her own
keen owl
Okay I see now. Re-reading the first line of thread::detach.
keen owl
I can't get your suggestion to not crash either. I reduced everything to minimum (almost fully) self-contained file:https://paste.ee/p/tThWz
glossy inlet
what's the callstack of the crash? your worker function?
keen owl
Don't know.. Actually not sure how I debug a dll run by Arma? Is there a guide somewhere?
glossy inlet
attach debugger to arma exe
keen owl
Exception thrown at 0x00007FF7BA85414C in arma3_x64.exe: 0xC0000005: Access violation writing location 0x0000000000000000. occurred
The call stack only lists Arma (and kernel32/ntdll)
keen owl
Nope it just repeats the same access violation
glossy inlet
🤔
keen owl
Also, it happens at the very end of quitting the game.. Debugger reports 4MB allocated for the process.
glossy inlet
any other mods loaded?
keen owl
only CBA
glossy inlet
And if you disable your thread it doesn't cause the issue anymore?
keen owl
Correct.
My testing is just
launching game, open eden, insert dude, add callback handler, call extension, verify callback, quit eden, quit game..
Normally I quit using ALT-F4 but I do not want that to be the cause of my issues so I avoid it for now.
glossy inlet
which game version are you on? Can you reproduce on latest dev branch, using diag binary, and send me a mdmp crashdump?
keen owl
Using the latest official develpoment branch... I am just going to try one thing first before that... I will manually request thread termination using callExtension before quitting the game.
glossy inlet
you could also try setting a breakpoint in the destructor, to make sure it's called before the crash
keen owl
I terminated the process ingame and the game froze.. I could not even kill it using taskkill /F and had to do a full system reset...
glossy inlet
😄 My computer also just froze and I had to do a full reset 😄
But I'm not doing anything arma
keen owl
Wauv, weird coincidence 😃
I will try a breakpoint the destructor, but I notice that no symbols are loaded for my dll, how do I load those?
Ahh, so the freezes are the VS debugger preventing taskill /F from killing Arma, and since I run fullscreen I could not ALT-TAB around properly.
It seems to be the case that the destructor is not run
And now I terminate it manually there is not violation at exit
glossy inlet
if you run a debug build, it should load symbols
keen owl
Yeah, but the practice of this debugging of dynamic loaded stuff is new to me.. So it hadn't loaded the symbols, because Arma hadn't loaded the DLL yet
But seems like the ~State is not run, or at least not early enough.
glossy inlet
maybe struct doesn't do it? I never used a destructor in a struct ^^
keen owl
No only difference between class and struct is default visibility
glossy inlet
I also have problems with that in intercept, but crashing at game exit is.. not really a priority for me to fix
keen owl
Well the proper solution is for BIS to add RVShutdown or the like which is DLL design 101
native kiln
It's like your grandma sitting in your car and knitting some socks. Before you destroy your car in a garbage press, you first have to take out your grandma.
take out your grandma.
neon flax
😂
smoky halo
I tried to use stl thread to finish gracefully all the jobs and quit after arma was closed, but it would crash every time, so switched to windows thread for the worker, worked like a charm
keen owl
Interesting, do you have some sample source code of it somewhere?
smoky halo
looking it up
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
startWork();
break;
case DLL_THREAD_ATTACH:
break;
case DLL_THREAD_DETACH:
break;
case DLL_PROCESS_DETACH:
finishWork();
break;
}
void startWork()
{
DWORD dwThreadID;
workerThread = CreateThread(NULL, 0, doWork, NULL, 0, &dwThreadID);
if (workerThread == NULL)
{
char const *error = "ERROR | Thread creation failed!\n";
fileioRef.logError(error);
return;
}
workerFinishedEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (workerFinishedEvent == NULL)
{
char const *error = "ERROR | Event creation failed!\n";
fileioRef.logError(error);
}
}
void finishWork()
{
if (!workQueue.isEmpty())
{
char const *format = "WARNING | Unprocessed queries at exit start: %d\n";
int jobs = workQueue.count();
fileioRef.logError(format, jobs);
}
workQueue.abort(); // signal thread to wrap it up and finish
if (workerFinishedEvent)
{
WaitForSingleObject(workerFinishedEvent, INFINITE);
CloseHandle(workerFinishedEvent);
}
}
something like this
keen owl
Hmm, that gives me back the access violation.... Think that's enough of this for me today... Hopefully Arma 4, will have Initialize and Shutdown calls.
smoky halo
¯_(ツ)_/¯ could be else where
glossy inlet
" I am just going to try one thing first before that..." you never did the second thing
keen owl
I am unable to replicate it by running it using the diag binary... In fact I can't replicate it using the normal development binary either... Was only using -mod parameter.. It crashes when run through the launcher... So I tried removing all my custom parameters so the launcher also only used -mod paramter, and it crashes there...
*Note the launcher itself however, does not crash
Where would the mdmp file be if created?
glossy inlet
in rpt folder
it probably always crashes
but the launcher not detecting the crash and throwing it right into your face. makes you think it doesn't crash
keen owl
Nope, there is no mdmp files
glossy inlet
bummah
smoky halo
I will manually request thread termination using callExtension before quitting the game.
Don’t think you can quit this way without crashing
You need to release the library first but there are no means to do this
You can use another extension in the middle which arma would launch and which would launch your extension. Then you can release your end point extension from this extension
Arma->ext1->ext2
You can release ext2 from ext1
fallen stone
Just don't load the library inside DLL_PROCESS_ATTACH
subtle smelt
Does anyone know whether someone already made a java library for battleye rcon? Just the handling of the protocol so I can write an application on top of it
fallen stone
So after calling x.detach() , x becomes non-joinable and x can be destroyed safely (using ~x())
Is there a reason you won't let it simply go out of scope? You shouldn't be worrying about this yourself (if it's stack allocated) - as long as the thread is aware of when it should finish execution
Given your locking and CV approach, have you seen https://en.cppreference.com/w/cpp/thread/notify_all_at_thread_exit before? @keen owl
scenic canopy
@subtle smelt google returns many results
subtle smelt
Ah, I tried some different keywords and appear to have found something
keen owl
@smoky halo The manual request, was just me sending a command using callExtension to send the same signal to the thread to shutdown as I was attempting to have happen during DLL unload.
@fallen stone I tried both with, and when Dedmen brought my misunderstanding of the purpose up, also without .detach. And I also tried letting it go out of scope manually instead.
smoky halo
And you will crash doing that
keen owl
No.
smoky halo
which thread are we talking about?
keen owl
In my PROCESS_DETACH if the worker thread was running I was signaling it to stop and waiting for it to terminate before returning... This crashes on game exit.... If I manually sent a a command from inside Arma to terminate the thread (set the same flag), then when I closed the game there would be no crash..
smoky halo
how would you terminate the thread. terminate()?
fallen stone
Generally if you've detached it, you'd let it fall out of scope - this requires the thread finishing execution, which you'd usually have a flag to achieve
keen owl
Okay just ran this https://paste.ee/p/5j6e3
If I exit I get the access violation.. If I send "terminate" before exiting no crash.
nocturne basin
Probably due to a race condition
Chances are, you try to access the already disposed state
Though... I am sleepy and not sure
keen owl
I have tried using a mutex to lock all possible shared state when accessed by the worker thread and the calling thread.. But I think that just causes a deadlock if in anyway related to PROCESS_DETACH.
smoky halo
is this a working code? because it wont compile
and I wont look at it unless it is a working code
keen owl
Yes. It is setup as a standard visual studio DLL with all written stuff put into the linked file, and
- I added
#define _CRT_SECURE_NO_WARNINGSbefore including windows in the autogenerated framework.h file. - To avoid the "buffer issue" for debug builds I disabled Project Configuration -> C/C++ -> Code Generation -> Security Check and set it to Disable Security Check (/GS-)
smoky halo
what VS you have?
keen owl
VS2019 community
smoky halo
ooooooo
you have 3 definitions out of 4 missing __stdcall how the fuck it doesnt tell you about it?
keen owl
Good spot... It doesn't actually matter since C++ takes those from the declaration, but yeah, I will add those to my own code.
fallen stone
probably compiling* x86_64
keen owl
yeah
fallen stone
Thus it's ignored and doesn't matter
keen owl
Okay... I merged the few remaining parts. So this is the entire source file in the project.. No precompiled headers or anything.. And I added __stdcall to the definitions... The only way this file differs is that I don't think there is a pragma to disalbe the buffer security checks.
https://paste.ee/p/jvnrJ
smoky halo
used std::atomic_flag and
~State()
{
if (workerThread.joinable())
{
lock.clear(std::memory_order_release);
workerThread.join();
}
}
and it didnt crash
void workerProcess()
{
int numSlotsAvailable = 99;
while (lock.test_and_set(std::memory_order_acquire))
{
...
Thanks @smoky halo ... Don't know what I did wrong here since I still get the access violation https://paste.ee/p/oNggB .. If possible please paste your full example somewhere and I will get to it tomorrow. And thanks to all who indulged me so far.
smoky halo
I encourage you to read about atomic flag
fallen stone
Or don't, and just use std::atomic<bool> here - it'll be lock free in any case you use it
Ignore the manual spin locks, use the higher level alternative for types where this is next to guaranteed
From your snippet (apart from the thread safety), it looks like the access violation (on exit) is because you're calling that function ptr. It's a 50/50 - if the game is exiting, this is likely going to bork
You could verify this by increasing your sleep (say 10 seconds), and quit the game just after it fires
std::atomic<bool> keepWorking = true;
Is all you need to do for the thread safety
Just set that to false when you're done