#arma3_tools

1 messages · Page 32 of 1

glossy inlet
#

" it looks like the access violation (on exit) is because you're calling that function ptr" that was also my thought after I went to bed yesterday.
When your DLL get's detached, Arma might've already destructed all its internals. And then suddenly you call the arma internal function, even though your thread is supposed to exit.
Turn the while loop into a do/while loop, and move the sleep after the callGame

Also you don't really need a atomic as a flag, doesn't hurt tho ofc.

keen owl
#

Thanks gents. If I raise it to 10 seconds I seem to be able to quit without crashing. However, it seems the destructor is never called in time, at least the debugger does not catch it. Anyway I singled down a more isolated piece of code that is causing issues that I think needs to be resolved first..

In the following code:

  • I pass the callback directly to the worker process. I also check my debugger, and the callback is only set once by Arma.
  • I also completely ignore termination for now.
  • There is no "external" access to data outside the worker, except the arma callback.. All is local
  • Arma crashes, sometimes within a second, but usually within 1 minute after starting the extension, with an access violation to, so far, address 0 and attempting to run a function at address 1 has been seen. Again this is all ingame, nothing to do with quitting the game, or shutting down threads. Did not even leave to go to the menu.
void workerProcess(int (*callGame) (char const* name, char const* function, char const* data)) {
    int msgNum = 0;
    int lastNumSlots = 100;
    do {
        std::string buffer = std::to_string(++msgNum);
        buffer.append(" - ");
        buffer.append(std::to_string(lastNumSlots));
        lastNumSlots = callGame("broken-extension", "worker", buffer.c_str());
        // Simulate background stuff...
        std::this_thread::sleep_for(std::chrono::milliseconds(5));
    } while (true);
}```

Sure it is silly to create a std::string 200 times a second, but should not crash (I still does this if I move it outside the loop body) it is not "incorrect", just unperformant.  I never use more than 5 slots for a frame (lastNumSlot >= 95), so it is not because I exhaust the number of slots.
glossy inlet
#

can you get me mdmp for the crash?

#

might aswell be bug in the new eventhandler stuff system

#

you have the callback eventhandler set ingame? and it get's called too?

keen owl
#

Hey looks like mdmp is working for this issue... Do you want me to run the arma3diag_x64.exe to gather them or is the current one fine?

glossy inlet
#

I can only look at current stable x64, or latest diag x64
and as it's not on stable yet.
yes diag

keen owl
#

This the code I run to test the script:

{ 
    params ["_name", "_function", "_data"]; 
    systemChat str [_function, _data]; 
}]; 

"ext1" callExtension ["Hello",["dude", 2]];```
#

ext1 is the dll.. I will try to crash using diag exe

#

Hmm, the diag exe only game me a bidmp ... I will try again

glossy inlet
#

ugh. I can get non-diag going but that takes an hour or so 😄

keen owl
#

I managed to get one with normal development, and two with diag binary including one where it crashes immediately on call extension (though not sure if that is the same cause)... Can you access this link? https://justbeamit.com/t8ab8

glossy inlet
#

Yep. I'll get on it

glossy inlet
#

Yes, the "after few mintues" thing is a crash in arma, inside the "callGame".
Ah I know why.
XXX

#

so yes @keen owl engine bug and not your fault

keen owl
#

Thank god, its a reassurance I have not gone total bonkers.

#

Not sure what an RString is, but can this also in any way have affected my other thread issues?? They have all been access violations too??

glossy inlet
#

could have been the same yes.

#

Actually... 🤔
Yeah might be that the allocators were already destroyed at that point.
But your game exit thing is most likely also caused by not properly exiting the thread.
You shouldn't do a callGame while exiting, but you already fixed that

keen owl
#

This is the latest iteration I tried this morning using do-while (though I already had my previous while such that the call was right after the check) and BoGuu's suggestion to try atomic<bool>:

void workerProcess() {
    int lastNumSlots = 100;
    do {
        lastNumSlots = gState.callGame("broken-extension", "worker", std::to_string(lastNumSlots).c_str());
        // Simulate background stuff...
        std::this_thread::sleep_for(std::chrono::milliseconds(500)); // or 10'000
    } while (keepWorking);
}```
#

If I change it to 10'000 ms it does not crash. If I keep it at 500 it does... However, it seems in both cases my destructor never runs, and keepWorking never gets set to false... This is bad of course if the thread does not know the game is closed, but I have no way to detect that.

glossy inlet
#
using namespace std::chrono_literals;
void workerProcess() {
    int lastNumSlots; //No need to initialize as you do that right away in next line
    do {
        lastNumSlots = gState.callGame("broken-extension", "worker", std::to_string(lastNumSlots).c_str());
        // Simulate background stuff...
        std::this_thread::sleep_for(500ms);
    } while (keepWorking);
}

very minor things.

#

the crash is based on pure un-luck.
It's a race condition
lower delay just means higher chance to hit the wrong spot

#

the destructor thing always worked for me, maybe that was broken with the new callback stuff?

keen owl
#

Yeah, but I use it in same line xD - "Run-Time Check Failure #3 - The variable 'lastNumSlots' is being used without being initialized."

glossy inlet
#

oooo

#

I need some glasses then

keen owl
#

Hmm, question, when the DLL is unloaded, or Arma is, are all waiting threads awakened?

#

Nvm, obviously not the 10'000 ms crash too

smoky halo
#

Arma crashes, sometimes within a second, but usually within 1 minute after starting the extension
just tried the exact code and nothing

#

int is always 99 sometimes 98

#

no idea what you are doing

keen owl
#

How long did you wait? I got it before 10000 msgs typically

glossy inlet
#

It's a race condition, together with the normal SQF execution going on. Delay till hit is random

#

latest diag binary says otherwise

#

not sure about thread wakeup, I assume not explicitly but it'll wake up sometime.
When the std::thread get's destructed and the thread is not detached, it will have to wake it up, as it waits for it to exit

smoky halo
#

40000 and going @keen owl

#

50000

#

ok crashed

keen owl
#

Ha gotcha!

glossy inlet
#

the heavier SQF load you have running in main thread, the more likely it is to crash

#

I assume you are testing in editor with nothing running in background?

smoky halo
#

yeah

keen owl
#

The scales of my sanity right 1 minute ago: M242 (you crazy Muzzle) <---------------- Muzzle -----------> Dedmen (you still crazy, but also game might bug too).

#

I was moving along the slider back and forth

glossy inlet
#

Diag binary XXX
XXX

keen owl
#

Alright about my race condition.. I completely verified what you guys already guessed.....

  • The thread after waking up, it checks the atomic bool, sees it has to keep running, and calls the game which is closing. This causes the access violation.
  • The destructor never runs, and therefore the atomic bool is still true, because of the access violation happening first.
  • I just refused to believe that I would experience that 100% of the times, since on average there was 250ms for the runtime to shutdown (and thus set the atomic flag before the thread wakes) inside the DLL..
glossy inlet
#

diag binary says otherwise.

smoky halo
#

what the actual fuck

glossy inlet
#

I have 0 idea why. diag binary's XXX

fallen stone
#
The destructor never runs, and therefore the atomic bool is still true, because of the access violation happening first.
``` 👍 👍
smoky halo
#

yes you are right

#

stores array of arrays

glossy inlet
#

array of array is no problem.
array of arrays of XXX is the problem.

#

Btw that works fine on linux because it uses the main allocator there

#

XXX are safe

smoky halo
#

FFFFFFFUUUUUUUUUUUU

glossy inlet
#

XXX isn't, that's XXX

#

you need to do the translation over to XXX

#

Btw when this is done we have to clean up some of the messages here ^^

smoky halo
#

i think you can have array of strings and pull them out in 3s

glossy inlet
#

theeeeoretically yes.

smoky halo
#

should be faster too

glossy inlet
#

yes, will be.
but muh readability 😄

#

might aswell just store structs with 3 strings in them?

#

Intercept one can 🤔

smoky halo
#

unless the struct is accepted type

glossy inlet
#

yeah probably don't wanna bother

keen owl
#
  • My current conclusion is that Arma might invalidate the callbackproc before DLL_PROCESS_DETACH (which also happens before static destructors), and that therefore
  • While you may use callbacks in practice without too much access violation there is in fact no proper way to prevent it fully. It just hasn't happened to you yet.

With your uber-disassembling abilities or what it is, can you see when methods on the dll and the callbackproc are changed?? E.g. can you verify that the extension is unloaded after invalidating callbackproc?

glossy inlet
#

"Arma might invalidate the callbackproc before DLL_PROCESS_DETACH" yes, I'd also say that is the case, struggling with that on intercept too that some game things are destructed, before the dll is detached
"can you see when methods on the dll and the callbackproc are changed" they aren't. I'd say the memory allocator is destructed, and as the callbackproc uses that, bam.
But M242 is fixing that too right now. I'd say continue thinking about that if it's still broken after the incoming fixes

keen owl
#

Are you Arma developer M242?

glossy inlet
#

no

#

Okey "is getting that fixed right now" better.

keen owl
#

In my opinion (take it for what it is worth, if possible to an Arma developer):

  • The idea of giving us the ability to make callbacks, implying we are allowed to use background workers, async, whatever and then not giving us the ability to detect when that callback is invalid is flawed (not only so we can shutdown nice, but also avoid access violations).
  • If they do not want to add proper RVExtensionInit, RVExtensionShutdown calls, then they should at least signal some other way, like perhaps trigger RVExtensionRegisterCallback again with a nullptr.
  • Anyway, just my two cents
glossy inlet
#

"and then not giving us the ability to detect when that callback is invalid is flawed" they don't have that by themselves.
"RVExtensionShutdown calls" even if they had that, that would run after the internal stuff is already gone.
"trigger RVExtensionRegisterCallback again with a nullptr" same

keen owl
#

Well they should trigger it before stuff is gone.

glossy inlet
#

dll unloading is currently not supported at all. So Arma never unloads your dll. It only happens when the application exits and windows does it.

keen owl
#

I don't understand?? If they detect the game is closing ALT-F4, WM_QUIT or whatever, and decide to destruct their memory allocator, they can call all extensions first?

glossy inlet
#

they don't decide to destruct the allocators, that happens automatically

keen owl
#

Well at some point they react to some form of quit message and on that event they get to run code? I mean all other windows programs get to do this.

glossy inlet
keen owl
#

I might take a look at atexit though that just add more concurrency issues.
I am not asking them to unload, even then any code is still run under loader lock. Just to run a function, e.g. in their WM_QUIT handler.
Yeah, I have actually been over that thread the last couple days.

glossy inlet
#

@smoky halo please clean up messages that XXX

smoky halo
#

Eh which messages?

glossy inlet
#

blue cookies! YUM!

smoky halo
#

Ok lemmie know if you find more

glossy inlet
#

all good thanks

keen owl
#

Yeah, I also want to know if you find more blue cookies

smoky halo
#

Are you Arma developer M242?
Kinda @keen owl

native kiln
#

elaborate please

fallen stone
#

Can you remove all the RPT spam then

obsidian sluice
#

Arma developer monkaHm

smoky halo
#

u jelly? :TrollFace:

obsidian sluice
native kiln
elfin oxide
dawn palm
#

mikero tools update
dll:
tools: website updated to url info to bytex (credit @native kiln )
wrp: added new DAYZ mapinfo type (credit zeroy)
rap: macro() expansions need adjacent () (credit eggbeast)
spurious semi colons are now silently forgiven.
trailing commas in arrays now silently forgiven.
makepbo:
prefix. auto removed leading \ or / if present
will throw if a drive specifier present (P:\ eg)
p3d:
reworded duplicated lod to duplicated physical lods (credit dahlgren)
pboProject:
+allowed clearing of cfgWorlds path in setup
+prevents 'clean temp' from deleting a temp\project\source folder (credit @limber moss )
+fixed missing name of catastrophic source\ folder in temp
+issued a warning message to user whenever a source folder is moved to not stop the processing until complete

Eliteness:
small adjustments to api and p3d displays

keen owl
#

@dawn palm I have a P:\md\bonnerup\ExcludeList.lst, and it is put in the pboProject "Exclude from PBO" field, file with a list of relative to that location list of excluded files and folders, but they still get copied to temp and put in the pbo.. Here's a fragment of the file

*.hpp
*.tga
bonnerup.Cache
bonnerup.Layers
bonnerup.Shapes
terrain.Cache
source```
#

What am I doing wrong

dawn palm
#

it needs to have a single comma,separated,line
the cache files should be in source\ and probably layers too

keen owl
#

I think terrain builder will be mad if I move them in there

dawn palm
#

they are exclusively used by t/b so should be there. along with tvx and all others

#

people, generally, make one of two choices, all t/b files are in a separate folder on p: , OR, they're all in source\ (which has become an accepted convention to mean 'ignore')

keen owl
#

Hmm a bit way because something keeps deleting my source folder

dawn palm
#

it is moved to P:\temp\your_project\source to get it out of the way of binarise crashing

#

it is moved back when pboPro finishes

keen owl
#

Well in that case it would be me that have deleted....... never would have though anything in a temp folder was important.

#

So I should structure it like this?
\md\bonnerup
config.cpp
MD_Bonnerup.wrp (exported)
data
source\

dawn palm
#

nice

#

in other words exact same as you see if you extract (even a bis) wrp pbo

glossy inlet
cinder drum
#

Hey everyone, I'm setting up CI for one of my project and I'm considering using armake.
However, I have questions regarding which version to use :
armake seems deprecated in favor of armake2 -- right ?
armake2 on crates.io has no publication since v0.3.0 on 12/2018 -- is it outdated ?
armake2 master on github has frequent activity -- is it stable ?

sick verge
#

I've been using armake2 for a while now and it is mostly stable. And if it's not you will find out very quickly (that is it won't build the pbo because it errors out).
Most problems I have encountered are related to the preprocessor (e. g. Comments in the same line as an attribute specification in a config file) but if you avoid one or two of these special cases, you should be good. If you want to use it for something else than packing PBOs I can't say anything about the stability of that. Never done that.

You will need to compile it yourself though as there aren't any releases yet (as you figured out yourself)

scenic canopy
#

Or download from CI

#

Or use Dicker image

cinder drum
#

Thanks for the feedback. I have my own Dockerfile and will PR it to armake2 official repo once I manage to integrate with travis for nice version tags.

vague shard
glossy inlet
#

Github! We can finally fix stuff!

obsidian sluice
#

Interesting...

scenic canopy
cinder drum
#

@scenic canopy thanks for the hint. Looks like HEMTT is intended for building addons pbo -- correct ? I'm currently setting up CI for a mission project, and will only use armake2 for pbo packing (and sqm binarization if that works). I will definitley have a look at HEMTT whenever I'll have to set CI up on my team mods (probably this fall)

scenic canopy
#

personally I only use pboproject due to the very cranky linting

#

and our CI builds a lot of islands

#

which pboproject is really great for

cinder drum
#

I did look at mikero's tools, but I'd rather support armake if possible.
PboProject can't run on linux which would be very useful in my situation.
Also, I prefer to support an opensource solution if feasible.

scenic canopy
#

it's available for linux afaik

#

but there are no proper binarize

#

armake has a WIP version

#

but there are a couple of known issues

#

and I don't think there's any work done on support for worlds

nocturne basin
#

urgh ... i just look at one enscript file and already can see ... that i will build a transpiler for this if i ever will mod it

cinder drum
#

I'll give it a try anyway, and provide feedback if need be

scenic canopy
#

if you only do packaging of mission pbo I think you'll be fine with either armake/armake2, even older versions

karmic niche
#

You want to write in sqf and have it converted to enscript? Are you out of your mind?!? :P @nocturne basin

nocturne basin
#

transpiler to make enscript into something not broken

#

and more user friendly

#

stuff like ... auto

#

utterly missing

karmic niche
#

And by something you mean? Say it!!! 😄

neon flax
#

sqf to enscript? yes please, we need that to port all that shitty scripts /s

nocturne basin
#

EnscriptF @karmic niche

#

the F is for Fixed

native kiln
#

doesnt enscript have the auto keyword?

glossy inlet
#

yes

native kiln
#

@nocturne basin what would you make more user friendly and how?

nocturne basin
#

whatever i encounter

#

iirc there is also a shitton of issues that exist in enscript in regards of scoping (surprise surprise)

#

or the ref counter

glossy inlet
#

Please fix this vector constructor vector up = "0 1 0"; // initialized by values <0, 1, 0> Thanks.

nocturne basin
#

or stuff like that

#

like ... literally

#

what is that shit?

glossy inlet
#

I would've even been happy with the fact that you'd have to write vector up = vector(0,1,0)
better that.... that.

native kiln
#

i think thats possible

nocturne basin
#

we live in a modern age with modern languages all over the place
would not have been too hard to just take a quick look into that stuff

native kiln
#

"0 1 0" is only the shorthand implicit version

nocturne basin
#

which is horrifying that this even exists

#

possible, ok

glossy inlet
#

Though..
We already have this array<string> pole2 = {"a", "b", "c"}; so... why?
vector up = {0,1,0}

nocturne basin
#

but that it is a thing

native kiln
#

im pretty sure you just do new vector(0, 0, 0) or something

#

ive seen it already

glossy inlet
#

proto native vector Vector(float x, float y, float z);
yep. there is a global "Vector" function, that returns a vector

elfin oxide
#

Yes excactly. there is a function where you could do Vector( 0, 0, 0 ) if you prefer that. I have been working with Enscript for well over a year now @nocturne basin and the language and it's power in regards to how close it works works hand in hand with the enfusion engine itself is outperforming everything sqf could ever do. Before making such statements as "urgh ... i just look at one enscript file and already can see ... that i will build a transpiler for this if i ever will mod it", please use the languge or a while or at least study the avilable script api that comes with the DayZ build. Also yes enscript has an auto keyword, it was just not used in that example. And @neon flax a 1:1 translation from sqf to enscript is either impossible or the results would be terrible. SQF and Enscript have completly different approaches, starting with the little detail that Enscript is full OOP.

neon flax
#

/s

nocturne basin
#

better then SQF != great
you should know the flaws better then other peoples then and know exactly what i have been talking about

elfin oxide
#

There are some things that would suprise a user, things that are done differently or simply do not exist. Yes. However enscript never had the intention to be a language used outside of the game context, it's not designed to be a c++ or java 2.0. It features most if not all classic OOP concepts a programmer these days would consider useful. Enscript does not need any fancy syntax or language features that 1% of people would know how to use. If you know just a little bit of OOP programming, or look at some basic tutorials for the concept, you will find that you can translate that mostly 1:1 to Enscript.
The are issues like you can not have two constructor prototypes for the same class, sure that would be nice if it's fixed, but did any of my desings ever even need that? No. Nor did any of the other scripters that work with DayZ.

Enscript is faster in terms of code performace, Enscript has a better integration into the engine giving you way more flexibility over what modders would want to do, and it's very easy to learn for both complete BI game noobs and those who have been working with SQF for a while.

To anyone who want's to give their opinion on the language I say this: Give it a try, do a few fun projects and be open minded about it. Give it a fair chance and manage your expectations. It is not designed to anything else than a language for modding BI games. If you keep that in mind, you will be suprised on how well it's working out. And especially don't get stuck on things where you say "Oh well but in c++ or java or C# I could have done it this way or that way, why can't I do that in Enscript". There are only VERY FEW cases where Enscript would miss a language feature that is actually so important, that it should have already been implemented. If you need any "fancy" stuff, chances are high that your code design is totally overengineered and you should reconsider your attemt overall.

glossy inlet
#

than*

#

for the constructor problem, could you make free functions with different overloads that just have the same name? like for vector?
say class foo. and then you have a free function foo Foo(int) and foo Foo(vector)
I guess that would be a acceptable workaround

elfin oxide
#

Yes that is totally doable. Function overloading works fine, just not for the constructor. You could make a factory with overloaded create params to resolve that, if you really need different parameters for the constructor. Often enough its just optionla parameters that you would pass in, which already works well. Say Foo( int always, float optional = -1.0 )

smoky halo
#

so what happens if you do not delete new? you get memory leak?

glossy inlet
#

yes

smoky halo
#

LOL you cannot expect people care for it

glossy inlet
#

It gives you refcounting stuff though.
So basically only the idiots who would get new/delete wrong anyway, won't know about refcounting and will get it wrong

smoky halo
#

"Why I have 1 fps help"

#

only the idiots
you mean 90%

dawn palm
#

irrespective of whether script.c has 6 faults or 6 million, there's one over-riding factor that means it's beaten sqf hands down:

auto aResult = SomeFunc(aFloat, anInteger, aString);

bis can no longer add extra arguments, move goalposts, or change the meanings of the args.

There is no longer any guessing at what SomeFunc is or what it pronbably does.

#

It also means if you don't like BI's SomeFiunc().. Roll your own, because each func() is a buildup of lego blocks.

glossy inlet
#

Ah yes. Mixed that up. They changed it to the opposite. things that you grab into a variable by "new" are now just scope local variables and go away at end of scope. you need to store them as a ref.

But apparently only when you directly assign?
The documentation is weird

void MethodC()
{
    MyClass o;
    o = new MyClass;
    o.Say(); 
    // This function doesn't delete the object, which causes memory leak
}

so.. memory leak?

void main()
{
    Parent a = new Parent(); // 'a' has 1 strong reference (local vars are strong by default)
    Child b = new Child(); // 'b' has 1 reference (local vars are strong by default)
    a.m_child = b; // 'b' has 2 strong references, because Parent.m_child is strong ref
    b.m_parent = a; // 'a' has still just 1 strong reference, because Child.m_parent is weak reference
  
    // local variables 'a', 'b' are released (reference count is decreased by 1)
}

sooo... no memory leak?
Although no, the second is circular reference and thus leak. But.. One says it leaks, the other says they go out of scope and get released/deleted

elfin oxide
#

@smoky halo for taking care of memeory leaks there is autoptr keyword. Also you will need to create your objects with strong references most of the time anyway, or else your data is deleted when exiting the scopte they were created in

#

@glossy inlet don't mind me, I missread. Creating a new instance without having any reference or autoptr to it will in fact cause a memory leak

glossy inlet
#

Nope. Docs say it doesn't

#
void function1()
{
    MyClassA a = new MyClassA(); // 'MyClassA()'
    a = new MyClassA();
    // new object is created 'MyClassA()',
    // 'a' is overwritten,
    // first instance of MyClass is released: '~MyClassA()'
    // any code here
  
    // the end of function, local variable 'a' is released: '~MyClassA()'
}
#

Actually docs says both, yes memory leak. And no, automatic destruction at end of scope

elfin oxide
#

if you hold a ref or autoptr to the instance then its collected

glossy inlet
#

Also if you don't

nocturne basin
#
{
    Parent a = new Parent(); // 'a' has 1 strong reference (local vars are strong by default)
    Child b = new Child(); // 'b' has 1 reference (local vars are strong by default)
    a.m_child = b; // 'b' has 2 strong references, because Parent.m_child is strong ref
    b.m_parent = a; // 'a' has still just 1 strong reference, because Child.m_parent is weak reference
  
    // local variables 'a', 'b' are released (reference count is decreased by 1)
}``` if this is a memory leak then BI forgot to add something in their GC
glossy inlet
#

according to docs

elfin oxide
#

if its just raw a = new Type(); then memory leak

glossy inlet
#

"forgot to add something in their GC" there is no GC

nocturne basin
#

oh... yeh

glossy inlet
#

"if its just raw a = new Type(); then memory leak" docs say no.

nocturne basin
#

forgot that part

elfin oxide
#

there is a garbage collector ...

glossy inlet
#

"Since script does not do garbage collecting automatically, all plain pointers are considered unsafe"

#

no there isn't

#

there is refcounting

#

that's not GC

elfin oxide
#

to me reference counting is one way to do GC

#

No GC would mean you need to delete everything manually

nocturne basin
#

nope

glossy inlet
#

GC and reference counting are different concepts

#

Kinda same end result in the end, aka no leaks.
But very different ways of doing it

nocturne basin
#

though ... kinda .. i giuess ...

dawn palm
#

common sense says:
{
MyClass A; //get's auto destroyed
};

{
A= new MyClass; // does not.
};
but , this bis, n'est pas

nocturne basin
elfin oxide
#

yes this is exactly what happens @dawn palm

glossy inlet
#

Well whatever. The wiki contradicts itself. Would need to test to find out what actually happens

#

"yes this is exactly what happens" not according to wiki

void function1()
{
    MyClassA a = new MyClassA(); // 'MyClassA()'
    a = new MyClassA();
    // new object is created 'MyClassA()',
    // 'a' is overwritten,
    // first instance of MyClass is released: '~MyClassA()'
    // any code here
  
    // the end of function, local variable 'a' is released: '~MyClassA()'
}
#

^ That says it get's auto destroyed

elfin oxide
#

that is completly fine

#

because there is a new instance overriding it

#

the second a instance remains

glossy inlet
#

No it doesn't

#

" // the end of function, local variable 'a' is released: '~MyClassA()'"

#

says right there

nocturne basin
#

this is utter clusterfuck

elfin oxide
#

yeah that is rather documentation issue

nocturne basin
#

but all fine and gold ... i remember someone saying that

elfin oxide
#

ingame its follwing one concept, there is no mix

#

either it is auto collected or not

#

i will do some testing later

#

i dont leave any memeory leaks anywhere anyway, so I never had the issue of dealing with it

#

just autoptr it away and dont care what happens to it

#

¯_(ツ)_/¯

dawn palm
#

neither do most C++ coders. it's part of our religion to use ~thingy() when necessary

cinder drum
#

The difference between refcounting and GC lies within subreferences.

void main() {
  ClassA a = new ClassA();
}

class ClassA {
  ClassB _b;
  void ClassA() { _b = new B(this); }
}

class ClassB {
  ClassA _a;
  void ClassB(ClassA a) { _a = a; }
}

With GC the above gets destructed ; with refcount it is not and will memleak

elfin oxide
dawn palm
#

too much emphasis here on refs. accumulating by over-writes is ridiculous

#

const *ref = new anything();

glossy inlet
#

This new stuff to watch out for while make Enscript programming more fun. And IMO you can now finally call it "programming" instead of "scripting"

elfin oxide
#

You mean like SQF was like HTML programming? kek

cinder drum
#

I think introduction of lot of OOP mechanisms is very nice ; it advocates for rapid porting of SeviceLocators and IOC/DI tools

dawn palm
#

whatever the hell that means.

smoky halo
#

Why learn this Frankenstein hybrid if you can learn perfectly good c++ and not pollute your brain with inconsistencies and shortcomings? It is like excel at Guitar Hero rather than spending all that time to learn playing guitar? One is waste of time one is an investment

dawn palm
#

an investment that turns productive in using all other higher level langauges that have based themsevles on C syntax

cinder drum
#

for two reasons : it makes it possible for OOP programmers to leverage that to make more complex features ; it allows functional programmers to continue and easily port their code

smoky halo
#

If BI considered Rust instead of Enforce I would have purchased DayZ so that I could have something to play with and learn the language of the future at the same time

dawn palm
#

well, while I might agree with you @smoky halo , encsript is sufficiently close to C standards to be useful. (in a similar way that config.cpp is). Put another way, would you put Enscript, or SqfShit on your CV when applying for a job?

smoky halo
#

Depends, if you are applying to BI, sure

glossy inlet
#

I will yes 🤣

cinder drum
#

if you are not experienced in OOP it's probably hard to see the advantages, but that's precisely the reason why I didn't spend much time developping in SQF and will probably do more with enforce

nocturne basin
#

that is a fairly cheap excuse @cinder drum
there is various OOP transpilers for SQF
plus you could just mod using something like Intercept

smoky halo
#

Sqf is more like a command language it was fit for purpose until ppl wanted more

elfin oxide
#

Yes that.

glossy inlet
#

SQF was just an attempt at fixing the terrible shortcomings of SQS which was just a quick implementation because BI wanted some scripting functionality

smoky halo
#

That too

glossy inlet
#

One could say Enscript is the first real thing

cinder drum
#

@nocturne basin Arma does not pay enough for me to look for more expansive excuses 😛

native kiln
#

so uhhhh, this in function is memory leak, yes? MyClass a = new MyClass();

#

and the solution is MyClass a = new ref MyClass();?

dawn palm
#

ouch

fiery thunder
#

my eyes

dawn palm
#

I assume @native kiln that there's no delete keyword in Encsript?

native kiln
#

not that I'm aware

#

but @elfin oxide will know better

smoky halo
native kiln
#

Oh damn

#

in my defense I always use ref and autoptr and whatever so I never used it :D

dawn palm
#

autoptr would be all I would want to do to except where I have passed control of it to some other class. But even then...


autoptr thing=func();```
glossy inlet
#

@native kiln yes,no. The documentation says it's a memory leak. And the documentation says it's not a memory leak.
Second thing no. Syntax would be ref MyClass a; a= new MyClass();
Plus also no, even without ref it would get destroyed safely and not leak, according to documentation.

The documentation also says as long as your class inherits from "Managed" it's automatically refcounted and safe.

elfin oxide
#

That manged bit is right though. Managed classes are completly automatically well ... managed

glossy inlet
#

So I'd recommend, just do the right thing and inherit from managed and not even try to figure out how the "wrong" stuff works 😄

vital chasm
#

You should consider to inherit from managed most of time, and only not inherit from it, if you know what you are doing

vague shard
#

whats a good, simple GUI prototyping system these days - with multiview/-page logic preferably. anything Arma based (ie ADC, GUI D3V 2.0) or related, or something like MockFlow?

smoky halo
#

Anyone tested callback fix yet?

glossy inlet
#

@keen owl ^ ❤

keen owl
#

@smoky halo @glossy inlet Yes, seem they fixed their own internal race condition. Not only that, (or maybe because of that), DLL_PROCESS_DETACH is actually also called now. Also the destructors for global objects are run... So not only does it not crash ingame, you have a chance to run something on unload at least.

#

Though, thread::join behaves strangely and does not seem to block, yet somehow a spawned thread dies without running all statements.

#

I am guessing the DLL unloading is just a side effect of the entire Arma process unloading and therefore all threads are halted, making thread::join effectively non-blocking.

#

If that is the case, then one should just work on the assumption that you need not (can not yourself) terminate your threads (unless you do it before game process exit). However, if you want to do some cleanup, you should do it in the DLL_PROCESS_DETACH synchronously and very delicately since any worker thread might have been halted at any point.

karmic niche
#

Maybe this warrants a subsection in the wiki? ^^^

vague shard
#

is Enscript less prone to abuse and cheating than sqf? and how much of a role plays callExtension (or a similar system for Enfusion) for this?

karmic niche
glossy inlet
#

there is no callExtension, or similar system for Enfusion.
Enscript is secure as. you cannot run any code without having it inside a bisigned pbo

#

SQF is insecure as you can literally just execute one line of code in editor, to write a script into a uiNamespace variable, and then just let it be executed when you join a MP server

#

Enscript doesn't have any on-the-fly scripting ingame yet

vague shard
#

not even with the diag.exe?

obsidian sluice
#

DZ has some diag exe?

glossy inlet
#

I don't think the debugger supports live editing

#

There is some api that seems to be able to compile enscript, which can be called from a enscript script. So you could maybe make your own wrapper. But I think that API isn't working yet

#

But it's very easy to just search for uses of that API, and thus will also be very easy to find potential security issues in the usage of that

vital chasm
#

Debugger is not working, scripting command linked with debugger is not active, and will not be possible to be active, dev said

#

The only "similar" of callExtension, it's will be in the future the implementation of cURL inside engine

#

But extension will not be implemented into DayZ

#

And for security, everything is executed at server side, the network synchronization is made by the server.

nocturne basin
#

I want some function to parse enscript assembly directly

#

custom language frontends

#

whooop whoooop

vague shard
#

well they should make a dummy project separate to DZ SA to give the Arma modders an unrestricted playground 😉

smoky halo
#

SQF is insecure as you can literally just execute one line of code in editor, to write a script into a uiNamespace variable, and then just let it be executed when you join a MP server
If you are talking about bis fnc parsenumber exploit, this has been fixed now

#

So how do you contact database in Enscript?

glossy inlet
#

I'm not talking about any specific exploit

#

You don't.
DayZ "database" are just binary files, using the serialization API (same as config.bin basically) exported in Enscript
files are read and written by enscript

smoky halo
#

So you can’t have normal back end MySQL or PostgreSQL DB yet?

glossy inlet
#

yes

scenic canopy
#

depending on cURL support it might be possible through an external API

glossy inlet
#

would have to basically implement everything over a REST server that you run seperately :U

smoky halo
#

Wouldn’t this be as slow as fuck?

glossy inlet
#

yep

smoky halo
#

Lol

elfin oxide
#

@obsidian sluice Yes DayZ has a diag exe and a full script debugger with breakpoints and watchers and everything. it's built into the workbench.

#

@smoky halo there might still be a hive implemenation coming that is open source that allows to usage of any database backend you want for persistency. Right now extensions are not a thing and won't be. cURL support is beeing looked into.

#

And REST API are not "slow as fuck" per default. You can very well have a close to realtime local communication.The HTTP overhead will not be noticeable for 99,99% of any use cases.

#

Direct memory access to DayZ is not a thing and any projects that attemt to hook into DayZ's memory are taken down.

smoky halo
#

Ok, see you in 100 years after DayZ also has helicopters

elfin oxide
#

The community is already adding helicopters as mod 😃

smoky halo
#

could happen sooner then for the stand alone, in 99 years mayabe

native kiln
#

60 years of you give cryo sleep a chance

nocturne basin
#

enscript and rest api access?

#

via some HTTP interface? or via an actual TCP socket connection and some HTTPClient implementation?

#

because depending on what exactly that thing supports, you may missuse the whole HTTP thing and eliminate most of the overhead HTTP has

elfin oxide
#

As I said, it's either cURL or nothing. It's passive

#

The game can ask third party services but they can not initiate anything on the game actively.

glossy inlet
#

if we get some kind of async stuff, we might be able to do long polling

smoky halo
#

Wait, you can’t async in Enscript?

glossy inlet
#

not sure, never tried

scenic canopy
#

@elfin oxide what's the planned scope for cURL integration?

#

only GET requests? Or POST/PUT/PATCH/DELETE with payloads?

smoky halo
#

So thread waits for cURL return?

scenic canopy
#

long polling is poor mans websocket

smoky halo
#

I’ve got a feeling Arma’s shitty SQF with primitive callExtension is miles ahead of “next gen” Enscript

elfin oxide
#

Enscript has async, callbacks and script threads, so it would generally not be an issue

#

@scenic canopy yes that

#

It should hopefully allow all the HTTP types with a payload yes

#

However the design it still beeing looked into, so I am not sure what will come out of that in the end.

#

The requirements and what modders want to do with it in DayZ were clearly communicted to the dev team

smoky halo
#

By threads do you mean proper threads?

#

How do you make one in Enscript?

elfin oxide
#

I said script threads. They are all executed in a time sharing system. Only the networking has it's own dedicated thread

smoky halo
#

So like spawn in other words?

elfin oxide
#

Yes like spawn

smoky halo
#

I cringe every time someone calls it a thread

elfin oxide
#

It's perfectly valid to call it a thread. How do you think what you consider "real threads" are done for mutlithreaded cpus? It's also just a TSS

#

You also have this

#

which allows to queue calles for later, as in execute function x in like 50 ms, or 10 seconds

scenic canopy
#

if there's support for generic HTTP method with arbitrary headers and body I'm happy 😄

smoky halo
#

Only 9 params?

scenic canopy
#

preferably some JSON library as well

elfin oxide
#

@scenic canopy Enscript has full json support already

scenic canopy
#

even if support for native extensions will be missed

elfin oxide
#

@smoky halo yes, "only" 9. This is how c++ handles prototypes as well. a lot std:: / boost:: functions only prototype for 9 params. If you need to pass more to a function, there is a param< N > template you can use to combine data in an abstract container

#

It goes up to Param6<>, which is 9 to the power of 6 params you can pass into the function if you would really need to do so

smoky halo
#

I suppose you have an example of such std:: function?

nocturne basin
#

i cringe at enscript not supporting endless parameters as it seems

elfin oxide
#

Unfortunetly I don't have acess to the source of a project at my old job anymore where I hit that snag. I was attemting ( cause of bad design ) to pass like 10 values into a boost::asio network buffer and one of the function was only protoyped to 11 parameters or something like that. It's a limitation caused by templates not having variable amount of parameters for the compile time. That is simply not a language features c++ has, no does Enscript have it. The only way around it is to either not send in your data raw, but prepackage your input info into a wrapper class, or have many placeholders and have a template / function overload prototyped for for each one of them. Or use void pointers and tell the function how much of what type is to be excpected and cast that from the raw memeory. So obviously in this case the just went for 9 params. If a function call has 9 params that do not belong together in any way, the desing is pretty bad. I run into that, and indeed my design sucked.

#

The reason for why they would even do that is, because the link the enscript protoypes to the direct engine call on the c++ side. So they need some reliable prototype they can find in enscript to hook up to.

smoky halo
#

I mean it is ok you can have boost and whatever have anything you want, but I am surprised std:: would have limit of 9 params, so I'd like to still see the evidence of that please

elfin oxide
#

I'll make sure to send the source link to you as soon as I can find it. std:: copied boost functions 1:1 starting a few releases ago. Stuff like the filesystem, so eventually some of those "unavoidable hard coded template variants" make their way into c++ even if you don't use boost

smoky halo
#

yeas please send it to me when you find

#

It's perfectly valid to call it a thread
Do these "threads" in Enscript run async/simultaneously?

#

Or does Enscript have some kind of scheduler like in SQF?

vague shard
#

@elfin oxide thanks a lot for all your insights 👍

glossy inlet
#

@elfin oxide boost limitation came from the fact that it was made before variadic templates were a thing

#

@smoky halo Or does Enscript have some kind of scheduler like in SQF? arkensor already said there is some kind of scheduler

elfin oxide
#

@smoky halo becuse enscript itself does actually run on one "real" thread, they are actually running behind each other as I saind, in a time sharing system. However on script might run 3 times and then the other one tine time, so it's not always a FIFO queue

smoky halo
#

So how async is achieved? There is no actual spawn a thread and it is blocked until it gets result then joins or signals, is it?

native kiln
#

You can make a timer to make something run later, that's put in the scheduler, and for stuff that goes into the C++ land, I can imagine they have some way to break execution at that point aswell so a http call would return back to enscript in potentially a different frame

wide cedar
smoky halo
#

So basically no proper threads or async, just poor imitation for kids to play 😂

fallen stone
#

what is 'proper' thread or async?

smoky halo
#

sequences of programmed instructions able to run concurrently and independently

fallen stone
#

Which would be achieved by a scheduler...?

native kiln
#

whats with the hate against calling it threads though

smoky halo
#

We used to call the concept Green Threads, that is threads that are scheduled by the runtime of the language and not mapped to the native threads of the operating system. Green Threads could in theory be mapped to native ones but usually they ended up being just runtime scheduled onto a single OS thread. Nodejs has done very well with this, it just suits IO driven activities rather than real processing work.

native kiln
#

hm, well I think Threads is still a good term for it, given the (this sounds a bit nitpicky) definition, as I mean they don't necessarily imply parallel execution, just concurrency

smoky halo
#

concurrency is parallel execution

native kiln
#

I was actually taught that it's not

#

Concurrency means that two things can start and stop in overlapping timeframes, however the execution of these two doesn't need to be in parallel to be considered concurrent

elfin oxide
#

That is the defintion of concurrency yes.

smoky halo
#

Yeah just reading on that it seems to be quite a lot of general misunderstanding. So lets just say overlapping concurrency, which means you cannot do this with a single thread

elfin oxide
#

In what situation would a code design REQUIRE two task overlapping?

smoky halo
#

when you have 2 independent task happening independently

#

when you dont want one task wait for another task or even portion of it to complete first

#

because if we take definition of concurrency in programming which doesn't seem to have anything to do with the definition of the word concurrency, then it looks like time sharing

elfin oxide
#

concurrency translates 1:1 to two things running along side each other, no matter the context

smoky halo
#

it is not

elfin oxide
#

I am not sure what you try to say, but in any case concurrency both as a literal word and as concept in computer scienes stands is an alias for parallelism

smoky halo
elfin oxide
#

and the defintion of parallelism does not require them to overlap

smoky halo
#

nothing is happening at the same time

#

when one thread runs another doesnt

#

and vice versa

elfin oxide
#

A time sharing system is the way concurrency is achieved. You have a physical number of cores, and thus all of your threads on a pc are running in a time sharing system. If you got 4 cores you just happen to have 4 of them running which by random chance could mean that one task of your program is actually running at the very exact same moment. However I have never seen any concept for a program that would required parts of it's program to run at two cores at once to make sure its 100% the same moment.

#

I may add, that thread 1 and 2 on your pictue you posted are also just running in a time sharing system, even if they are considered parallel in the scope of the program, but as soon as the program is executed it's split across pyhsical cores or not, depending on what the cpu wants to do

nocturne basin
#

uhm ... parallelism actually does require them to run in parallel (hence the parallel word)
concurrency however, does not

smoky halo
#

This is nothing to do with what we are discussing here. I also haven't seen anything required to make sure it runs at the same moment (not saying there isn't such thing). But there are plenty of times when you dont want one task wait on another and that's is what you have this definition of concurrency implies. So my bad if I used this term incorrectly even though it goes against the very definition of the word, so yes, lets stick with parallel. You cannot have parallel execution in DayZ and this is what I was trying to find out in the first place.

elfin oxide
#

No, neither can you do parallel execution in arma. It's still a single script vm working in a time sharing system

glossy inlet
#

in vanilla arma*

elfin oxide
#

However this does not limit you at all in the way you want to design scripts

#

Shhhh you and you magic things @glossy inlet

smoky halo
#

Yes you can in Arma

#

not in scripts of course

#

But you can lauch several videoplaybacks at once and each will run in own thread

#

and ofcourse you can use extension with as many parallel threads as your CPU will allow

nocturne basin
#

uhm .... CPU is not the actual limiting thing for threads

#

your OS will schedule the threads to different CPU cores

#

some run concurrent

elfin oxide
#

i would assume the video playbacks are actually all just a time sharing system

smoky halo
#

no you assume wrong

elfin oxide
#

what makes you believe arma would be able to tell your pc to make sure to run all video playbacks on different cores?

nocturne basin
#

would be able
you can force a thread/process to run on a specific core
though ... there barely is a reason to do so

elfin oxide
#

all your videos are still run within the same cpu and gpu threads arma 3 has

smoky halo
#

Arma doesnt but c++ it is written on does

elfin oxide
#

and arma i think has just 1, maybe 2 like dayz with one that handles the network traffic

#

So not in arma?

smoky halo
#

you want to be nitpicky? extension is extension of Arma, it has access to arma's memory, so is it Arma as well?

elfin oxide
#

Nah you are just moving goal posts here. We were talking about scripting, not using the memory space of arma to spawn new threads. But I guess we can all agree that vanilla DayZ and Arma can not do that. Needless to say that I still can not see how that would matter at all.

#

Enscript runs concurrently, per strict defintion. Which is all you need to know

#

Does it influence any design choices? No.

nocturne basin
#

still need the instructions list of enscript VM inb4
just in case somebody got too much time on their hands

elfin oxide
#

I doubt there is a need for enscript vm ... Don't waste your time on that

nocturne basin
#

there is and there will be

smoky halo
#

I don't really care if you can do this in DayZ scripts or not, I was more interested if you can do this at all with DayZ but it seems you cant at the moment despite your efforts to convince me otherwise

nocturne basin
#

alone for stuff like this:

pole1.Insert(new MyClassA());``` to show proper errors
#

plus one may learn a thing or two from just being able to compile it custom and from game (when somebdoy at some point finds a way to extract compiled scripts)

native kiln
#

I think that code would actually result in a NULL entry in that array :D

nocturne basin
#

plus you may not always have the game at hands

glossy inlet
#

according to biki that results in a null yes. But according to wiki it also doesn't :reeee:

native kiln
#

Well in my tests it did

elfin oxide
#

I'll update the wiki with proerly tested cases. It's on my todo

nocturne basin
#

if there was a VM for enscript, one could just write @Enscript CODE now

#

plus i will create some BNF file for enscript when i actually have something ... right now, there is no reason for me to do anything with enscript but blame everything i miss or do not like about that lang

native kiln
#

Or maybe I did something slightly different, I know that when you have reference counting to a class, and put it inside an array that doesn't say <ref MyClass> but instead only <MyClass>, the game things the instance is not being referenced anymore and dumps it

smoky halo
#

I am pretty sure there will be a demand for being able to run some sqf in A4

nocturne basin
#

for the "run SQF in ArmA 4" part ... SQF-VM port to enscript

native kiln
elfin oxide
#

I don't see sqf->enscript be a thing, BUT If someone does make a half decent converter that would be some interesting thing indeed.

nocturne basin
#

just gimme dem instructions already

#

by end of today i will have some sort of VM Stack ready

#

no language frontend

smoky halo
#

you think you can write it in Enscript?

nocturne basin
#

SQF-VM?

smoky halo
#

yeah

glossy inlet
#

I don't see anything that would stop you

nocturne basin
#

i could write SQF-VM in SQF if i wanted to

smoky halo
#

yeah but does Enscript has everything you need for that

nocturne basin
#

literally only need two things for something like that to work:

  • lists
  • Methods
  • Datastructures
#

SQF-VM used to be a C implementation up until the point where i oriented heavily on the game

native kiln
#

Please make Enscript VM in Minecraft redstone, thanks

nocturne basin
#

there even still is this

#

i actually did created a simple redstone computer once ...

smoky halo
#

If you make one and add on github for ppl to write methods for it it might take off

nocturne basin
#

but i am far from patient enough to do something like that

smoky halo
#

by methods I mean SQF commands in Enscript

nocturne basin
#

that is probably the smallest problem

native kiln
#

I think it's time to move past SQF :P

smoky halo
#

I think on contrary, have you seen how many sqf commands out there?

nocturne basin
#

just gimme dem instructions then @native kiln

#

Enscript -> SQF Assembly compiler FTW

elfin oxide
#

@smoky halo SQF has so many commands becasue its limited in it's design to not be OOP. Enscript is able to do more than sqf does. An arma implementation would have everything sqf could have done, just directly exposed as the enscript classes that are linked to what sqf commands currently link to

smoky halo
#

what is OOP design has to do with the number of commands in SQF?

nocturne basin
#

Player.GetPosition vs position player

smoky halo
#

Still a method

nocturne basin
#

technically, position and player are not commands though

#

thus the correct wording has to be: SQF has many operators

native kiln
#

in enscript, you dont need commands, everything you need is given

#

the only thing you can really ask for specifically is exposing something internal via an enscript interface

#

most things are available, other stuff like some SQF commands might involve doing multiple things in enscript (just like the SQF commands do under the hood)

#

like uh, nearestObjects for example (i imagine)

smoky halo
#

How many position formats in Enscript?

native kiln
#

@elfin oxide

glossy inlet
#

so far I've only seen two 🤔

smoky halo
#

thats an improvement

native kiln
#

what, you didnt like 1000000 different systems? 😄

#

always a pleasure constantly translating positions in arma

smoky halo
#

Arma has as many position formats as it had programmers working on positions

glossy inlet
#

you have extra functions for getting sea/terrain level and such

native kiln
#

id say enscript is much more

#

intuitive

#

but one thing i absolutely love, and its the memory management

#

i feel like it could have been made so much easier

#

constantly having to bother with refs and autoptr becomes a bit exhausting

glossy inlet
#

I guess you can make your own shared_ptr implementation

#

enscript gives you everything you need for that (new/delete/constructor/destructor/copy-..... oh.... wait...

native kiln
#

I mean I get the benefits of this much control over what happens in memory and how it behaves, but does it really need to give me this much control? Is the overhead of doing it similar to how C# or Java does it really that big?

#
// Memory leak
class A {
    autoptr array<B> list = new array<B>();
    
    void A() {
        list.Insert(new B());
        list.Clear();
    }
}

// Not a memory leak
class A {
    autoptr array<ref B> list = new array<ref B>();
    
    void A() {
        list.Insert(new B());
        list.Clear();
    }
}```
#

I just find this more confusing than helpful

smoky halo
#

...function overload @glossy inlet

glossy inlet
#

I don't think you can overload assignment operator

#

first is only memory leak if B is not managed i think

native kiln
#

yeap

#
class A {            class A {
    void A() {            void A() {
        new B();            B();
    }                    }
}                    }```
#

i just think it would have been fine without the additional control

#

over stuff like does this go on stack or heap

#

its not going to be relevant for so much stuff

smoky halo
#

B();
it doesnt consider it as function call?

native kiln
#

it makes a new instance of that class afaik

#

on stack

glossy inlet
#

I haven't seen second thing

#

I have never seen a instance be created without new

native kiln
#

not? then i must have been dreaming, but point still stands 😄

#

what im also interested in is this

#
    autoptr array<B> list = new array<B>();
    
    void A() {
        auto b = new B();
        list.Insert(b);
        list.Clear();
    }
}```
#

memory leak or not?

#

if i remember correctly, it knows b goes out of scope and will delete it, then it should become null in the list, thats fine so far, but what happens here then:

#
    autoptr array<ref B> list = new array<ref B>();
    
    void A() {
        auto b = new B();
        list.Insert(b);
        list.Clear();
    }
}```
glossy inlet
#

Ah
B myb; is how you declare a instance on stack.

#

but that just gives you a null object

#

you need to put a new'ed thing inside it

smoky halo
#

can you have multiple constructors?

glossy inlet
#

no

smoky halo
#

so default constructor has no arguments?

#

can you copy instance?

glossy inlet
#

you define your own constructor

#

with how many arguments you want, but only one

smoky halo
#

so you can have args but only 1 constructor?

glossy inlet
#

there is no copying stuff, only if you make your own function for that

#

yes

nocturne basin
#

to make thigns quick: Enscript cries out loud for a transpiler that fixes obvious design issues like the lack of constructor overloading ...

elfin oxide
#

Constructor overloaded is not that big of an issue, really. Nice to have, very few times to be required for a clean design, especially considering that most stuff is sorted with optional parameters, which work for functions and constructors

#

There is only one position format. Vector( X, Y, Z ). No above sea level or anything like that. Positions can be global space or local space of a parent object

#

You can do stuff like sea level, or terrain level, via functions that will return the sea / terrain level Y value

cinder drum
#

any reflexion feature in the language ?

nocturne basin
#

would love to have such things
but i extremly doubt it

warm niche
#

hey anyone know how to fix extract pbo

#

it pops up in the task bar but cant see the app

glossy inlet
#

It's a command line app

warm niche
#

when was that changed

#

i used the app like 2 weeks ago

glossy inlet
#

Never afaik

scenic canopy
#

There’s both a cli and gui version

glossy inlet
#

wuht? Never seen a gui version of it

warm niche
#

gui is broken i guess

dawn palm
#

the gui takes precedence from a dos prompt with no arguments, OR, a dbl click on the exe itself. (Many, not all, the tools have this duality)

#

Important update to makepbo for those of you suffering it not being detected as being installed. This irritation only affected subscribers (sigh).

smoky halo
#

Just saw a server out there with 39 mods

#

Tripped both signature and mod overflow flags

#

address: 85.214.248.63:2302 ping: 84 respondedwell: true donotrefresh: false gamedir: arma3 map: gm_weferlingen_summer description: Liberation appid: 107410 players: 0 maxplayers: 30 botplayers: 0 passworded: true secured: false lastplayed: 0 serverversion: 192145639 gametags: bf,r192,n0,s7,i2,mf,lf,vf,dt,g65545,h5e08dd6a,f0,c0-52,pl,e15,j0,k0, steamid: 90127684315138050 server name: T.I.G.E.R

#

`RulesRefreshComplete: T.I.G.E.R
Version: 2
Global Flags: [00000011] ModOverflow SignatureOverflow
Required DLCs: [0000001101110111] Kart Marksmen Heli Expansion Jets Orange TacOps Tanks
Difficulty Index: 2
AI Level Index: 2
Flight Model: Standard
3rd Person Camera: no
Weapon Crosshair: no

N Custom Mods: 39
Custom Mod [DLC - no]: @gm
Custom Mod [DLC - no]: TFAR
Custom Mod [DLC - no]: Tembelan Island
Custom Mod [DLC - no]: @splendid_smoke
Custom Mod [DLC - no]: BW-Uniform-Replace
Custom Mod [DLC - no]: Ruha
Custom Mod [DLC - no]: Rosche, Germany
Custom Mod [DLC - no]: RHS: United States Forces
Custom Mod [DLC - no]: RHS: Serbian Armed Forces
Custom Mod [DLC - no]: RHS: GREF
Custom Mod [DLC - no]: RHS: Armed Forces of the Russian Federation
Custom Mod [DLC - no]: Redd'n'Tank Vehicles
Custom Mod [DLC - no]: @rds_civ
Custom Mod [DLC - no]: Project OPFOR
Custom Mod [DLC - no]: Cambodian Playground
Custom Mod [DLC - no]: @niarsenal
Custom Mod [DLC - no]: @mbg_buildings_3
Custom Mod [DLC - no]: LYTHIUM
Custom Mod [DLC - no]: @jbad
Custom Mod [DLC - no]: Gruppe Adler Mod
Custom Mod [DLC - no]: Gruppe Adler Mod
Custom Mod [DLC - no]: GRAD Trenches
Custom Mod [DLC - no]: Gruppe Adler Sling Helmet
Custom Mod [DLC - no]: GM: Demo
Custom Mod [DLC - no]: FHQ Accessories Pack
Custom Mod [DLC - no]: DUI - Squad Radar
Custom Mod [DLC - no]: @cup_wheeledvehicles_suv
Custom Mod [DLC - no]: CUP Terrains - Maps 1.12.0
Custom Mod [DLC - no]: CUP Terrains - Core 1.12.0
Custom Mod [DLC - no]: Community Base Addons v3.12.0
Custom Mod [DLC - no]: BackpackOnChest
Custom Mod [DLC - no]: Anizay
Custom Mod [DLC - no]: Achilles (1.2.1)
Custom Mod [DLC - no]: Advanced Combat Environment Extras 3.4.2
Custom Mod [DLC - no]: @ace_optionals
Custom Mod [DLC - no]: @ace_compat_rhs_afrf3
Custom Mod [DLC - no]: @ace_compat_rhs_usf3
Custom Mod [DLC - no]: @ace_compat_rhs_gref3
Custom Mod [DLC - no]: Advanced Combat Environment 3.12.6`

smoky halo
#

there is one with 45 mods, jeez

#

looks like over 30 is quite normal

neon flax
#

My unit has 53 in basic preset 🤔

glossy inlet
#

I think I'd have more if we didn't pack small things together

smoky halo
#

But why, I bet many mods are just scripts packed into mods

glossy inlet
#

yep

#

the ace compats up there are literally just a config.cpp and nothing else

scenic canopy
#

GM: Demo thinkinghuh

neon flax
#

We have a ton of little mods, easier to manage and pick mods per mission like that.

smoky halo
#

but is it against TOS to pack all modes into one pack?

glossy inlet
#

which TOS?

#

Steam, absolutely.

smoky halo
#

the steam protocol was clearly not designed to handle millions of mods and so is linux server by the look of it

#

this is why BI invented -par thing

orchid shadow
#

I think we have like 80 mods

#

Ofcourse we are not really public and don't use steamworkshop because ya know why would you. <.<

smoky halo
#

Version control of ones mods is IMO really important. If you want your game to work at the time you want it you need to lock the versions in and ensure everyone has those. It makes upgrading the expensive time consuming part, but it also means you get to test the upgrade rather than chase the bugs and have them ruin a game. The Steam workshop as it default works just doesn't facilitate good mod hygiene, you can't do the right thing with it by default. That same is true of the game updates, you can't by default choose a version. There are clear dependencies in the mods and the game itself and there is zero practical management of it from Steams perspective.

nocturne basin
#

Actually ... steam workshop is perfectly fine with its updates always running the latest

#

in an ideal world, software (and mods are software) would just work without new bugs getting "patched in"

#

but we kinda all agreed on just accepting that software tends to crash, not work etc.

#

so it is more the modders who are to blame here

#

for making stuff like that so complicated

rancid relic
#

don't forget that some of the mods my relay on 3rd party stuff

vague shard
#

steam would only need to expose version management

#

its already in place in the backend for a while, if not from the start

karmic niche
#

but we kinda all agreed on just accepting that software tends to crash, not work etc.
We also kinda agreed that people tend to crash cars and yet we're installing bumpers and airbags, instead of saying "oh well, he should have driven more carefully; now where's my mop and bucket? Gotta remove all this blood now" 😛

scenic canopy
#

@vague shard except then people would be "stuck" on old versions and will complain to mod makers about not fixing issues 😛

#

like, some of the communities we play with insist on using a broken mod with splash screen errors etc which hasn't been updated since 2015 🙃

sonic pond
#

Hey question . Is there an alternative to task force radio? In other words , is there anything that i can use for an ingame phone/radio ?

glossy inlet
#

ACRE

sonic pond
#

Thanks !

scenic canopy
#

Without requiring addons

#

But it uses the builtin VoN behind the scenes

glossy inlet
#

addon free arma radio is a addon on the workshop 🤔

scenic canopy
#

Says scenario

glossy inlet
#

aww

scenic canopy
#

So you can rip it out

#

I think the scripts are linked as well?

#

One of out members made a similar one back in alpha/beta days before acre was officially ported

#

It works for just basic comms but acre or tfar is of course A LOT nicer

cinder drum
#

The actual problem with Steam Workshop is not that it auto-upgrades mods to latest version, it's that you can't auto-upgrade a dedicated server. If everyone had the latest version pushed automatically, there would be no problem. Whenever a mod author releases a new version, unless it has a compatibility feature with previous versions, you have to either manually downgrade clients (which defeats Steam Workshop purpose), or wait until the server administrators have upgraded the servers modpack and restarted the missions, which can take some time. Moreover, new versions of mods and game are frequently pushed a few hours just before most European units start their evening missions and events. A lot of them have been canceled because of breaking changes that were released just before gather time, ruining everyone's experience ...

scenic canopy
#

sure you can, just write your own server tooling 🤷

#
  • check if any workshop items are outdated
  • download new/updated workshop items
  • send in game message using rcon to all players about reboot
  • wait X amount of time units
  • stop server
  • apply updates
  • start server
narrow bough
#

you can do all that via batch file on server executable restart, pretty simple using steamcmd in batch.

cinder drum
#

Don't misunderstand me on this one, I know I can tool this, and I know how to do that. It still is a seriously missing feature from the workshop that this is not available out of the box. And a reason for units not to use it.

#

Not every unit can afford to have a competent sysadmin in their ranks

nocturne basin
#

@karmic niche to get to that place, we took years
Plus it is the car that has those safety measures in place... You do not put them there yourself

karmic niche
#

Uhh.. don't we have people writing software for dozens years as well?
I just disagree with you stating that workshop is working perfectly fine and it's the programmers that are doing their job badly, and are to blame. No! Steam should take into account that humans and humans (and will always be, unless you want to tell me something) and that making mistakes is what just happens so you can't expect everyone to ALWAYS have the latest version of mod X.

Take linux distributions, for example. Why do you think we have LTS releases? I thought that we accepted that software was meant to crash and we all wanted to have the latest buggy Ubuntu version, instead of that LTS that is supposed to be (more) stable.
@nocturne basin

scenic canopy
#

@cinder drum probably because every game implements workshop, servers etc differently 😛

karmic niche
#

Thing is, the world isn't ideal and you always should account for it. Not doing that is YOUR fault, period.

glossy inlet
#

Having written idiot-centered Arma launcher for the last years. I am with you on that.
Always have to account for dum dum

scenic canopy
#

😛

#

it's interesting to see how users can even break things with even just one button

#
  • It doesn't work
  • Well... did you run it?
  • No
  • ...
smoky halo
#

what is this bullshit? If you dont use launcher you cant use mods. You can subscribe to a mod and download it but it will sit in download directory inaccesible unless you start launcher! Then launcher will move it into !Workshop dir, and only then you will be able to -mod it.

#

launcher is not optional and I dont like it

glossy inlet
#

launcher doesn't move it creates symlink

#

and if you are desperate, you could load directly from the steam folder without symlink I think

smoky halo
#

yeah symlink

glossy inlet
#

not sure if global mod paths still work?

smoky halo
#

how do you load from steam folder? if is outside of arma dir

glossy inlet
#

my last sentence

smoky halo
#

dunno what that is

glossy inlet
#

full path in -mod

#

C:\bla bla

smoky halo
#

why would it it was axed long ago

#

or something was

glossy inlet
#

I think my launcher still uses full paths

#

and it works

#

filePatching full paths was removed

smoky halo
#

but when it is downloaded it just a nubered folder

glossy inlet
#

yep

smoky halo
#

what you link to is as if it was a mod?

glossy inlet
#

yes

smoky halo
#

hmm gonna try

glossy inlet
#

Load diwako dui ->
-mod="F:\Steam\SteamApps\workshop\content\107410\1638341685"

smoky halo
#

and launcher moves files

#

it moves them from downloads to content

glossy inlet
#

no thats steam

smoky halo
#

no thats launcher

glossy inlet
#

It gets moved for me without ever opening the launcher

smoky halo
#

it was in downloads until I launched the launcher and steam didnt have any option to install or anything

glossy inlet
#

I just tried

#

it got moved

#

without launcher

#

It's steam

smoky halo
#

why id dint work for me

glossy inlet
#

maybe some old files that steam forgot about

#

and the launcher poked at

smoky halo
#

dunno, i even reistalled steam

#

i used beta though maybe it was due to that

#

but no

glossy inlet
#

steam does the same with game downloads too

#

download into a "downloading" folder, and move to common when done

smoky halo
#

makes sense only it didnt for me and moved when I lauched the launcher

#

i downloaded hours ago

#

switched pc off

#

went shopping

#

came back

#

started it, steam updated, the mod was still in downloads

#

clicked on launcher.exe boom

sonic pond
#

I honestly have no idea what category this falls under however my question is …. how do I sign my pbos with my bikey?

smoky halo
#

Should be sign utility in A3 tools

scenic canopy
#

not sure where to put this really but did you consider improving the SEO on Steam Workshop for Unsung by not having the underscores in the title like I mentioned previously @quick fog ? or have did you tried it already?

#

currently it's almost impossible to find it by searching on steam workshop

#

as for example our mod "Anrop Unsung JSRS" gets a perfect match when searching for "Unsung JSRS"

#

"unsung vietnam mod" puts unsung at place 3, after "jungle threat" and "vietnam"

#

"unsung vietnam" puts unsung at place 9

#

based on limited testing it seems like steam doesn't split on "_" in their search engine properly

#

"unsung vietnam war" results in place 1 at least 😛

quick fog
#

we use google

scenic canopy
#

it's a shame the pbo header doesn't have offset as property. if so data could be deduplicated inside pbos. would've been useful for pbos with mission collection pbos using same assets/scripts. on the other hand it would've been waste of header space for everything else 😛

glossy inlet
#

? header has offset to files. And you can deduplicate duplicate files in same pbo 🤔

#

Or doesn't it?

#

Isn't it offset,size for each file?

#

I wonder if you can just insert dummy files with negative size to move the pointer around 🤔

karmic niche
#

Isn't this an offset to the start of files chunk? edit: there's no offset at all

glossy inlet
#

It's been too long 🙈

karmic niche
#

And then all files declared in the header are simply concatenated, one after another?

scenic canopy
#

I thought it had offset as well at first but doesn't seem like it

#

was a while since last time I did a pbo parser 😛

#

is there even an offset at all? isn't it just that data starts after the last "empty" header entry?

#

negative size, that's an interesting approach 😄

#

signed vs unsigned etc 😛

#

datasize is unsigned long

glossy inlet
#

maybe you can make it overflow 😄

scenic canopy
#

😛

vague shard
glossy inlet
#

yes

scenic canopy
#

oh right, I never finished and released my web based PBO tools 🙈

neon flax
#

👀

scenic canopy
#

I got as far as browsing around inside PBOs

#

I was working on extracting files and previewing images etc

fossil glacier
#

@scenic canopy i was working on an online pbo manager too :D Mine is on arma.zone and the git repo for pbo.js will be going up today

glossy inlet
#

web assembly for the win

scenic canopy
#

nice 👍

#

you should add a derapifier for binarized text files

#

and paa preview 😛

fossil glacier
#

It's not really done, maybe gonna add support for compression as well and some sort of deobfuscation stuff

#

Could make another js library for PAAs and binarized stuff and other arma file formats

scenic canopy
#

@glossy inlet yes, using armake and wasm makes things very easy 😄

glossy inlet
#

Could make another js library for PAAs
Already exists, even a official one from BI, on their github

scenic canopy
#

only jpg/png -> paa/pac though

glossy inlet
#

Just found -command= start parameter. It lets arma open a named pipe via which you can
-shutdown the game
-send systemChat messages into the game
-retrieve session information (playerid, terrain name, mission name, current server IP)
-connect the game to a server (ip,port,password)

Weird that this interesting thing is not documented. I guess I'll write a blogpost about that and other stuff 🤔

smoky halo
#

But does it work?

glossy inlet
#

couldn't try it yet. but i think i already found that in the past.
I doubt that the connect to a server stuff works properly. But for the other stuff, I can't see a reason why it wouldn't

smoky halo
#

Sometimes things in arma don’t work for no reason

neon flax
#

maybe it would be possible to use it as a poors man rcon when not using BE?

#

Ping me when you'll be releasing the blog post, I'm interested what you will find ;D

scenic canopy
#

I guess it's more intended to be used with clients?

glossy inlet
#

yeah.. well if you can use it to tell a client to connect to a server, it's clearly intended for client.
And it prints to systemChat which also makes no sense on dedicated server.
And there is nothing useful in there for rcon

scenic canopy
#

I wonder what session returns for a server, if it works at all

glossy inlet
#

Only works on windows.
It's supposed to work on server, it returns a boolean whether it's currently hosting a server or not.
It doesn't return the IP. Just terrain and missionname.
🤔 someone made a discord arma thing right? maybe that could be done via that, without extension

scenic canopy
#

you can just query the server for most info 😛

#

using steamquery

#

I use that for state info in my server manager

#

localhost query

#
{
  "environment": "w",
  "folder": "Arma3",
  "game": "Escape Stratis CUP USMC vs RU 1.9",
  "gameid": "107410",
  "listentype": "d",
  "numbots": 0,
  "numplayers": 0,
  "port": 2302,
  "protocol": 17,
  "rules": {
    "\u0001\u0003": "\u0003\u0001\u0002\u0001\u0003\u000b\u001b\u0001\u00021�\\1\u001c�W��n�\u001a׃\u0006�w̗E\u0003�K�\u0001\u0003,\u0001\u0002%�:\u0005����\\��\u0001\u0002Q�\u001b�*\u0007}�\u0018J\u0004\u0011�S5\u0004TFARG�FT\u0004尩\u001d\u0012CUP Weapons 1.12.05(��\u0004s�L \u0013CUP Vehicles 1.12.",
    "\u0002\u0003": "0BT��\u0004ڷ�\u001d\u0010CUP Units 1.12.0�\u0003\u0016\u001f\u0004�0�\"\u001aCUP Terrains - Maps 1.12.0c�\bf\u0004�q�\"\u001aCUP Terrains - Core 1.12.0Λ\u0005 \u0004\u0015��\u001a\u001dCommunity Base Add",
    "\u0003\u0003": "ons v3.12.0\u0001\u0002"
  },
  "secure": 0,
  "steamappid": 0,
  "steamid": "90127935456967680",
  "tags": "bf,r194,n0,s3,i3,mf,lf,vf,dt,tescape,g65545,h30261c92,f1,c-2147483648--2147483648,pw,e0,j0,k0,",
  "version": "1.94.145903"
}
#

probably should update library that queries that info 😛

vague shard
#

is the lib on github?

#

@glossy inlet is this connected to Steamworks, or not?
maybe the left-over from Ondra from late A2:OA days when he implemented named pipe for withSIX (and other tools) - i thought it wasnt ported to A3 source

scenic canopy
#

but any steam query library should work

#

above is the raw output

#

it also offers a more santized version

glossy inlet
#

no that stuff is not steam

#

Is there really no good server query library for C#? I can only find things that were abandoned since 2014

smoky halo
#

Why c#

glossy inlet
#

Cuz my webapp is C#

smoky halo
#

Steamworks docs all c++

scenic canopy
#

Its not the same as the steam master server

smoky halo
#

But some messed up

#

I guess no one really maintains even official ones

glossy inlet
#

I have a steam C# library already, but it only does web api stuff

smoky halo
#

You need spastic callbacks and shit for Steamworks

glossy inlet
#

I don't want steamworks

smoky halo
#

What do you want

glossy inlet
#

steam server query

scenic canopy
#

😛

smoky halo
#

So you want protocol and direct socket connection?

scenic canopy
glossy inlet
#

yeah sure I can write it myself. But guess at that point I'd rather just write a intercept extension on my server to just send me the data

scenic canopy
#

Theres a list at the bottom

glossy inlet
#

I've seen that. Abandoned since 2014

smoky halo
#

Arma uses Steamworks api

glossy inlet
#

Yeeeees I know

smoky halo
#

So you gonna be using that if you gonna do it through intercept

scenic canopy
#

I can check my c# archive, I did reimplementation of our manager in c#

#

@smoky halo no, you don't need to

#

if you just need basic server info you can query with udp instead

karmic niche
#

@glossy inlet Regarding parsing the steam packets, I'm considering rewriting my steam library (yes, I'm writing one) to Kaitai Struct (I'm currently using python-only Construct).
Kaitai generates parsers in different languages, so once I do that, you could use my rules to generate a parser for the packets and the rules entry, even in C#

#

But that's only if you're willing to implement the UDP packet sending yourself

glossy inlet
#

So you gonna be using that if you gonna do it through intercept
no. I'll just use scripts on allPlayers 😄

if you're willing to implement the UDP packet sending yourself
I'm not

smoky halo
#

so the new protocol v3 is supposed to have GM listed with dlc flag, and I think it does, but the name is blank

Custom Mod [DLC - yes]:
Custom Mod [DLC - no]: @Blastcore
Custom Mod [DLC - no]: SMA
Custom Mod [DLC - no]: Bundeswehr Mod
Custom Mod [DLC - no]: TKO Addon
Custom Mod [DLC - no]: Community Base Addons v3.12.0
Custom Mod [DLC - no]: Extended Base Mod
Custom Mod [DLC - no]: Exile Mod
#

can anyone confirm?

glossy inlet
scenic canopy
#

👍

#

hah, nice lookup 😉

smoky halo
#

why bots is 100?

glossy inlet
#

I needed a server I could quickly find the IP for :D
But playerlist is broken, and "VAC" is "49" although it should be a boolean flag.
Need to fix it up a little I guess

smoky halo
#
address: 66.70.180.218:2312
ping:  130
respondedwell:  true
donotrefresh:  false
gamedir:  arma3
map:  Anzus_Life
description:  Kamdan Life 2.0
appid:  107410
players:  35
maxplayers:  120
botplayers:  0
passworded:  false
secured:  false
lastplayed:  0
serverversion:  194145903
gametags:  bt,r194,n0,s7,i3,mf,lf,vt,dt,trpg,g65545,h4c502ad3,f0,c-2147483648--2147483648,pw,e15,j0,k0,
steamid:  90127934130891779
server name:  ANZUSGaming Kamdan Life - mods @ www.anzus.life
#

wanna see rules?

glossy inlet
#

Hm. It doesn't answer the A2S_PLAYER request.
It sends the challenge request, get's challenge response, sends correct request. But the server just answers with a challenge response again 🤔
watching it in wireshark

smoky halo
#

If you read docs it says that protocol is for guidance because not everyone follows it to the t

glossy inlet
#

🤔 well if the server responds to a A2S_PLAYER request with a challenge response, I don't really know what to do now 😄

smoky halo
#
Version: 3
Global Flags: [00000000]
Required DLCs: [0000101111111111] Kart Marksmen Heli Curator Expansion Jets Orange Argo TacOps Tanks Enoch
Difficulty Index: 3
AI Level Index: 3
Flight Model: Standard
3rd Person Camera: yes
Weapon Crosshair: yes

N Custom Mods: 1
Custom Mod [DLC - no]: ANZUSGaming Kamdan Life

rules for the same server

glossy inlet
#

the protocol in wireshark looks correct to me, and I don't have any tool right now which is known working that I can use to compare

smoky halo
#

player names returned look like AI names

glossy inlet
#

I want to use it as some automated way to archive mission attendance. When a mission is running just query the server every 5 minutes and see who's there and who's not.
Might aswell just scan the teamspeak logs but I thought this be a nice gimmik ^^

smoky halo
#

send list of players from the mission

glossy inlet
#

or I can probably just parse the server console log

smoky halo
#

or hook on player connected disconnected

#

is that your server?

glossy inlet
#

anzus gaming is not my server no 🤣
I think parsing console log is easiest, as it also gives me steamID and is easier to access, no need for extension.
My webserver stuff runs on same server and I already access mpmissions from it

scenic canopy
#

we store attendance with our AAR

smoky halo
#

Good idea so you can ban some players for truancy 😂

native kiln
#

@glossy inlet is checking the connected playernames via rcon not an option?

#

then you could just do a quick connect, grab the players and disconnect again

glossy inlet
#

I don't have rcon

native kiln
#

you dont have rcon but access to the whole thing? oO

glossy inlet
#

it runs on my server

#

and i don't use battleye

native kiln
#

ah

#

its very weird, i remember these linked libraries working just fine and pulling names etc without issues

#

but that was like 2 years ago

scenic canopy
#

yeah, names should work

native kiln
#

steamquerynet only returned gibberish for names when i just tried it

scenic canopy
#

I'd guess the decoding is wrong

glossy inlet
#

same for me. But that's because the server doesn't reply properly to the request.
Every request with challenge doesn't work. Even though according to wireshark the data it receives/sends is correct

scenic canopy
#

same for other servers?

native kiln
#

tried 3 different ones

glossy inlet
#

Haven't tried other servers 🤔

scenic canopy
#

I can't say for 1.94 but I'm 100% certain that before we got valid names for players

#

anzus always brags on public forums about their cool anti cheat etc

#

maybe they obfuscate some data?

glossy inlet
#

Wireshark says
Send challenge request
FF FF FF FF 55 FF FF FF FF
Get challenge reply
FF FF FF FF 41 4B A1 D5 22
Request A2S_PLAYER with challenge
FF FF FF FF 55 4B A1 D5 22
Get challenge reply
FF FF FF FF 41 4B A1 D5 22

Don't understand why the server answers with a challenge reply to a valid A2S_PLAYER request.

scenic canopy
#

do you have ip + port for anzus?

glossy inlet
#

I doubt that's their doing

scenic canopy
#

and I can check with our lib

glossy inlet
#

66.70.180.218:2313
that's already query port

#

M242 already tested rules and that worked.
rules also didn't work for the .net lib there

scenic canopy
#

nvm, m242 posted it 😛

#

🙈

#

"error":"Received more than one challenge key","query" thinkdark

smoky halo
#

This is what I have

Player: Thomas Red
Time on server: 10004.2
Player: Shroud Schneider
Score: 0
Time on server: 10004.1
Player: Luke Polski
Score: -2
Time on server: 9998.96
Player: Joe Smalls
Score: -1
Time on server: 9887.31
Player: Will Turner
Score: 0
Time on server: 9779.5
Player: Buddha Dubbs
Score: 0
Time on server: 9700.55
Player: Mick Carter
Score: -10
Time on server: 9043.49
Player: Willem van der Boterkoek
Score: 0
Time on server: 7878.72
Player: Willem Van De Koek
Score: 0
Time on server: 7733.61
Player: Nick Catt
Score: 0
Time on server: 7448.06
Player: Daniel Wyrus
Score: 0
Time on server: 7015.28
Player: Dan Delgado
Score: -5
Time on server: 6898.78
Player: Jack Krays
Score: -2
Time on server: 6015.69
Player: Aiden Pearce
Score: -1
Time on server: 5942.42
Player: Jimmy  Hunt
Score: 0
Time on server: 5766.95
Player: Bill Jenkins
Score: 0
Time on server: 5758.39
Player: Tomas Richardson
Score: 0
Time on server: 4348.29
Player: Taylor Spogell
Score: 0
Time on server: 3415.19
Player: Dominik Thomson
Score: -1
Time on server: 2991.84
Player: Michael Wellington
Score: -1
Time on server: 2975.38
Player: Bob Fraggerarm
Score: 0
Time on server: 2720.81
Player: Bob Kilbourne
Score: -1
Time on server: 2671.44
Player: Jamie Sabic
Score: -7
Time on server: 1495.15
Player: Caleb Fazio
Score: 0
Time on server: 818.577
Player: Ray Jackson
Score: 0
Time on server: 697.809
Player: Connor Sumting
Score: 0
Time on server: 464.25
Player: James Snow
Score: 0
Time on server: 460.381
Player: Colby DeCuz
Score: 0
Time on server: 448.844
Player: Charl Jargost
Score: 0
Time on server: 390.866
Player: Terry Black
Score: -2
Time on server: 250.092
Player: William Sumting
Score: -2
Time on server: 175.013
Player: Jack Knight
Score: 0
Time on server: 63.6116
Player: Jay Reagan
Score: 0
Time on server: 32.8142
Player: Jot James
Score: 0
Time on server: 22.6068
Player: Nathan Reed
Score: 0
Time on server: 8.61989
#

from that server

glossy inlet
#

I guess there is some arma specific stuff that M242's thing takes into account

smoky halo
#

no I use steamworks

native kiln
#

maybe steamkit has an implementation for querying a server

smoky halo
#
ISteamMatchmakingPlayersResponse *fnc2 = new ServerPlayers(srv->GetName(), iServer);
SteamMatchmakingServers()->PlayerDetails(sip, sport + 1, fnc2);
#

why AI names though

native kiln
#

its not AI names, thats roleplay names

smoky halo
#

so you have to create a roleplay profile?

scenic canopy
#

Bob Fraggerarm

native kiln
#

some roleplay servers force players to rename their profiles to something that can be taken as a full name

#

atleast ive never seen good ol' bot Connor Sumting and his brother William Sumting 😄

smoky halo
#

Mick Carter is doing well -10

#

I guess he hates Life

#

rules also didn't work for the .net lib there
rules need to be decoded first then parsed

#

the rules for rules are

std::vector<unsigned char> DecodeData(std::string const &data)
{
    static unsigned char constexpr NullValue = 0x00;
    static unsigned char constexpr EscapeValue = 0x01;
    static unsigned char constexpr ZeroEscapeValue = 0x02;
    static unsigned char constexpr FFValue = 0xFF;
    static unsigned char constexpr FFEscapeValue = 0x03;

    size_t const countData = data.size();
    std::vector<unsigned char> decoded;
    decoded.reserve(countData);

    for (size_t pos = 0; pos < countData; ++pos)
    {
        if (data[pos] == EscapeValue)
        {
            if (++pos < countData)
            {
                if (data[pos] == EscapeValue)
                    // 0x01 0x01 -> 0x01
                    decoded.push_back(EscapeValue);
                else if (data[pos] == ZeroEscapeValue)
                    // 0x01 0x02 -> 0x00
                    decoded.push_back(NullValue);
                else if (data[pos] == FFEscapeValue)
                    // 0x01 0x03 -> 0xFF
                    decoded.push_back(FFValue);
            }
        }
        else
            decoded.push_back(data[pos]);
    }

    return decoded;
}
scenic canopy
#

okay, I got it to work with UDP

#
{
    "bots": [],
    "connect": "66.70.180.218:2312",
    "map": "Anzus_Life",
    "maxplayers": 120,
    "name": "ANZUSGaming Kamdan Life - mods @ www.anzus.life",
    "password": false,
    "ping": 115,
    "players": [
        {
            "name": "Thomas Red",
            "score": 0,
            "time": 13982.4521484375
        },
        ....
        {
            "name": "Sebastien Smith",
            "score": 0,
            "time": 12.671430587768555
        }
    ],
    "raw": {
        "environment": "w",
        "folder": "Arma3",
        "game": "Kamdan Life 2.0",
        "gameid": "107410",
        "listentype": "d",
        "numbots": 0,
        "numplayers": 37,
        "protocol": 17,
        "rules": {
            "\u0001\u0001": "\u0003\u0001\u0002\u0001\u0003\u000b�\u0001\u0001\u001b���\u001aj���n5�qJ�\u0019�+��I\u0018nv~�\u001fי�\f���\fh7W���\u0001\u0001�ۙ\u000f\u0004@\blO\u0017ANZUSGaming Kamdan Life\u0002\u0002a3\nkamdanlife"
        },
        "secure": 0,
        "steamappid": 0,
        "steamid": "90127934130891779",
        "tags": "bt,r194,n0,s7,i3,mf,lf,vt,dt,trpg,g65545,h4c502ad3,f0,c-2147483648--2147483648,pw,e15,j0,k0,",
        "version": "1.94.145903"
    }
}
#

omitted most of the player list 😛

#

@glossy inlet seems to work fine with the .NET library too

native kiln
#

is that net framework or net core?

glossy inlet
#

I use core

#

why does library not work for me do :U

native kiln
#

it doesnt work for me either, exact same code

scenic canopy
#

good question, either mono or core thinkat3d

#

net core

native kiln
#

now im even more confused

glossy inlet
#

maybe it only doesn't work for german IPs 😄

scenic canopy
#

I had some problems at first too but then it worked thinkinghuh

smoky halo
#

Sounds like undefined behaviour

native kiln
#

*surprise mechanics

scenic canopy
#

Tired server? 😛

karmic niche
#

@smoky halo Could you tell me what to do with these rules for this server?
https://pastebin.com/2wFnEJhF

Because what I'm getting (I was at the time of the data capture, at least) is two sets of rules, one containing 6 chunks and another one containing 7 chunks (15 chunks total)

#

That's what I get

ListContainer:
    Container:
        name = b'\x01\x06' (total 2)
        value = b'\x02\x01\x02w\x03\x89\x01\x01\xd5\x16N\xf5\xb9\xa3\xfb='... (truncated, total 127)
    Container:
        name = b'\x01\x07' (total 2)
        value = b'\x02\x01\x02w\x03\x01\x02\x01\x02\xd5\x16N\xf5\xb9\xa3\xfb'... (truncated, total 127)
    Container:
        name = b'\x02\x06' (total 2)
        value = b'TREBUCHET\x0c\xe7\x85A\x04\x9b0'... (truncated, total 127)
    Container:
        name = b'\x02\x07' (total 2)
        value = b' TREBUCHET\x0c\xe7\x85A\x04\x9b'... (truncated, total 127)
    Container:
        name = b'\x03\x06' (total 2)
        value = b'tillery"\x01\x01\xddL\x04\xb8y\xb2'... (truncated, total 127)
    Container:
        name = b'\x03\x07' (total 2)
        value = b'rtillery"\x01\x01\xddL\x04\xb8y'... (truncated, total 127)
    Container:
        name = b'\x04\x06' (total 2)
        value = b' Mod]+|\xeb\x04\xca&#&\x0fAd'... (truncated, total 127)
    Container:
        name = b'\x04\x07' (total 2)
        value = b'g Mod]+|\xeb\x04\xca&#&\x0fA'... (truncated, total 127)
    Container:
        name = b'\x05\x06' (total 2)
        value = b'@CONSOLEq\xc7\x16\x9a\x04\xf7k\x9d'... (truncated, total 127)
    Container:
        name = b'\x05\x07' (total 2)
        value = b'\x08@CONSOLEq\xc7\x16\x9a\x04\xf7k'... (truncated, total 127)
    Container:
        name = b'\x06\x06' (total 2)
        value = b'\x9a\x04\xaf\xba\xaf-\x12KKA3 ACE '... (truncated, total 127)
    Container:
        name = b'\x06\x07' (total 2)
        value = b'5\x9a\x04\xaf\xba\xaf-\x12KKA3 ACE'... (truncated, total 127)
    Container:
        name = b'\x07\x07' (total 2)
        value = b'\x02' (total 1)
scenic canopy
#

isn't that the bikey data or whatever?

karmic niche
#

What do you mean? In the link, you can get the raw a2rules response that I received and above, you can see what I made of it, after preprocessing it

#

(and before concatenating)

#

Also, what's interesting is that they seem to be shifted by one byte. Maybe an off-by-one bug in Arma that causes the sets to be duplicated?

scenic canopy
#

bikey data is 127 bytes minus the header, I think?

#

not sure 😛

#

I'm probably mixin it up with something else

karmic niche
#

I still don't really understand what you mean by "it's the bikey data"

scenic canopy
#

that's the A2S_RULES payload

karmic niche
#

I know that link 😉

scenic canopy
#

mod data is included in there

#

chunk format is there too

karmic niche
#

Read my first message again, please

scenic canopy
#

I don't understand your question then 😛

karmic niche
#

Because what I'm getting is two sets of rules, one containing 6 chunks and another one containing 7 chunks (15 chunks total)

#

There are 15 chunks in this a2s_rules packet

#

Check the name fields in what I posted above

#

you should get keys: 1/6, 2/6, 3/6, 4/6, 5/6, 6/6

#

And What I have is keys: 1/6, 2/6, 3/6, 4/6, 5/6, 6/6, 1/7, 2/7, 3/7, 4/7, 5/7, 6/7, 7/7

#

15 in total

scenic canopy
#

multi packet?

karmic niche
#

Don't remember. If that was multipacket, I have unpacked it into a single packet and THEN parsed

#

Besides, the key name doesn't have anything to do with the lower level being single or multipacket

#

And that's something that I also had a year ago when I was toying with these servers. Back then I thought it was a bug in a library that I was using. But now, I have reimplemented everything myself and got the exact same error

smoky halo
#

You will need to know how to manipulate bits

#

Tired server? 😛
Actually it failed for me now so it is not guaranteed you get response when you ask

#

Now I remember it happened before

karmic niche
#

@smoky halo okay, I see that I have formulated my question in a way that suggests that I don't know what I am doing. I do know what I'm doing :D

Please read my discussion below: the a2s_rules packet contains TWO sets of chunks, one set of 6 chunks and one set of 7 chunks instead of only one set of X chunks.
The data in both of these chunks (after concatenation, separately) seems to be duplicated and differs by only one byte in the first chunk (of both sets)

#

And I need to repeat: I have had this issue by both using an external library and by using my own parser 😛

#

This is not the first time that I find a server that returns such an a2s_rules packet and I though that you guys may have a solution for this

#

If you don't believe me, just parse the packet yourself, the one I have uploaded to pastebin, above

hushed parcel
#

Is there a c# pbo tool that can process preprocessor commands?

#

or any other

scenic canopy
#

the most common ones I guess are arma 3 tools, mikero's or armake(2)

smoky halo
#

I dont know how your data is put together. Arma puts the mods in gamedata keyvalue pairs, when you request rules, steam sends them to you 1 value at a time, you have number of packets and number of packet you supposed to append in the right order. Then decode. if you have duplicate data maybe your library is not working correctly

#

This is how I collect them

void RulesResponded(const char *pchRule, const char *pchValue) override
    {
        size_t total = static_cast<unsigned char>(pchRule[1]);
        size_t index = static_cast<unsigned char>(pchRule[0]);

        if (message.size() < total)
            message.resize(total);

        message[index - 1] = pchValue;
    }
karmic niche
#

Like... both of them?!?

#

Have you tried parsing all of the servers available out there, for example?

#

There's always at least one that gives that strange rules set

smoky halo
#

this is callback function, it will be called as many times as there are key/value emtries

karmic niche
#

I mean both libraries?

smoky halo
#

Arma takes mod information then chops it into parts that fit key/value game data limit

karmic niche
#

Thing is: I'm giving you a whole packet that I have received. AFAIR rules are never multipacket because... Arma.

#

Guys. Really. I know!

#

Just please take me seriously 😦

smoky halo
#

key becomes packet number and total packets, and value is the payload

karmic niche
#

Yes. And now look at what i got, please

#

I have a response that has TWO different "total packets" numbers, at once!

smoky halo
#

I don't know if I can help you. 1. I have no idea how the rules you got put together 2. I use steamworks API. Ask Dedmen maybe

karmic niche
#

under the "rules" key, of course

#

Concatenating rules' values is Arma's invention, isn't it? So steamworks API doesn't have anything to do with this, I think

#

Thing is, again (for everyone that reads this):
I have found a server (for the nth time) that returns an a2s_rules packet that seems to contain keys that don't form ONE set of rules but TWO

#

Now, my question: has anyone looked at other servers besides their own, found such a server as well, and figured out what to do with it?

smoky halo
#

The rules are one or multiple key value pairs. Again no idea how the library you are using puts it together to give you one continuous chunk

karmic niche
#

It does not put it together since I'm just parsing it myself and inspecting the keys/values myself, by hand. Has anyone stumbled upon such a server as well?

#

And, as I said previously: a library that i was using a year ago (so not my own code) also got me the same broken (?) set of rules so I'm confident it's not a bug in my code

smoky halo
#

What’s the server you found?

karmic niche
#

It's aaaaaall in the pastebin link

smoky halo
#

I’ll check it in a bit

karmic niche
#

Not sure if that server is still returning that data. I would have to check it again now, and I cannot do that. Maybe tomorrow

#

Thanks @smoky halo

smoky halo
#

its not there

glossy inlet
#

@hushed parcel SQF-VM has C# bindings afaik

nocturne basin
#

Lets not call it bindings @glossy inlet

#

It is more like a very very basic cli interface 😂

#

Wrapper*

glossy inlet
#

Then make proper bindings 🏬

scenic canopy
#

it's a shame that the bis-file-formats repo only has derap if I remember correctly

#

it's very nice for C# otherwise

hushed parcel
#

k ty

glossy inlet
#

For reference about what my steam query stuff will go into. Currently building a web interface for my unit.
Our mod management system lets members request mods, comment, vote, review.
Displays an indicator if an update is available that we haven't installed yet.
And now also displays pbo list (thanks to workshopcrawler) and changelog entries that happened since the mod was last updated in our modpack.
It also has integrated mission planning/attendance stuff, and mission upload to server, and it also now checks serverlogs to see if members that signed into a mission really took part.
I could probably sell this shiz.

nocturne basin
#

not before you at least put it into some more fancy UI

#

i just got cancer only from seeing this

glossy inlet
#

well as you can see UI is very WIP. I'm waiting on others to do the designing for me

nocturne basin
neon flax
#

just slap bootstrap on it and it will be fine 😄

native kiln
#

I love how the ServerControl tab has a wheelchair as it's symbol

glossy inlet
#

That's actually the admin tools tab. It's a dropdown menu with servercontrol, member absence list and mission list

scenic canopy
#

😛

#

and also ties into our native client launcher with automatic subscription of mods etc

glossy inlet
#

We have our own mod launcher which takes care of that.
Mission listing only shows the modlist. I'll also add that a warning is displayed in web app if the correct mods aren't currently selected in the launcher

scenic canopy
#

the web admin for the servers also ties into the site listing active game sessions with required mods etc

#

since not all popup games during the week have a signup event on the site

#

and the recorded AAR can be linked to the event for attendance tracking, player stats etc

#

2730 recorded game sessions 😛

neon flax
#

Right now I'm preparing to write something similar with guys in my group.
Web api where administration can add-remove workshop mods, worker on server that will download/update the mods on server. (planning to use SteamKit for that)

#
  • additional page where mission maker can create mission and add available mods to it.
#

I'm planning to opensource all of that when it will be done. (most likely not until next year pepe_sad not enough time)

native kiln
#

@neon flax tell me when you've written some code that downloads via SteamKit

neon flax
native kiln
#

I use SteamKit too for the crawler, but damn is it hard for me to keep CPU usage low and maximise download rate

neon flax
#

They already have demo project for that

native kiln
#

Meh DepotDownloader has really messy code

neon flax
#

Haven't checked it yet.

native kiln
#

Plus it doesn't perform well as far as I could tell

#

I tried a couple of things but it's either fast download rate and high cpu usage or lower CPU usage but not so great download rates

neon flax
#

Plus I've not coded in .NET for sooome time (and never was good at it, lol) so I need to re-learn it first 😄

scenic canopy
#

I noticed that our JS downloader spends a lot of time on crypto so I'll need to make it properly async for concurrent downloads

#

the default crypto lib in nodejs is sadly synchronous

native kiln
#

I haven't done proper profiling to check where my issue comes from but I'm guessing yeah it's either crypto for me too or I'm just using something the wrong way

#

Steam just casually downloads with 500mbit and low CPU usage, meanwhile I roughly hit 400mbit but almost nuke my 4 core in the process

#

Something's really bad 😬

neon flax
#

This DepotDownloader code looks very verbose.

#

I hoped it's a little bit more high level library 😬

native kiln
#

Negative

neon flax
#

Like "here is steam id, download it to this folder"

native kiln
#

I made a small wrapper for SteamKit myself if you want to use it as a reference

scenic canopy
#

depotdownloader is a cli tool, not a library 😛

neon flax
#

but I see it's handling the chunks and everthing.

scenic canopy
#

it's funny because the same week I did the steam workshop db they changed how CDN urls are handled 😄

neon flax
#

I know, but I hoped that the Library used for it is more high level.

scenic canopy
#

it's better if it doesn't do it high level since it would be hard to implement delta patching etc if the chunks are abstracted

neon flax
#

I made a small wrapper for SteamKit myself if you want to use it as a reference
Do you have it on GH or somewhere? I want to gather as much info as I can before we will start writing our stuff.

scenic canopy
#

if you're doing client stuff you should use steamworks instead

native kiln
#

@neon flax whats your gh username?

scenic canopy
#

then you need to handle steam guard etc 😛

neon flax
#

same as here without KP tag 😛

scenic canopy
#

cause those are done by steam if you're using steamworks

native kiln
#

steamkits steamguard stuff is pretty straightforward

#

theyve got an example for that too in their repo, its a few lines

neon flax
#

I have steam guard disabled 🤷

#

I dont need it on server account

scenic canopy
#

entirely or still email? 😄

neon flax
#

entirely, Bought arma for the server account

#

generated 32 chars or smth long password 🤷

scenic canopy
#

cause afaik steam adds email steam guard by default for "new" accounts

#

so you would need some way to provide that into your application

#

from personal experience the SteamRE downloads are quite slow

#

takes quite a while to download for example the GM depot with it

neon flax
#

It's not like it will matter that much, I just need to get mods on the server.

#

Update them from time to time.

#

I don't need to proces streams of data like the workshop crawler.

scenic canopy
#

if you can use Steamworks I'd definitely recommend it over SteamRE

neon flax
#

With steamworks I would need to have steam running in background wouldn't I?

scenic canopy
#

correct

neon flax
#

I wonder if you can just open SteamCMD and keep it in background

scenic canopy
#

good question... I think no

neon flax
#

It seems to initialize like normal client. When I was downloading mods on my local server the desktop client was saying that it sees new streaming source.

scenic canopy
#

haven't tried that one thinkat3d

#

but I think it's the same with SteamRE, if I remember correctly it severed my Steam client connection

#

I think it's dependent on the steam client identifier which seems to use internal ip/mac if I remember it correctly

#

which is why you can have multiple clients within the same LAN

#

not sure about linux but running steam in the background is at least no issue on Windows

#

our CI boxes uses steamworks for all uploads of builds and mod downloads for arma integration testing on our GPU slaves

neon flax
scenic canopy
#

yep

#

high priority is scary, it might break the rest of the queue 😄

native kiln
#

What do you need to use Steamworks

#

running Steam?

#

or does it run independently of anything and you feed it your steam account data? never used it before

scenic canopy
#

steam needs to be running

#

it communicates with steam using the steamworks dll (steamapi)

#

so all auth is handled by steam

native kiln
#

so it means i need to have steam running at all times on my target machine

scenic canopy
#

correct

neon flax
#

It's the api for games to integrate with steam.

#

Vanilla launcher for eg uses it to tell the steam to subscribe mods etc.

native kiln
#

if I didnt had the requirement of being able to download specific files that would be been a great solution lol

scenic canopy
#

yeah, using manifests and steam cdn directly is a lot better for a workshop crawler

#

but for a more generic updating tool steamworks is a lot less to maintain

#

they've changed how steam cdn works at least once this summer already so 😛

native kiln
#

would have loved to use it just for the sake of having fast and efficient download code 😄

scenic canopy
#

yeah, it's very nice to work with

#

it's like a couple of lines for all download related stuff in our launcher

#

and then a boat load of code for bridging steamworks into reactive mvvm to keep ui updated 😛