#arma3_tools
1 messages Β· Page 26 of 1
does a refresher course on Jovial and Cobol PIC 9999 anyone?
feeds punched cards into the reader and walks away......
spilling the box's contents as he goes
Good old times - said no one ever
careers in IT used to end in that way folks.
btw. it looks like you can install DayZ tools only if you already own DayZ. What a bummer π¦
a sub license i guess, accredited to your username and profile on steam.
Aww yes. The DayZ Particle editor is neat https://s.sqf.ovh/DayZ_x64_2018-11-07_21-47-57.png
Output format:
EffectDef {
Emitors {
EmitorDef emitor1 {
ShapeType Sphere
BirthRate 5
BirthRateVel 56
Material "{735D641EB0FC5209}system/wbdata/particleeditor/defaultmaterial.emat"
CenterX 0.142
SizeRND 1
RandomRotDir 1
VelAngle 1
RandomAngle 1
VelocityRND 2
Wind 1
Color {
0 0.2763 0.3947 0.7039 0.1004 1 0 0.3816 0.6466 0.1447 0.9079 0.3618 1 0 0.2566 1
}
}
}
}
Doesn't look like you could easily convert to Arma particles
But does it look great?
Or is it retextured Arma particle?
Params are similar, different names
Looks great to me π Dunno if it looks great to you
of course params are similar, it's still a particle -.-
nice to see that they do develop new tools for it (and future engine)
anyone had success yet using dayz tools to pack a model.pbo ?
So wait is dayz using Qt for UI?
or just workbench?
I have no idea what any of this means when im looking at it :3
is there a fancy DZ SA/Enfusion modding discord to join?
@vague shard https://discord.gg/NEX27Gx
ty
Oh.. Yeah didn't try just double clicking a GUI layout file π
Mikero Tools interim update (prior to dayZ/SA compatibilty)
Rapify: (credit @sage zodiac )
significant improvements to macro expansions for arrays. Array manipulation know works as you would expect it to and is incompatible with bi's version of same. Which is irrelevant since bi doesn't work at all.
some minor irritants now auto fixed (like trailing commas, and spurious semicolons)
this again, is incompatible with bis compiler, but i don't let bis get anywhere near paramfiles.
pboPro (credit @limber moss )
protection agains deleting ~\source folders in temp, if a previous crash caused them to remain there (not fully tested)
derap: (credit @pliant lynx )
almost all 'text' files are now have utf8 boms . Lack of them was an oversight on my part for Windoze. Linux assumes utf8, windoze assume cp1252 (sorry @vague shard , this will affect you)
wrp:
better decoding of some structures
eliteness:
improved bitmap displays of the various wrp structures (further work still needed)
improved display details for some mapinfo items
enjoy
that whole bom crap is weird though ...
most of the time, you just can read the file perfectly fine π€· https://github.com/SQFvm/vm/blob/master/src/fileio.cpp#L35-L98
though ... utf-16 etc. would require special care
because you would need to proper parse the actual character
first time i experienced it, it drove me crazy
because it was randomly introduced by a random change
and i thought my bloody file loader was corrupted
as the first few bytes always have been garbage
most of the time, you just can read the file perfectly fine
tel that to any European using umlats
for an ascii file ...
yes, agreed
no
yes
localized page
a localised codepage assumes (for most people) cp1252
in parsing this also does not matters as you can just skip over that
they are incpmpatible with bis my friend
only relevance is anything larger then 8 bit
that is nothing you need to take care during read π
the umlats are mistranslated in bis (text) output and file accessing
it's actually anything larger than 7 bit X39
your general assumption is correct, for most files it doesn't matter. Until you hit the 7bit barrier
yup
at that moment however ... you are fucked anyways π
not anymore
maybe for you
but i am still
because either i introduce a wide character reader
or i just say "not supported"
now ... guess what i did π€·
utf-16 for sqfvm?
what about no?
will only parse the first byte right now as the rest is '\0' most of the time
utf8 16le is supported by bis. I cannot be certian of that with ALL sqf files
i suspect init * anything is screwed because they screwed up (that part of) the sqf input reader
all others should be ok
however, be that as it may, i only ouput utf8 into the pbo. So wide char is not relevant (at that point)
does not matter*
You (X guy) always make the same mistake
You always make the same mistake
Such people usually quickly change their attitude when they start being woken up at 4am because their code is crashing (again) on production π
bis, to their eternal credit, are almost fully utf8. they have made some mistakes, but they are minor.
Forgive me folks, i will repeat the important message again, because it gets lost in noise.
I will make no further comment because this channel rapidly gets poisonous with snipers:
Mikero Tools interim update (prior to dayZ/SA compatibilty)
Rapify: (credit @sage zodiac )
significant improvements to macro expansions for arrays. Array manipulation know works as you would expect it to and is incompatible with bi's version of same. Which is irrelevant since bi doesn't work at all.
some minor irritants now auto fixed (like trailing commas, and spurious semicolons)
this again, is incompatible with bis compiler, but i don't let bis get anywhere near paramfiles.
pboPro (credit @limber moss )
protection agains deleting ~\source folders in temp, if a previous crash caused them to remain there (not fully tested)
derap: (credit @pliant lynx )
almost all 'text' files are now have utf8 boms . Lack of them was an oversight on my part for Windoze. Linux assumes utf8, windoze assume cp1252 (sorry @vague shard , this will affect you)
wrp:
better decoding of some structures
eliteness:
improved bitmap displays of the various wrp structures (further work still needed)
improved display details for some mapinfo items
enjoy
Good evening, I would have liked to know how can apply normal with object builder?
ask in model makers
Bis strings are multibyte Iβd stay away from widechar if I were you
SQF-VM will be able to catch VM exceptions (stuff caused by eg. 1 + "") <3
now the real question is: why is the real try CODE catch CODE not able to do so?
I was just talking about your *VM project and I have a question: would it be able to do some static checks? (I know it doesn't currently do that but would it be hard to implement an extendable mechanism for static checks?)
Like right now, if you call an SQF function that doesn't exist, there is an error on run-time
But let's say that it would see that there is such a call and then go through all the existing functions and notice that that one doesn't exist and it WILL fail if you ever call it
I can see such a feature being used by programmers (static checks for the code)
If in enscript you have strong typing (you have to declare the type of a variable prior to using it) and you have a function that accepts other types of variables, calling a function with wrong variable types could also be automatically detected, etc...
i am not entirely sure what you mean by that
you mean pre-execution checks for stuff like that?
or?
Yeah, let's call it like that
Sanity checks without even actually executing the code
[] call notexisting could be theoretically catched, if one puts the effort into it
the vm is generating a CST and creating instructions from that
Someone can make typos in the function name, for example
but right now, there is no such check inside
and building it will require loads of work and all variables available at runtime π€·
but yes, possible
IIRC @scenic canopy wanted to do something like that (correct me if i am wrong)
as said: all possible
if you want to put the effort into it
and until then, stuff like sqf { 1 + ""; } except__ { diag_log _exception; }; would need to be enough for SQF-VM
btw. I wonder if it wouldn't be easier to implement an LLVM back/frontend for Enscript, if it's as close to C++/C#
BEXP4;0;140726677343296;
ARRAY;0;2;[]
BINARYOP;3;4;call
VARIABLE;8;11;notexisting```
is what that generates
And then you could actually have a real compiler generating real executables for it π
nah ... script languages get interpreters
real executables will cause trouble
as script languages tend to be quite changable
with methods appearing or disappearing at will
further talking, a static analyzer would then need to check for the BINARYOP call and look if it ever has seen the variable following before
yes, exactly
ASSIGNMENT;0;13;
VARIABLE;0;8;existing
CODE;11;2;```
and register that this would assign a variable
Or rather, to queue the check after it will have had processed everything
however, you also would need to execute partial SQF code if one creates variables using missionnamespace setvariable ...
True, but there is a place where you have to draw a line and state that you won't allow loopy programming π
treat em as warnings rather then errors and you would be fine π€·
And disable checks on a case by case basis
but as said: all possible
just somebody needs to do it
parsing literally boils down to this:
auto h = sqf::parse::helper(merr, dbgsegment, contains_nular, contains_unary, contains_binary, precedence);
bool errflag = false;
auto node = sqf::parse::sqf::parse_sqf("code as char*", h, errflag, "filepath");```
You could do: [] call dynamicname; // sqfvmignore:E123 and that would disable the check for Error number 123 on that one line
That's how other checkers do it
as said π do it
it is possible, i will not
especially since that is not the goal of the entire project
I will only consider it if it works with Enscript ;P
SQF-VM also was never supposed to be able to syntax check files
but it automatically was added when i actually needed to rewrite the parser from scratch
and yes, enscript will get the same sort of parser from me
my toolchain should be capable of generating a parser already (but only in C)
You previously said that not a lot of people use your sqfvm. That was just a feature that I really could see myself using if it existed, so I just wanted to share that thought with you. If you're not interested in ever implementing it, that's perfectly fine.
nah ... real static analysis is pretty much as time consuming as writing the vm would be π€·
but everybody will be able to use the parser as soon as i start writing one
That's why I said "extendable"
Like you provide a framework for writing static checks and other people (like me, for example) can then implement Check Number #24 for it
will do it exactly like i did with sqfvm π
no extra fancy stuff for it
so if one wants static analysis, it will be the same as usual: take the parser and "just do it"
@karmic niche there are already a couple of static linters of sqf such as https://github.com/LordGolias/sqf and https://github.com/SkaceKamen/sqflint
I don't think either of them actually validates a tree of functions thou
not 100% sure about that
@nocturne basin I had an idea to use it for unit testing, so close π
I don't think either of them actually validates a tree of functions thou
This is correct
probably wouldn't be impossible to add
preload a list of functions/files from a directory, make sure all call/exec/execVM matches
then you have variables with code added to global scope in files.... π
many of the editor plugins have preseeded lists of bis, cba and ace functions
that could also help I guess
ohh btw. @scenic canopy
you now actually then can do that π
preprocessing is in
ah, thats true
something to catch exceptions even at vm level
and the release is also out π https://github.com/SQFvm/vm/releases/tag/1.2.0-RC1
I wonder what would take the most time, validating all calls or just running all of them π
running all of them
running might be hard if they appear out of order as well
could be a cool feature if it was done well
dayz2p is now available for subscribers
expect moving goalposts while Julien has to wrestle with us <grin>
pboProject (and frendz) updated to binarise and pack dayz addons
expect wrinkles
binarsing dayz format wrps (maps) now works
What does dayz2p do differently than the workdrive included?
pls no bulli, i actually don't use arma2p either, i have some weird python stuff for silly specific unpacking instead there
also no subscriber tools
I updated my Armaclass library
(pure-python library to parse sqm class-like files)
It now also supports generation of sqm files from the parsed data.
It's supposed to let you programmatically update the file now.
Also I was asked if I could make a DayZ generator that would help you port your class-like files (like UI to DayZ .layout files). This commit is the first step in that direction.
License: MIT but PRs are welcome.
The parser is an almost 1:1 ported version of Fulsselwurm's JS parser.
The generator is mine.
Also I was asked if I could make a DayZ generator that would help you port your class-like files (like UI to DayZ .layout files).
Which madman would ask something like this
What does dayz2p do differently than the workdrive included
At the moment, nothing substantively different except it works (probably) 3 x faster. But as the bugs start pouring in, like arma3p , it will supply patches to silently fix (some of) them.. It already supplies some additionals if you're making maps, but nothing you can't live without.
you have the skill set to roll your own (using python) others don't,
The revised pboProject is a very different story and anyone who uses it already knows what addon breaker cannot do.
@karmic niche But let's say that it would see that there is such a call and then go through all the existing functions and notice that that one doesn't exist and it WILL fail if you ever call it
You cannot ever reliably do that. A function might not exist in the visible script files but still exist when you actually run the code. Whether it comes from a unknown mod, or is publicVariable'd by another client or server. Which is a thing that's rather commonly done in Arma. But with "ignore this one" comments plastered in tons of places that would work of course.
Also what about things like setVariable format ["playerstuff_"+str _forEachIndex]
@scenic canopy I don't think either of them actually validates a tree of functions thou SkaceKamen's Linter does that. It goes through CfgFunctions and warns you if a function variable might not exist. Sadly no support for CBA/ACE/TFAR/ACRE PREP.
Cool, havenβt noticed!
I constantly notice. Because it yells at all my functions for being undefined :u
ACE/CBA/ACRE/TFAR * ;)
if your goal was to note that T comes after A, you failed @pearl beacon
Note was ACRE missing. I added ACRE and reordered the mods after importance now.
π€· ok
so TFR is supported then? π
"importance" everyone knows the truth @glossy inlet π
CBA is clearly least important
Also what about things like setVariable format ["playerstuff_"+str _forEachIndex]
I don't want to sound like a smartass but I really don't care about pathological constructs :P. I'm talking about the possibility to add static checks to the Enscript language where you will be able to have normal structures so doing "playerstuff_"+str will be a big code smell and a sign of sloppy programming.
The same things that you've described can be done in Python as well (creating dynamic variable names, functions, etc...) that I happen to use on a daily basis at work and I don't remember having many issues with my linters. If you have to plaster your code with "ignore" comments, then something is clearly wrong with your code (you, lol!) and not the linter. Thus this shouldn't be an argument against the meaningfulness of such a linter.
π
oh WOW, there is a god, after all. Bis have reverted back to stringtable.csv for dayz instead of the bloody awful xml. I salute you bis, at least one change that makes sense.
xml (in this usage) was bloatware.
and no again, x39, the csv was and is tabulated so tha you can instantly see, the name you want in each language column. The methodlogy too was far superior in handling the 'default' language for the game (not necessarily Engles), and the default str to be used when one wasn't supplied was straightforward.
nope
it was useful due to no tools existing for XML to edit
CSV is a horrible format
you have a love of if its new, it must be better.
nope
it was not perfect
but far better then using CSV
because everything is better then CSV
only plaintext is worse
you have some things to learn. But you are also quite correct that (l;ack of) a decent xml editor made it shit.
CSV will be smaller, but that's about it, pros and cons to both...
there also is no standard for CSV that is proper implemented
eg. the actual separator, can be changed
but there barely is a parser that supports it
the xml was fine'
actual value can be passed with or without a string, strings however allow to use the separator inside of em
depending on your parser, this is either ignored, not implemented correctly (eg. string characters are kept) or some other fuckup
CSV itself is just horrible
and should only ever be used if you need to implement something quick and dirty
as it always was that: dirty
pass your opinion to excel, they'll be interested to hear your ways of doing things better.
and while your at it, perhaps you'll like to change the international standards for map making of xyz or asc.
Excel is not using csv @dawn palm
Only supporting it
hell, even the dos command line uses csv, but you know better.
We use tools to edit stringtables anyway. If it's CSV or xml in backend doesn't really matter in the end.
csv is smaller tho. And I like that alot π
π
well, for dayz, i'll re-activate my lintchecker for stringtables (in the dll). for xml, i could not be bothered.
Is there a way to turn off the behavior of makepbo to include the filepath to the file and get rid of the folders? I am calling it as this
makepbo -W myMod
" include the filepath to the file and get rid of the folders" can you explain what you mean?
i think he wants a non prefix pbo?
the answer if so, is yes, using the $ option
you could of course read the documentation.
or maybe pbo manager does not show it correctly, i am not too sure
there is a $PBOPREFIX$ is the base folder
from memory, it does not.
use eliteness
as intuitively easy to use as pboManager is (the did a great job there), they don't debinarse rvmat or config.bins for you when extracting. so it's more or less useless
i use it promarly to look into pbos, extracting is done with depbo most of the time on linux, as my AV really hates the windows installers...
what the hell is 'depbo' ?
aha sorry, meant extractpbo
haha
my tool will simply list contents of pbo with (from memory) the -b option
so i don't really understand your first question.
it was due to pbo manage listing the items in the pbo in a weird way. as in there was no folder structure.
hence my question. tho it seems the issues is not makepbo
agreed
you may not be aware that:
extactpbo NameOfPbo
will extract to a folder called, NameOfPbo\
that name is information purposeonly
the contents of the pbo itself, is, the contents of NameOfPbo . It is NOT part of the prefix
yeah I am aware of that. The reason I used extractPbo after posting the qustion is my lazy ness to fire up the linux shell on my windows pc. as mentioned, my av hates the installers for some reason, hence i work with the unix tools
fair enough. it's quite common for freeware av's to put nagware in the code to entice you to buy the real deal. Along the lines of 'suspected' virus. When you buy the real deal, those messages magically no longer happen.
Currently the best Av around is kapersky. That, will obviously change over time, but right now, that's where you should be.
doubt it, avira probably being dumb in that case
tho i managed to get some tools on my pc by extracting the binaries via 7zip. now i am too lazy to add the registry keys in windows
Β―_(γ)_/Β―
Can confirm that.
Can also confirm that kaspersky is nice. Their support is so professional that Mails get directly forwarded to the reverse engineering specialist somewhere in russia.
I don't use AV though as it's kinda useless and creates more problems than it solves Β―_(γ)_/Β―
The lib.so of the penguin version has more muscle power than the dll, it doesn't have to stuff around with utf16le file names and function calls, and sticks to ansi c (gcc) stdlib, Killswitch does a fantastic job of maintaining it.
only thing i would want more is the possibility to update the libs via apt-get
it's only one lib, by today's standards it packs one hell of a punch in such a small footprint that overcomplicating isn't too sensible. Although the debian versoion was a good one.
nothing stopping you creating it. providing you maintain it.
Sadly not nearly enough time and knowledge aboutit
so is this the DZ modding and tools situation atm?
- launcher cannot load modfolders
- server browser doesnt show mods yet - ingame browser is just placeholder
- game and workbench need same version or otherwise no worky
- no documentation
- no dev console ingame
- unclear situation of porting BI APL data, and your own stuff made with BI tools
- data extraction by DayZtoP.bat by mikero or tool by Kegan
- SP testing mod provided by Arma modders
- configs, models, textures, rvmat, wrp pretty much the same (what about sounds and their definitions?)
- animations, dialogs, "scripting" new/very different
- no call extension support yet
- basic car simulation, no tank/helicopter/plane simulation
- just basic zombie ai, nothing complex like RV ai (the configuration seems very exposed to configs though)
- DZ enfusion engine version needs DX11
anything else major missing or wrong here?
dayz p3d's can now be decoded via eliteness/dep3d etc
it has been reported that wrp and dep3d have been succesfully created using pboProject (and tested in game)
i am unaware of Kegan, can you ppoint me to a url please @vague shard
- there is a collision with ob for dayz/a3 because they both use the same registry settings
- animatons do NOT use rtm files
- sounds are ogg, wss (and probaly wav) are not supported
thanks kju, it's just a dialog button to the bis tool
(don't mean that disrespectfully. he's putting things back into the community)
unclear situation of porting BI APL data No that's not unclear. APL licensed data cannot be ported to other games. I'd call that very clear
You didn't mention particle effects. But otherwise I don't see much missing
APL/license situation is currently in discussion
APL pretty clearly states: Arma Only - You may not convert or adapt this material to be used in other games than Arma.
interesting to know how far that would stretch with cup objects, very few, if any, of the structures would have come from a2 anything nowadays
guys you may be dealing with stupid kids all day yes, but not everyone is stupid
the point here is if BI is to republish their APL data pack under a new license
and allows community people to adopt such new license too for their work if they so desire
vehicles and weapons won't fit the dayz scenario, so i don't see too much of an issue anyway.
so just because its called "DayZ" you are only allowed to make zombie mods?
well, if you make your own zombie, yes.
if ppl would had the same closed mindset with OFP/Arma, the modding community would not have gone anywhere
i've lost you a bit there kju. I don't see a big issue. There's not that much to port (from bis)
ahhhh, now i understand. you're thinking ahead to the enfusion engine, not daysSA
I can't see any problem for you porting ww2 to the dayz (enfusion) engine (other than technical)
who cares about DayZ in here - almost no one obviously
however its a test bed and learning experience for Enfusion, Enscript and its new parts
agreed fully
i would also assume BI will port further engine upgrades to it eventually as "early access" for the "full" Enfusion engine
maybe, but almost all of us are way past 'bis samples'
thats kinda the whole point here. like do you guys seriously expect "proper" documentation, importers, wide range of samples, etc???
haaaaaaaaaaaaaaaaaaaaaa
BI will care again about DayZ itself, its console releases, etc
I would expect them to purposely withhold tanks and planes implementation for Enfusion to have something "extra" to release for A4 tough...
yep
like one main thing i am really curious about is if Enscript really gives very low level access in the various areas
could we write our own AI now
our own editor
our own vehicle simulation
etc
or how extendable is the new editing platform really (workbench)
The question is how much work do you want to put into features that will be implemented on a lower level by the devs anyway (vehicle simulation for example)
"could/can" not "should"
if i am not mistaken, it was said enscript is written in enscript - was it not?
well, let's at least put one thing to bed, i don't think lack of apl_sa means anything anymore. no one needs it. (big foot in mouth there, but it aint life threatening)
I'd be begging nonov/milkman to give me their source mlods, not bis.
or how extendable is the new editing platform really well there are plugins. But the API doesn't seem that extensive yet.
our own vehicle simulation I've seen things like speedlimits and cutsom gear shifting already
yep it's a could/can. so a positive
in theory if I want simulation=pink fairy; i can write one.
I was already in awe when I saw that Mines are simulated in Enscript. From the placement over the click till the boom.
So many opportunities.
Especially for halloween missions XD
Introducing the "Boo"-Mine!
so far, my very limited, knowledge is a positive one:
+they use industry standard sound files which they cann't fiddle with and break
+they use industry standard dds files for directX which they can't fiddle with.
+they've got rid of that ghastly sqXshit. And they are forced to stick to function (rules) in the new syntax
+they've moved back to universal csv for stringtables (some one on the enfusion side must have smacked them)
Does anyone know if xml stringtable is still supported? I mean.. They may use csv for the game itself. But modders can still choose xml if they didn't cut the code out
i suspect it remains enabled.
probably using procmon checking what file it looks for at startup would give some insights
Well. It looks for pbo files on startup.
RV engine was looking also for some unpacked data at various locations - among them csv and xml
and various combos of config.xxx
The game loading a config or stringtable that's not inside a pbo and thus not protected behind any bisign checks?
Certainly a very good idea.
protection came later
obviously? buldozer mode still uses it
it's not like anyone would add some silly stuff to a config file outside of pbos, right?
no. i add silly stuff inside π
anyway, a positive folks, there's a lot in there we're used to, including the tools. so the learning curve won't be horrific.
for those of you making models, you're working with a type52 arrowhead p3d (lzo compression and no physx)
they call it a type53 because they've added one new pixelshader and an extra specular array from an rvmat.
for those of you making maps, there's no (effective) difference to an a3 type24 except your layer size (for roads) is restricted to 512 instead of 1024. Not sure if the 5+1 color range for tiles still applies, someone else can figure that out.
if they've reverted back to 4 tiles. frankly, you're better off with visitor3
@dawn palm
+they use industry standard dds files for directX which they can't fiddle with.
They are "Enfusion DDS" although I don't know the difference
You happen to know?
I noticed that and was at least happy with that though, means my trusty ol xnViewMP can already browse through files nicely
haven't looked furhter. they are 'edds' and 'dds'
they will be just differently organised pxcture details. size, or color depth or.... What they can't do, is change the format required for dds itself. So there will be no moving goalposts
any tools written for this stuff won't cbreak tomorrow because they changed the rules. they can't
my visual studio already 'understands' and edits one of them.
i hope an IDE like thing comes out for VS code or so comes out quickly for enscript
I'm using it more most things now and much rather use that than the script edtior in that program (Especially seeing you need to install the game to edit basic things)
yes. it's only mildly dissapointing that semicolons aren't required. that makes a mess in vs
vs c++ editor
its also annoying the configs and how things like escaping and so is done (Or just not using strings)
That messes up basic syntax highlighting
https://i.imgur.com/QJhzpBT.png then again this is probably not used anywhere
Love me them random SQS legacy things
have we discussed this before? the scripts= line is a typo, and the mods.cpp isn't used.
well, more correctly, that mods.cpp
i mean it doesnt HAVE to be a typo
you could use a backward slash like that just fine in configs, and if yo except that at the end of a config getText or something it would 100% be fine
it's a typo. cfgConvert throws it's hands in the air. But I think we are actually saying same thing.
ignore me.
btw, mods.cpp must be encoded as winansi
will announce now, SQF-VM is going to have "MultiPlayer" support with the next official (1.4) release
https://github.com/SQFvm/vm/tree/networking/src
currently i target roughly end of next month
what exactly do you want to support ?
networking with the vm (Net API),
networking between game and vw,
or multiplayer simulation?
In the first case, you should have a look at NetworkingTS :)
Remoteexec @meager falcon
As example
Pretty much just Server and Client
Communicating with the vm is already possible
Using debugging api
Alright, sounds great. I guess I should really take the time and have alook into the VM :)
Anyways, having a look at NetworkingTS is really worth it
has any of you ever managed to use an unbinned pbo that was more than 2gb in size? it seems that regardless of tool used when the pbo exceeds 2GB it becomes unreadable by the game and i can't even open it, it just says truncated pbo
32bit Version probably
any ideas?
Max pbo size is 2 GiB. No idea what you mean with "unbinned" .. Pbo's are not binarized..
Some guy at BI had the georgeous idea to make the 32bit size of a pbo a SIGNED 32bit number.. Genius!
2 147 483 647 Byte is max
by unbinned i mean without binarizing the models and such
to use with unpacked data and reloading models on the fly
YES! FUFFING SIGNED
well that's very bad news
(β―Β°β‘Β°οΌβ―οΈ΅ β»ββ»
π€¦ π π π π π π
Funny enough that this would be fixable so bloody simple
Same with their Array types. Hey, we have an array with a size variable. What datatype should we choose? Mh.. How about "int"? Int means number right? and we need a number! That fits perfectly!
π
I even created a feedback tracker bugreport about that one
Guess that is one of those ancient "nah... We will never exceed even a GByte" things
more like: a negative number is esier to detect than a underflow Β―_(γ)_/Β―
@meager falcon https://discord.gg/vpdwJtG you can test sqfvm in here Btw
Hey! A pbo with negative size! That see..... Mh... π€
Classes in binarized configs are stored as a number that denotes offset from start of file.. What if I make that negative π€
Or I think PBO files also have their offset. And as we know that offset is signed int π€
Also @meager falcon no... Just no
Not in actual data structures
There is no underflow btw in max size variables as they never change
If you manage to create a dynamic array container, where you can delete more elements than it contains, turning the size negative.
Then you deserve a darwin award
I pointed that towards array sizes and so on..
and well.. if its something more complex.. with all the things we have seen from some BIS people... not to hate but Β―_(γ)_/Β―
not from being renamed, but any reference to the pbo and it's contents will be preserved by the pboprefix if it is renamed.
The opposite.
putting in a pbo prefix will allow your pbo's to be renamed
As if you don't have one, the prefix is generated out of the filename
this is what I said.
The opposite of what he said/asked
Some progress report on my Armake to C++ conversion.
Finally also got some performance numbers to report
TFAR handhelds build takes 3,2 seconds with armake and the integrated binarizer for models.
Only 1,8 seconds with armake++.
Most influential changes are:
- using a proper hashmap for the preprocessor macros (instead of a linked list, that's linearly traversed everytime a macro lookup is made),
- getting rid of TONS of strlen calls by just storing the string length in std::string,
- the config parser using c++ types and move semantics (getting rid of many mallocs),
- multithreaded macro resolution in the preprocessor (that one actually didn't work out as I hoped. Makes it only about 30% faster with 8 threads)
A test with the TFAR handheld config (the one with thousands of radio items)
armake
preprocess: 915ms
parse: 800ms
binarize: 30ms
armake++
preprocess: 216ms
parse: 32ms
binarize: 73ms
I clearly fuffed something up on the binarize step tho.
Things that I have to fix now:
Armake spends 220ms copying the whole addon folder to a temp folder on C drive (I'm sure map makers with SSD's as OS drives will love that)
Armake spends 270ms iterating all files in that tempfolder to find their filesizes (it already iterated once to binarize all files that need to be binarized, it can collect the filesizes in there)
Btw it also opens and closes the file handle to the pbo for EVERY header that it writes.
Armake spends another 376ms iterating all files in that tempfolder again to write the files to PBO. (instead of iterating, it could just keep a list of files in memory after the first iteration it does)
Okey I replaced the second iteration through all files, by just creating a list of all files and their sizes in the binarize step.
Total runtime is 1,4s now
Another run with armake original for comparison: 2,9s
So about twice as fast. And I still have to get rid of the copying and the second file iteration. which will yield about 500ms again.
π
syntax error, unexpected T_NAME, expecting T_SEMICOLON in line 3027.
class Mission1
{
missionName = "A3\Missions_F_Bootcamp\Campaign\missions\BOOT_m01.VR\";
difficulty = "Regular";
};
π€
\" !
π€¦
Wanted to compare speed when binarizing a AIO config.
Didn't we talk about that \" in config last week or so in this channel? I remember seeing that recently
I mentioned it in regards to syntax highlighting. Like scripts path entry ends with that too
Still the case in dayz, where it is a valid entry for arma, but basically any tool will not like it
backspace? π
All arma parsers that think " is a escaped quote need to be slapped with a large trout
Does that mean they treat \" as an escape sequence instead of "" or they treat them both as those sequences?
Also I beg to differ: fusselwurm's and mine work fine with escaped strings π
sqfvm also works perfectly fine on em
besides, for the AllInOne config, it spends 2.588s parsing it and 1.821s for preprocessing (translating it into a proper config node tree also consumes some time π€· but that is fairly ignorable)
Okey now to the big showdown! Who can rapify a 1.84 AIO config the fastest!
Armake: Hey. I have a comma here! And I have a comma token that would match perfectly! Let's check if this is not maybe a 40kb long unquoted string instead! Oh crap. My buffer is only 16kb big. CRASH
Well. That was a fail.
Mikero: In File AIO_A3_1.84.144923.cpp: Line 5822 rap: missing inheritence class(es)
Well then... So much to that.. Maybe Bohemia can rescue us?
CfgConvert: File AIO_A3_1.84.144923.cpp, line 5820: /CfgAmmo/ammo_Missile_CruiseBase/Components/SensorsManagerComponent/Components.DataLinkSensorComponent: Undefined base class 'SensorTemplateDataLink'
Ahhh cmon dude...
Armake++: 8.6 seconds!
Congratulations armake++! You are the fastest!
armake works because it resolves parents after parsing the config, not while parsing the config. Because of that it can see the base class that is defined lower down.
I can't find a single non-broken AIO config.
I guess the ones that I made as a direct engine export last year should work. But they are probably on my dead HDD.
Well no AIO performance benchmarks then until I implement AIO export into my Intercept config library thing
Heh, nice
My preview image generator had to resolve in game class parents order to generate a valid cpp for mikero
Is the class tree available from the armake++ lib?
Yes. But the classtree is ugly AF. I will rewrite that.
@scenic canopy sqfvm
again
wait a second ... armake++ needs 8.6 seconds for that?
how long is that without rap?
new run is 7.7s
preproc 2.047s
parse 2.789s
rapify 1.786s
https://s.sqf.ovh/amplxe-gui_2018-11-14_22-41-38.png
The 177ms after the rapify is just destructors running. Cleaning up the class tree.
So 7.7s for TFAR pack? What was the old?
That is for the aio config
Ooh.
funny how sqfvm is somewhat equally fast π€·
though ... i am way faster done with my preproc
uhm...
and my parse
if i do not include the navigate (which is essentially the rapify but to something different)
maybe i really should include just a "gimmeapboyoubitch" mode
TFAR handhelds pack is 0.9 seconds.
https://s.sqf.ovh/amplxe-gui_2018-11-14_22-47-24.png
I love profiler graphs
About 300ms (1/3rd of the total time) is actually spent uselessly copying files into a different folder and deleting them again π€¦
Why are you so lame X39. I need giant colorful graphs
gotcha a speedy gonzales config parser instead
sorry π€·
I need my speedy colors X39.
though ... useless unless dedmen does this
needs to be done on the same system π
AMD or Intel?
AMD FX 8350
Lol. Isn't Ded a 8350
That means my code is probably better if it runs just as fast as yours! HAH!
it actually does not π
Damn...we have a head to head test now?
spending 1,885 seconds to preprocess
1.263 to actually parse the whole file
and then 1,137 to get it into my sqfvm data structure
throws stones
π€ need to do it both ways
the bloody documentation is horrible though ..
"documentation"
its easier to read code and replicate it. than reading docs
https://community.bistudio.com/wiki/PBO_File_Format#PBO_Header_Entry whoever wrote this deserves ... more orless nothing
mikero probably
https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format this is how one does this
starting with the offsets properly π€·
It has all the important info tho.. I think
ye
still
i need to fiddle around myself to write a parser
which is bad
reading JPEGs was kinda trivial task though ... challenging
i mean .... do not get me wrong
it is quite handy to have this stuff pretty much Copy & Paste ready
but fuck sake i still needed to open my old hex editor i have somewhere on my HDD (need a recommendation for a better one btw... using NEXT-Soft Hex Editor MX)
HxD is what I use. It can also open Ram memory. But for that I usually use Cheat Engine
for example: got the tfar pbo here
what the hell are those first few bytes already?
00 73 72 65 56
.sreV
why is this followed by 16 bytes 0
why is there explicitly written prefix.task_force_radio.Mikero.DePbo.dll ...
that marks it as arma/resistance/elite, like mentioned on the wiki page
see the structs
why is there explicitly written prefix.task_force_radio.Mikero.DePbo.dll
key:val
there is no mention of any of that in https://community.bistudio.com/wiki/PBO_File_Format#PBO_Header_Entry
nothing explaining why the first byte is 00
the second 73 and so on
this tells me, the first thing in there should be a file name
yes there is
then i am obviously blind so enlighten me
ha ... haha ... FUCK π€¦ ye ... forgot about that one
could puke already
but what is filename telling me?
because ... it seems to be always terminated immediate
now, what is not clear to me: why is the DataSize also 0?
and ... what tells me if there IS a header extension or not?
filename is telling you the filename for the entries
Just an anomaly in the initial
If there is >0 len strings in the header extension, they'll be present, if not, you'll run into just \0, an empty string
an empty asciiz string marks the end of it, so you'll be expecting to read simply that, in the case there is none
so after the header part, 00 instant is expected if there is no extension?
if there is no strings in the extension, it'll simply be 00 yes
You read until an empty string, which 00 alone would present
that is already not matching up with what i have in front of me
00 | 73 72 65 56 | 00 00 00 00 | 00 00 00 00 | 00 00 00 00 | 00 00 00 00 | 00
filename | method | orig. size | reserved | timestamp | datasize | extension
really hard to align here π
00 -> filename
73 72 65 56 -> method
00 00 00 00 -> orig.size
00 00 00 00 -> crap
00 00 00 00 -> stamp
00 00 00 00 -> datasize
70 72 65 66 69 78 00 -> ProductEntry```
this is what i have
there is no extension in there
Can you DM me the file you're looking at?
I'm not sure where your second product entry is coming from, it also doesn't match up
enough for now ... in case somebody wants to see where i am now: https://github.com/SQFvm/vm/tree/rapify/src/rapify
currently got trouble finding some neat "write" method
can't find a single non-broken AIO config
this is as its a straight dump from engine how it read it
and the engine is not as strict and accepts some bad config from BI/A3
eliteness crashes on me when I select some LOD's for some models
it does so for models I have looked at in the past, so I'm not sure if there were changes to those models or if there is an issue with the tool
@slow epoch best PM the issue and model to mikero directly
@dawn palm
well every model has at least one LOD that consistently crashes
eliteness GUI has stopped working is what it says
I am certain it didn't do that before
Wiki should probably say that 0x56657273 == Vers/sreV.
packingmethod == "Vers" and datasize==0 and timestamp == 0 is a "magic" value for "this is a pbo".
You can write whatever into orig size and "reserved". 8 bytes of fun
Also filename has to be 0 length.
Essentially the pbo header. After that there are the key:value pairs for the properties (prefix or mikero or whatever really) end of key:value pairs is a 00 byte after them, where a new key name would usually start.
Btw the headers "Reserved" value is internally used by the engine to store the file offset from start of pbo. It ignores the value when reading, so you can write whatever you want into there :D
Steganography ftw.
@dawn palm do your Tools implement exec and eval?
See #arma3_config
can you run DZ tools and A3 tools at the same time?
heard it can? happen DZ overwrites existing A3 tools (registry stuff?)
New error messages for my Armake c++ port
Before: error: Unknown array element type 95.
After:
error: Exception occured in Config::fromBinarized: Unknown array element type 95
Trace: /CfgMagazines/PylonMissile_1Rnd_BombCluster_02_cap_F/hardpoints
File Offset: 272639
Should be especially useful for those of you writing your own binarizers and wanting to test them.
I should add at which element in the array the error happened too I guess. Although the File offset plus a hex editor will tell you that anyway.
Currently I only have that in Derapify, I'll add it to the rest too. Like pbo parsing and stuff.
100x speedup in deleting the tempfolder by replacing
// MASSIVE @todo
char cmd[512];
sprintf(cmd, "rmdir %s /s /q", folder);
if (system(cmd))
return -1;
with
std::filesystem::remove_all(folder);
π€¦
// MASSIVE @todo, so pointless comparison...
The main factor is here that it has been a MASSIVE @todo for 3 years. While it only took literally 2 seconds to replace it with appropriate C++ code.
I guess in C it would've been too much work? Although windows should have a recursive folder delete function I suppose
It's not as trivial, no
Windows has DeleteDirectory() I believe
Remove you mean?
https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-removedirectorya This path must specify an empty directory
Pre C++17, it was still not simple in C++ either
Linux: https://stackoverflow.com/a/5467770 also lots of work
This is the winapi soln to non empty dirs: https://docs.microsoft.com/en-gb/windows/desktop/api/shellapi/nf-shellapi-shfileoperationa
Unless you want to use COM
(see IFileOp in link)
It's a shame to stick a high std. requirement on it though
Here is a comparison of before and after
https://s.sqf.ovh/amplxe-gui_2018-11-17_17-49-06.png
https://s.sqf.ovh/amplxe-gui_2018-11-17_17-49-12.png
I want to remove the whole tempfolder thing completely. Because copying the whole modfolder to C drive tempfolder on every build is not a nice thing to do to SSD's.
Basic pbo reader implementation done. Next task: Letting Armake read external files directly from PBO's. Instead of having anything like a P drive.
Need Arma header files? Easy -ip "C:\Steam\SteamApps\Common\Arma 3\Addons"
Need CBA? Of course! Here ya go! -ip "C:\Steam\SteamApps\Common\Arma 3\@CBA_A3\addons" Done!
What? You want it even easier? Sure! -armaPath="C:\Steam\SteamApps\Common\Arma 3" -mod=@cba_a3,@ace3
I don't know if I can trick BI Binarize with that as armake still uses it. But worst worst case, User creates a empty p-drive, and armake automatically extracts only the needed files to there
why not use Dokan for this?
Dokan is not any better than standard P drive on CI systems
not faster? saves HDD/SSD use?
You still need all files somewhere, and then virtually mount them
Why not read them directly?
I for sure am not bothering with substituting virtual drives on CI machines
well for our CI server HDD/SSD use matters
as you need steam running, it downloads the game and mods after each update. so gotta delete those pbos each time
also for non CI use no longer needing to extract would make things easier for the average guy
Because dokan is a extra tool that you need.
Dokan also needs kernel driver
Which makes most public CI systems unusable
It would also need a separate FUSE module for *nix
dayz2P is now part of my FREE toolset
what makes more sense?
-i --input for files or -f --file for files?
--input-file <mindblown> π
But more seriously, I'd go for input (although that also kinda depends on the context)
sqfvm
want to get away from separate input stuff for sqf files and config files as pbo files might get added soon
thus only the extension then telling what file it is
(though something like --input-sqf, --input-config, --input-pbo still will exist)
and did so
think the changes to the CLI "interface" will be much appreciated π
input-mod ? Where you give it a @mod folder like in arma?
π€
could be possible
but for starters, the actual --input-pbo should work π€·
only the CLI logici s implmented yet π
hi , i have a problem with pbo project, when packing, with full build, the log says, command rapify not found, even though the cmd clearly states, that rapify has been found
is that a known issue?
i did. and yes, they are in different folders
didnt help
you cant change the installation path in the installer
nevermind they are in the same folder
they are in different folders? By default they should all be in same afaik
yeah
And I meant PATH. The environment variable. Not path as in filepath
thanks @glossy inlet a reboot fixed the issue
People who write pbo packers are always too serious about it π
Does anyone have anything better that fits into 20 characters?
What if it stops being ignored in the future? Won't that make those PBOs invalid?
It has been ignored since OFP
Sooo..
And there is absolutely no reason to stop ignoring it. Unless BI wants to allow having null named files in pbos
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 - my suggestion, yes, I'm a party pooper π
My thing doesn't work anyway π€¦ little endian. Need to swap all characters around π
DeySeeMePackinDeyHatin is too long π¦
DeyCMePackinDeyHatin
It's painful to look at, though π
I also have 4 bytes per file entry to fill with random stuff. Could write essays into there :D
But noone will read that anyway
What about "obey Cthulhu"
Lorem ipsum...
IMO that's not a good idea, anyway. Can't really talk much right now, but I was writing a repacking PBO script for my torrent updater and I can see having those magic strings in random places possibly messing up my script
Then your parser doesn't obey the "standard"
If you parse reserved values that should be ignored and check their valus. Then your parser is wrong
Not "crashing", FYI. Rather probably making me download more than needed
Well the bytes are always there. Whether they are all 0 or set to other stuff
Putting whole big texts into there would mess with compression algorithms yeah. But that one is more meant as a joke. As a optional steganography function.
And the 20 bytes at the end of header don't really matter compression wise
Sorry, can't go into detail right now as I'm in the middle of something at work
meh boring
But I'm done with the implementation of my new PboWriter.
https://github.com/dedmen/armake/commit/8c050e268ba6f77386a3e6fa066a2f8c70a1b2fb#diff-3564370d57a2438579ffb914ac39978eL381
It's way more compact now and easier to use as a library.
Also armake now doesn't traverse the mod directory 3 times anymore. And also doesn't constantly open, seek, write, close the pbo when writing a file entry.
And it also doesn't read the whole pbo back from disk again to create the checksum
@inland flare is there documentation about what changed with v3 bisigns?
For all the #arma3_tools who have their own bisign creation tools?
Is it possible to use armake as a library easily?
Not yet. But when i have converted everything it should be yeah
Well library aka include the source into your project. Not dll-like library
C-style include yes
I'd consider making python bindings if the library code would work fine
use case?
Before everything was hardcoded directly into the command functions in exactly the configuration that was needed for exactly the purpose that armake had.
I'm moving things out to seperate, more configurable classes now. That's about it
use case?
Access PBO files' contents from within python isn't enough? π
With some code that's fast (written in C++) and independently used and improved by other people
As opposed to my own python lib written by myself
I'll make some benchmarks between old and new when I'm home. But it should be quite the difference.
For a pbo with 100 files in it.
Old armake would open/close the output pbo file about 206 times.
And also read back the whole file after writing it to create the hash (which I am now doing inline while writing to output file with a custom streambuffer)
Also: if you're building stuff from make.py (and you are). Why even call an external armake executable if you can execute the same code from within make.py?
we aren't with armake, a Makefile and done
Also installing the latest armake: pip install armake and you're done.
I can see only upsides tbh π
yay -S armake π
Have fun on windows ;P
I don't want to π
we already have a script that grabs the latest executable as well
Me neither and that doesn't make me windows-free
but it makes you Windows-free'er xD
Also: something that I actually need: stuff working on my cheap 2.99 Eur/month ARM machines. The fact that I can type pip install armake anywhere with the same outcome will make it MUCH easier for me to actually use that binary
having cross compiled binaries is not enough?
with python someone will always complain about 2 vs 3 π
It's November 2018. They won't
Python 2 will die in 1 year and 1 month. It would be foolish to complain at this point.
Also, if you create bindings for either version, making additional wrappers for the other should be a piece of cake
if you're bored, just read up on calibre for python 3
No there is only a year and half left before python 2 is freed from the shackles of its current maintainers.
π
I just read the same statement.
Anyway, that's a moot point. There will always be someone stuck with Cobol because it will be impossible for them to port some part of their code to other languages.
In the same way, there will always be some bitter programmer that will stick with Python 2 and we can't do anything about it, besides telling them that 99% of the world made the switch (in a year and a half).
Seriously, all those people had 9 years to port the software. At this point I'm not even sorry for them
my favorite part of calibre is
My current plan is to simply support python 2 myself
yeah π
Yes, I also have my own application written over the years in Python 2 and I don't plan on porting it to 3. I will just let it die π
http://py3readiness.org/ - just saying...
I'm not against python 3 but I know a lot of grumpy python devs π
the trick is to not use python
problem solved forever π
I heard python is venomous (me knows the difference between venom and poison.)
best solution: use SQF
nothing is as stable as SQF
just stuff gets added, nothing ever will get changed
Remember when they changed string escaping not hugely long ago, after it was broken for an eon :D ?
mhh?
life servers remember @fallen stone
you mean the final fix of str?
again then here: not changed but rather finally fixed
but I mean practically it was still a change
no
it was a fix @native kiln π€·
wouldn't cha say that's still a change
if one fixes an operation, it is a fix, not a change
But by definition, fixing something is making a change, yeah π ?
unless you just tell people that the bug is a feature. Then you've fixed it without making a change in the code
Are you a lawyer?
currently trying to create some XML for some database extension i create. ..
<connection name="connection 1">
<uri>tcp://127.0.0.1:3306</uri>
<database>database</database>
<username>user</username>
<password>pass</password>
</connection>```
or rather
```xml
<connection name="connection 1" database="database" username="user" password="pass">tcp://127.0.0.1:3306</connection>```
for context, a statement is set up in the config currently like this:
<statement name="statement A">SELECT * FROM table</statement>
<statement name="statement A"><![CDATA[INSERT INTO table VALUES (?, ?)]]></statement>```
π€ looks like syntax highlighting lacks CDATA sections
One too many ] perhaps?
indeed, but still no change π€·
π¦
ye .. but ... nah
XML looks more verbose for theese things
it is purely a "ohh wow this feels so advanced" thingy here π
ohh ... and in JSON i would need to use arrays π€·
xml is the most utter sheeee if you dont have to use it
though ... gimme a minute
used to use my own format
but need to reimplement to get it clean and ... nah
so ... a chance to work with RapidXML π€·
I don't know who would say "oh wow this feels so advanced" about a very inefficient 20 year old format
what about .ini ?
what about pure binary? you just edit it with a hex editor
ini is kinda broken
depending on the parser, you either can do it with inis or not
if you need something human readable (sort of) stick with JSON: less overhead, faster&easier to parse (+ nice parsers in any language)
{
"name": "connection 2",
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
}
vs
<connection name="connection 1">
<uri>tcp://127.0.0.1:3306</uri>
<database>database</database>
<username>user</username>
<password>pass</password>
</connection>
personally, XML looks more clear to me
but that also could be because i am now used to reading XAML π€·
also, i like the idea of not having the name inside of the actual object
some feature i highly miss from JSON :/
JSON looks nicer
what i mean gets more clear on statements
if you dont want the name inside, wrap it in another "layer"
{
"connections":
[
{
"name": "connection 1",
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
},
{
"name": "connection 2",
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
}
],
"statements":
[
{
"name": "statement A",
"query": "SELECT * FROM table"
}
]
}```
You don't have to have the name outside, more objects π
besides that "statements" block, there is no relation
{
"connection 2": {...}
}
that is horrible dude
though ... would solve the "unique" problem π€
why not:
{
"connections":
{
"connection_1": {
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
},
"connection_2": {
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
}
},
"statements":
[
{
"name": "statement A",
"query": "SELECT * FROM table"
}
]
}
and iterate keys ( if thats you are after)
{
"connections": {
"connection 1": {
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
},
"connection 2": {
"uri": "tcp://127.0.0.1:3306",
"database": "database",
"username": "user",
"password": "pass"
}
},
"statements": {
"statement A": "SELECT * FROM table",
"statement B": "INSERT INTO table VALUES (?, ?)"
}
}```
exactly
now need to check what a comment looked like in json
skipped statements
// and /* */
but they are not allowed in some standards I think
default json doesnt have comments yes
so ... no JSON again π
even attempted to use YAML
bot the bloody API drove me nuts ...
configuration:
allowNonPrepared: true
connections:
- connection1:
uri: tcp://127.0.0.1:3306
database: database
username: user
password: pass
- connection2:
uri: tcp://127.0.0.1:3307
database: database
username: user
password: pass
prepared_statements:
- prep_statement1: SELECT * FROM table
- prep_statement2: INSERT INTO table VALUES (?, ?)
could not read shit
il like how linux config files look
maybe just use SQF arrays
[
["KEY", []]
]```
best solution IMO
and global vars for config
bwahahha
lets make this an industry std π
SQF-VM already is existing
soooo ... π€· not much is needed to spread your fav scripting language π
get it popular
runs 10% sheeetier than JS boi! its lit, use it!
create games with it
Why not
[[], "KEY"]
It's obviously better to have the order reversed :]
π€
specialized commands for configs:
[...] asConnection "connection 1"```
this actually would be damn nice Oo
for SQF people at least
or rather [...] as [connection, "connection 1"]
think i will stick with XML for now just to get something done finally ... thinking about this for about an hour now and still got no progress π
@hardy leaf ofc it is.. however its not really good for processing
JSON and xml are a thing because of OOP and the ease of categorizing the properties
Okey I got benchmarks about my armake pbo writer remake.
Pure armake current master branch: 699.31ms 618.64ms
armake++: 171.9ms 174.17ms
Old armake spent about 60% of the time closing and re-opening the same file.
And 32% of the time to read the whole pbo back from the file it had just written to hash it and make the checksum. Which armake++ does inline while writing
What is the best license to publish my pbo stuff under? I don't want the problems of GPLv2 vs v3 where you can't link or copy-the source from one into the other.
But I want to enforce that the code stays open-source, unlike MIT.
Well.. I could publish it twice. GPLv2 and GPLv3 π
"GPLv2 or later"
Stack (Overflo): "GPLv2 or later"
But then you're at the mercy of Stallman, who could create a GPLv4 and claim that all software under that license is his!
So you have to believe him that he won't do that, in the same way as he wants you to believe companies and corporations who say that they won't enforce some part of their closed license. Oh wait... he doesn't! π
Okey Recently I recommended HxD as hex editor.
I'd like to correct my recommendation and recommend Hexinator insteead.
https://s.sqf.ovh/Hexinator_2018-11-24_20-35-30.png
This is just awesome
HexEd.it is a free hex editor for Windows, MacOS, Linux and all other modern operating systems, which uses HTML5 and JavaScript (JS) technology to enable hexediting online, directly in your browser.
that's what I usually use
HxD has a beta?! β¬ββ¬ γ( γ-γγ)
But you usually don't upload whole pbos to website, eh? π
Oh wait, it doesn't need to upload it? 
it runs locally π
Cheat engine, 3 hex editors, IDA and visual studio open.. Trying to debug mikeros tools to find out what a error message means and where my mistake is π€¦ π
Something somewhere having something to do with lod's is 5 bytes too big π
And it somehow only happens when I try to pack with a SuperExt shader material with a Stage8 in it.
I don't want to spoil your fun but... you can always ask him directly about an error, you know? π
Why waste his time when I can figure it out myself :D
HAH! Found it!
Atleast I found the offset in the file where the wrong number is. Now I have to find out what it actually is.. Shouldn't be too hard
vertex table offset π€ should be 0x1146 but is 0x114B :U Okey. Great.
π€¦ not my fault Β―_(γ)_/Β―
BOOM!
Hopefully I can now finally pack my SuperExt shader with armake without crashing the game
Noope. Still crashes. But atleast Eliteness doesn't fail reading the p3d anymore
Adding Stage 8 in the material crashes Arma when placing the model. No matter what pixelShader I use :u
Wow. Arma crashes because something with StageTI is wrong. But it only happens when I add Stage8 π€
Bam! Bug found! There are only 8 TexGen's. But 15 Stages.
Armake always prints 1 TexGen per Stage. Arma itself shouldn't even read more than it can have. So I guess I found a buffer overflow bug π
StageTI was wrong because that's right after the TexGen's and the TexGen write overflowed into the TI stage
Btw mikero. Eliteness didn't detect the 9th TexGen as an error when loading the odol. Though it's not something that usually happens really.
https://s.sqf.ovh/arma3_x64_2018-11-25_04-47-39.png Well. I got what I wanted. only like.. 6 hours of testing and debugging.
And armake++ has a dozen new todo's about everything that's wrong with rvmat's.
dedmen's excellent information on texgen stages in a p3d has been incorporated into my tools (including the 8 count limit, and a 'better' decode of baked in rvmats )
No wasted time there dang π nice work π
π€ Why cannot I add a plus one
Not cool enough sorry
Cuz you english first have
Can someone explain why not returning anything is not a compile error?
My "loadLod" function returns a success bool. I forgot the return true at the end. So like.. Every hundredth build randomly fails.
This has been haunting me for days now
Not very good answers. But it is completely possible exit functions without returning, e.g. using longjmp or exceptions (not sure if you can goto out of a function). So making it an error might violate (poor) programs.. In C++ it is UB but only if it actually happens and in plain C it is only UB if you actually attempt to use the return value at the call site. Some other mechanism like a global variable or pointer might indicate the return value is not to be used... So any particularly attractive reason to allow it, except for backwards compatibility? No...
So apparently the flag for warning is -Wreturn-type, automatically enabled by -Wall.. You can turn it into an error (reading from the docs here, not tried it), by adding -Werror=return-type
That is for GCC
Okey armake++ is now intelligent enough to compress uvTransforms down.
Even if you don't use texGen's, armake++ will generate them internally, and combine uvTransform/uvSource that are the same.
So even my 8 stages SuperExt that previously either ignored the Stage8 or crashed the game because it previously tried to create a texGen per stage. Now builds down to just 2 texGen's automagically.
Armake does the same if you actually use texGen's. So if you have your own texGen's, but two or more texGen's actually have equal values, armake will throw all but one of them away and save you a precious 52 bytes per "useless" texGen
Also means you are not forced to use texGen's for multimat's if you are building with armake. As long as you don't have more than 8 distinct uvSource/uvTransform combinations.
If you do it'll throw you an error
Does anyone have any experience with http://marma.io/ as a Server-performance-monitoring tool?
I tried it back when it just came up. Was a chore getting to work on linux and after it broke the 3rd time I just gave up
When did it come up?
little over a year or two back
Okay so maybe those problems are fixed now. Apparently the developer claims Linux-compatibility ^^
Well it was linux compatible back then.
Just the same usual chores as with any linux mod basically. Same with extDB too.
Could've gotten way easier now though.
Also the need for a server monitoring tool like ASM dropped by a lot. So I didn't feel like trying it out again
Alright - I guess I'll give it a try and report back if someone's interested
On linux or windows? If you do linux I'll also be very intersted at how the setup went along
problems with linux extensions? mhh?
I should add those statistics to my server manager
I had a web ui setup using ASM before 
Linux @glossy inlet
this is how you use that lil class https://pastebin.com/UA4av4r4
will be the base of sqf-vms networking stuff
Blender 2.8 Beta is finally out. Now the wait for blender addons to be updated.
could use some help in drafting a networking class for server-client communication ..
noticed that i really have no idea how to realize it π± (idea in the sense of i simply do not have any idea how to design it nice and sweet without me later rewriting the whole stuff 20 times)
goal is to have: a netserver class that hosts an actual server and a netclient class used to connect to said server
both need to support async adding and removing of messages
a message should get written and read with a streamable (https://github.com/SQFvm/vm/blob/networking/src/networking/streamable.h)
also, different message "packages" need to be differentiable
(in regards of network protocol: max package ID is 254 as 255 would take the next byte to again have 254 possible packages)
ideas are appreciated
π€ just realizing that my streamable class is not compatible with sockets
soooo .... i need some network library that wraps sockets into iostream
and is not in any way connected with boost as i do not want to download close to 500mb of a fucking monster just for a single component and slowing down my compilation by a magnitude + increasing the output to a level of pure insanity
have you looked into boost asio π
You don't have to download the entire of boost
last time i did, i had to download more and more components of boost up until the point where i would have had to include the entirety of boost ...
at which point i just skipped over this and implemented my own OS independent socket wrapper in C
I mean, regardless, I don't see why you wouldn't have a copy of boost anyway
I don't think asio carries many deps, esp. after ++11
I'll see
π
Seems beast is fully fledged now too
Yup...
in the process of making an extension for arma, c++ using KKids callExtension.exe, using just the examples, whilst i get it to work, the _64 version is working, yet the x32 and i compiled both versions, any ideas ?
what?
what's the filename of the 32bit one?
That can't be correct. In your screenshot it's Arma3Fetch
ok now its giving a different error, method is not implemented yet the _x64 is still working
correct exported name?
its litteraly the example from the wiki whilst im trying to get atleast the build right.
#include "stdafx.h"
#include <string>
#include <cstring>
#include <sstream>
extern "C"
{
__declspec(dllexport) void RVExtension(char *output, int outputSize, const char *function);
__declspec(dllexport) int RVExtensionArgs(char *output, int outputSize, const char *function, const char **argv, int argc);
__declspec(dllexport) void RVExtensionVersion(char *output, int outputSize);
}
void RVExtension(char *output, int outputSize, const char *function)
{
std::strncpy(output, function, outputSize - 1);
}
int RVExtensionArgs(char *output, int outputSize, const char *function, const char **argv, int argc)
{
std::stringstream sstream;
for (int i = 0; i < argc; i++)
{
sstream << argv[i];
}
std::strncpy(output, sstream.str().c_str(), outputSize - 1);
return 0;
}
void RVExtensionVersion(char *output, int outputSize)
{
std::strncpy(output, "Test-Extension v1.0", outputSize - 1);
}
``` this one
then check the created DLL to make sure that the correct names are exported (and correctly decorated)
how would i go about doing that ?
Google: "Dependency Walker"
It also tells you if you have some dependencies to external libraries which you might now want
Ye
they are both the same.
i tried that and it didn't work earlier.
#include "stdafx.h"
#include <string>
#include <cstring>
#include <sstream>
extern "C"
{
#ifdef WIN64
#define _RVEXTENSION RVExtension
#define _RVEXTENSIONARGS RVExtensionArgs
#define _RVEXTENSIONVERSION RVExtensionVersion
#else
#define _RVEXTENSION _RVExtension@12
#define _RVEXTENSIONARGS _RVExtensionArgs@20
#define _RVEXTENSIONVERSION _RVExtensionVersion@8
#endif
__declspec(dllexport) void _RVEXTENSION(char *output, int outputSize, const char *function);
__declspec(dllexport) int _RVEXTENSIONARGS(char *output, int outputSize, const char *function, const char **argv, int argc);
__declspec(dllexport) void _RVEXTENSIONVERSION(char *output, int outputSize);
}
void _RVEXTENSION(char *output, int outputSize, const char *function)
{
std::strncpy(output, function, outputSize - 1);
}
int _RVEXTENSIONARGS(char *output, int outputSize, const char *function, const char **argv, int argc)
{
std::stringstream sstream;
for (int i = 0; i < argc; i++)
{
sstream << argv[i];
}
std::strncpy(output, sstream.str().c_str(), outputSize - 1);
return 0;
}
void _RVEXTENSIONVERSION(char *output, int outputSize)
{
std::strncpy(output, "Test-Extension v1.0", outputSize - 1);
}
``` something like that ?
yeah. I don't think that'll work...
how would i go about changing the exports based on arc ?
What jumps into my eye though is how you are missing __stdcall. Why did you remove that?
how was the x64 version working without it
Because the names for x64 don't have that suffix and don't need it
just put __stdcall there like every tutorial tells you to?
im getting declaration is incompatible
extern "C"
{
__declspec(dllexport) void _stdcall RVExtension(char *output, int outputSize, const char *function);
__declspec(dllexport) int _stdcall RVExtensionArgs(char *output, int outputSize, const char *function, const char **argv, int argc);
__declspec(dllexport) void _stdcall RVExtensionVersion(char *output, int outputSize);
}
"redefinition"
yeah.
don't define it twice with different types
Look at all the messages at the top of your screenshot
could someone update the wiki with that ?
ok i completly broke it now
changing ```cpp
__declspec(dllexport) void _stdcall RVExtension(char *output, int outputSize, const char *function);
__declspec(dllexport) int _stdcall RVExtensionArgs(char *output, int outputSize, const char *function, const char **argv, int argc);
__declspec(dllexport) void _stdcall RVExtensionVersion(char *output, int outputSize);
```cpp
__declspec(dllexport) void _stdcall RVExtension(const char *output, int outputSize, const char *function);
__declspec(dllexport) int _stdcall RVExtensionArgs(const char *output, int outputSize, const char *function, const char **argv, int argc);
__declspec(dllexport) void _stdcall RVExtensionVersion(const char *output, int outputSize);
``` no errors in vstudio, but method not implemented
i think i got it
#include "stdafx.h"
#include <string>
#include <cstring>
#include <sstream>
extern "C"
{
__declspec(dllexport) void _stdcall RVExtension(char *output, int outputSize, const char *function);
__declspec(dllexport) int _stdcall RVExtensionArgs(char *output, int outputSize, const char *function, const char **argv, int argc);
__declspec(dllexport) void _stdcall RVExtensionVersion(char *output, int outputSize);
}
void _stdcall RVExtension(char *output, int outputSize, const char *function)
{
std::strncpy(output, function, outputSize - 1);
}
int _stdcall RVExtensionArgs(char *output, int outputSize, const char *function, const char **argv, int argc)
{
std::stringstream sstream;
for (int i = 0; i < argc; i++)
{
sstream << argv[i];
}
std::strncpy(output, sstream.str().c_str(), outputSize - 1);
return 0;
}
void _stdcall RVExtensionVersion(char *output, int outputSize)
{
std::strncpy(output, "Test-Extension v1.0", outputSize - 1);
}
thanks for the help @glossy inlet
update wiki with what?
the example on the wiki didn't work ...
@nocturne basin u goof
VS complains about redef with different type cuz __stdcall is missing in the definitions
https://community.bistudio.com/wiki/Extensions#Visual_C.2B.2B_3
was copy cat from old with changed inner code
@nocturne basin that i posted above works
yup
now i just gotta learn how to c++
because you added the stdcall stuff into the method declaration
you may also use sqfvm to test your extension btw.
it is a lil bit "richer" in scope then kks extension tester
i would have no idea how to use it
the same way as you would use kks tester
double click and throw extnesion in same dir
but for now, learning how to write an extension is the real challenge for ya π
yup
anyone got any pages for tips
i have someone who offered to write my extension for me, but want to try learn myself first
just the tip
@glossy inlet cannot change currently due to my mobile phone being lost ... somewhere (and thus i have no access to the auth app)
π
@nocturne basin does your uieditor use pixelGrid ?
i made a little tool in node.js to export photoshop positions base on pixelgrid defines
you can check using the config view
but i do not recommend using it at all simply because it is no longer maintained
POSITIONS(0,0,80,5)
``` this values are all from photoshop
Only about a hundred //#TODOs missing to get armake's p3d binarizer up to odol v73 and onto the same level as BI binarize. Down with BI Binarize!
not sure if this is the right channel, but is there anyway to resolve the horrid performance of Object Builder in W10?
Use directx viewport
Intercept broke with 1.86, fix here: https://github.com/intercept/intercept/releases only need to update host dll.
That is.. If anyone here is even using intercept π (I know you guys are out there! stop hidin!)
I was thinking about using it with brofiler this weekend. Does it work with new update? :P
Nothing actually seems to have broken.
The only thing that broke is a intercept pattern match that I wasn't actually using since 1.78, just forgot to comment that out π€¦
Atleast core and cba are running fine for me. Didn't test profiler, and won't today.
Brofiler needs separate dll doesn't it?
ye
uh
Latest intercept yeah. That's what I pushed in intercept releases.
Oh right. Yeah you need to not use the intercept.dll included in brofiler. Use the one from the intercept github
IIRC there used to be a way to look at the file size of a bisign file and identify it is was an old sig or a v2 sig file. Do we have the same for v3? is it documented somewhere?
no
no to both thoughts?
There was a diff between v1->v2
diff between v2 and v3 is different version number (in file header)
and instead of a file blacklist it uses a file whitelist for hashing now. The hash size didn't change though
Thanks
new tool in... 3... 2... π
Version number is on.. eh... Offset.. Right somewhere in the middle of the file π€¦
Frack!
https://s.sqf.ovh/HxD_2018-12-03_20-15-29.png
There. v2 π will be 3 for v3.
offset changes because the keyname is at the start
wow
offset from end might be constant
there are 4+128+4+128 bytes after that
0x108 from end. Yeah. Matches up with my screenshot
or +4 to the start of the 32bit int
armake seems to write more after the signature?
"more" ?
ah no, it was writing to two "files"
you're correct
two files? ^^
and that's wrong line I guess, must be the one with temp = 2
Yeah. See my PR.
for what? armake?
could add to armake as well I guess
to figure out v2 or v3
I was just gonna throw together a small golang cli thingy that read the version
Ah. yeah.. Easy of course, but not really needed. Honestly. Just make a javascript web thingy for it? ^^
I can make one Β―_(γ)_/Β―
time to fire up dsutils then I guess π
thanks
https://github.com/4d4a5852/a3lib.py/blob/master/a3lib.py returns the version, too
a3lib.py bisign <somebisignfile.bisign>
got it working in the browser, thanks for the help! π
Give it to us! We love sucking on these tasty v3 bisigns
using powershell and some regedit, one could also make it pop-up in a message box
Add-Type -AssemblyName "System.Windows.Forms"
[System.Windows.Forms.MessageBox]::Show("text", "title", 0)```
regedit won't work with *nix π
mhh?
talkiung about something like this: https://superuser.com/questions/700997/adding-powershell-command-to-right-click-menu-both-on-and-in-a-folder
ah, that would be nice
here's a cli tool in golang that can be compiled cross platform, binaries included, https://dahlgren.io/bisign/cli/
I'll put it on github once I'm done with my dinner π
@obsidian sluice how would i go about using the directx viewport?
oh my god thank you
ive been enduring shitty performance for like a year and a half now
how did i never notice that
has anyone looked into/tried cloud gaming offers for mod development yet?
this service https://shadow.tech/dede/discover/specs has quite decent hardware with win10 home and 256 GB space for 30$ (one year contract) or 35$ (3 months) - basically you rent an unlimited windows machine
@scenic canopy from what i recall you have some machine with GPU for your CI/build server, dont you?
Yep
Shadow is not meant for 24/7 or headless access
You could just use a spare PC as well
Can someone PM me the latest arma3_x64.exe? I'd like to investigate some issues and the #perf_prof_branch isn't on dropbox/google drive yet
@scenic canopy i am looking for a manual test environment (not CI/build server here). the flat rate of shadow seems most sensible for that if you do it full time - on the side dev probably pay per use is cheaper
how much hours do you have on average per month for your CI use?
@glossy inlet still actual?
@vague shard we pay $17 a month for 24/7 usage
another thing to know, almost no remote streaming software works with the arma editor
camera controls are broken
you mean the Eden editor or even the old 2d?
also what package do you have specifically for that $17 if you dont mind asking? G1 with some upgrades?
A legacy package not available anymore AFAIR
@vague shard Air with 100GB storage, legacy pricing. Air costs $17 excluding VATand storage now
@scenic canopy ok tx. what are you using it exactly for if you dont mind asking? just CI/build server/autotests or more?
if you can live with occasional down times, those prices are hard to beat: https://cloudatcost.com/
They don't seem to offer GPUs, which is the requirement in this case. For really low-end servers with unlimited transfer (not suitable for Arma hosting, of course) I personally recommend Scaleway at 4eur/month [no, they don't offer GPUs either]
@vague shard only arma integration so autotests, data extraction etc
rest of our CI runs on our own VM farm
@karmic niche I've had quite a few scaleway machines but the drives kept dying
except for VMs hosted on our dedicated servers we now mostly use https://www.hetzner.com/cloud
Cloud servers starting at β¬ 2.49. A little money gets you lots of cloud. Our flexible billing model and clever interface make it easy to use our cloud servers for all your IT needs.
works really good, they offer on server NVMe RAID or Ceph network storage
I've also had a few issues with servers dying around hmm... π€ about a year ago. But since then, no server has died on me.
I'm using them for our custom launcher with torrent mod hosting so the unmetered transfer is really a plus. Especially that they are billed per hour and really disposable machines. I guess that for CI stuff that can take some time that would be ideal.
I think that now that steam has released DXVK, I may revisit my idea of using xvfb to have a headless testing rig on linux, on cheap machines. Something that wasn't possible some time back, because of the DX11 requrement
I can say that our FPS with non fancy GPU isn't that great
so using arma with xvfb would probably be really bad
I guess it depends on what you wanted to test. My use case was primarily for crash-testing intercept on dev-branches so I don't care about performance at all π
Crash testing intercept? @karmic niche
whether you can run some intercept test code in a mission without the game crashing π
As if Intercept would be unstable. pffft
it seems smarter to me to do it once in arma π€·
Good thing that it's been working fine for the last... wait... π€
π
Why do it once if you can have a test that launches the newest arma-dev automatically everyday and calls a few (hundred) of intercept calls? (although dev branch releases will now be done less often)
the point is actually: why do you do it π€ unless you intend to fix that stuff you are out of luck anyways
Actually, I have already fixed intercept once, after an Arma release FYI. That was long ago, but I did. π
(fixed a few offsets that have changed in the internal structures)
But anyway, you'd then have enough time to poke Dedmen, before the dev version gets merged to the main Arma branch
We just had that happen with 1.86 release. We didn't have anyone test on dev-branch so no one noticed that it crashes
That was precisely the point of having those crash-tests
is there a way to fix this with mikeros tools https://i.imgur.com/SRP4Ia9.png ? Or should I just avoid it
Well someone has to fix that Β―_(γ)_/Β―
mikeros tools are theoretically able to fix it. But I don't think that you can make them do it
@scenic canopy release DokanPbo binary? π or is there something you are waiting for?
oh cool, @vague shard !
I should probably upload the binary to github as well and tag it
together with a matching dokany release
@scenic canopy Thanks much! DeP3d worked.
There shouldn't be any issues if the object is used?
probably not no
just mikeros checking that is now ok with