tooldev-general

[worthy cape]

(I run win11 on all my boxes, nothing overly bad has happened yet)

[rapid pagoda]

PC hardware manufacturers are, as a whole, spectacularly bad at software

[simple ravine]

I refuse to upgrade my i7 7700K just to run win11

[dusk coral]

I have zero RGB, and only msi afterburner, I need logitechs crap for having greater than 5 mouse buttons 😦

[simple ravine]

msft needs to figure their shit out

[worthy cape]

Shenzen I/O feels like a documentary video game.

[dusk coral]

they seem to alsways half commit, so it sucks for everyone, novice and expert. they managed to kill skype when it had massive market share

[simple ravine]

there are different camps at microsoft for sure

in the developer space as well

those who make .net runtime/sdk etc and azure stuff are mostly top-shelf people.. then you have the office 365 and low-code/no-code people, sharepoint crap... they're not top-shelf

[dusk coral]

I work at Intel and we seem to have the same weird flippy floppy half measure attitude. I wonder if all big tech is like this? I want to get out of manufacturing and get into ideas

[worthy cape]

I'm gonna blame kernkraft for the unrevertable firmware upgrade that made my Intel SSD perform worse 😄

[simple ravine]

then perhaps some of your colleagues might be able to tell me why my TPM 2.0-compatible CPU won't be able to have Windows 11

because Microsoft is pointing fingers to other manufacturers

[worthy cape]

@simple ravine I thought they were backpedalling hard on that?

[golden bane]

kernkraft pls make Optane cheaper. My NAS needs wants it 😄

[dusk coral]

Our CEO is from vmware, but we use the most horrible remote desktop setup, It crashes like once an hour, and I email IT, they tell me my laptop isnt good enough, the one they issued me! I dont have a choice

[simple ravine]

they are "investigating" if it's possible... they half-assed some excuse saying it's because "they want to ensure performance experience"

like my i7 7700K beat a lot of 8th gen entry-mid-level CPUs etc

so it's a moot point from them

then they said "cuz security", but 7th gen has tpm 2.0

[dusk coral]

I used a 3570k in 2020, before i upgraded to 10850k, and it was nearly still fine, I gave it to a friends kid, and it plays the fortnite fine lol

[simple ravine]

right!

i refuse to spent money on an upgrade just "because"

i am super happy with this CPU still

[dusk coral]

I wanted to go ryzen, but with employee discount the i9 was only $200!

[simple ravine]

half off, not bad

[golden bane]

I still use my 3770 (OC to 4.1 GHz) and I don't have any need to upgrade

[dusk coral]

there seemed to be an inflection point over the past year where more games are utilizing more than 4 cores consistently

[rapid pagoda]

I was using a crusty old 2500k up until 2020 🙂 it was starting to struggle a bit in poe but otherwise still quite usable

[dusk coral]

im spoiled, I play at 160fps/240hz at home, and it us a HUGE differance in my apparent skill compared to using my 60fps laptop that dips when screen is busiy, I even controlled for monitor and peripherals

I made a minion build specifically for laptop to play at work, kept dying on my bow character

[simple ravine]

#wfh-forever

[golden bane]

I can still get to stable 144 Hz in e-sports titles, PoE performance on the other hand is all over the place

[dusk coral]

I did it one week when I was quarantined, and I cant do it, i dont have the self control to not go to bed halfways thru shift

[simple ravine]

what do you do?

[dusk coral]

Process technician, kinda hard to explain it in the terms of intels mfgr process

[simple ravine]

bUT KuStoM enGINe iS beTTER

[dusk coral]

Im like a non-exempt engineer with no signing authority, like a nurse practioner is to a doctor

same, pay, just not on a management track without a degree

[simple ravine]

that is greek to me

[dusk coral]

half the technicians at the fab are former military technical jobs, I was a nuclear reactor operator in the navy

[simple ravine]

sub marines?

[dusk coral]

yea, not a fan lol

[simple ravine]

it's pretty cool though

[dusk coral]

it was an experience, I wish I just would have done good in school instead tho

chose the painful path

[simple ravine]

got to have some balls swimming around under water with a nuclear plant onboard

[dusk coral]

lol the nuke plant is super safe, im a very anxious and scared person naturally, and after my training and knowledge of how it works I had complete faith

[simple ravine]

in case of war though, if one of those gets blown up by a torpedo?

[dusk coral]

that would suck, I wouldnt do it then

[simple ravine]

i mean sure, the men onboard would of course not survive either way, but for the rest of the population

[dusk coral]

it wouldnt be as bad as you think, the design is pretty solid and the ocean is big

the fear of radiation is overblown, yes we should minimize uncontrolled release, but you get more from smoking cigs for a year than a career as a nuke

[simple ravine]

I mean, blown up submarine, nuclear waste in water, water moves around etc

[dusk coral]

yea, dilution is the solution

[simple ravine]

heh

[dusk coral]

we'd still be concerned as a society, but the risks are statistical, you are broadly increasing overall cancer rates by some small amount like 0.01%

20+ years after the fact

[simple ravine]

that's just 700,000 more people around the world dying of cancer, no big deal

statistically

[dusk coral]

lol its still taken seriously, because theres not consent to exposure, but people take micro-risks all the time like eating fast food or speeding

https://en.wikipedia.org/wiki/Micromort

[simple ravine]

anyhow.. ramdisk = cpu bound reads = more stutter, due to not offloading async read to IOCP

(I think?)

[dusk coral]

im gonna try it on payday, i wanted 64gb anyways for vm

[rapid pagoda]

tbh I'm curious whether an explicit RAM disk actually performs any better than having the files in standard disk cache

[simple ravine]

probably not

the stutter in the beginning when loading when on a faster thing, like my Raid 0 2x NVMe disks, are probably because it loads the DATs and other assets faster while the animation is running, and saturates the data to memory faster

if that makes sense

just a theory

[dusk coral]

itd be cool if they just kept hideout loaded

[simple ravine]

that I agree to

[rapid pagoda]

(also curious whether the organization of DAT files into bundles is optimal… I suspect it could be improved quite a bit)

[simple ravine]

the best thing with killing Sirus is not to have to load Oriath

[simple ravine]

I mean, they could stop fooling around with 128 bit fields to start with 😂

[dusk coral]

people have gotten so technical with poe, they just need to make a 2d hideout workflow, a PoE OS

[rapid pagoda]

actually I take that back, the data bundling is smarter than I thought

[simple ravine]

I load Mods.dat (not all fields tho) in about 35 ms right now, and if I code-gen a Mods model based on spec, I'm sure I can get that down to like half

I need to play with the new c# code-gen feature, it's pretty neat

so many new toys

also NativeMemory ... what could go wrong? 😂

look mom, c in c#

[golden bane]

I'm porting my GGPK tooling to Rust now, while also learning the language. Doing low level performance optimisation is super fun to me, mainly because I only used higher-level languages until now

[simple ravine]

yeah it's fun, when you have a language that allow you to do that

I suspect there's a lot of things you can do with Rust in that regards

.net in this day'n'age has a lot of that stuff as well, hence why this is my I think 4th rewrite of this code since 2017 😄

[golden bane]

I found this nice program for Rust profiling, the only one that works on Windows without trouble right now

[simple ravine]

nice, it looks like the profiler built-in visual studio

[rapid pagoda]

I just find it hilarious how many languages we've ended up with tools written in here 😆

[golden bane]

Yeah, for example for the bundle stuff, I can swap out the default HashMap hasher for FNV-1a and use hash memoisation with the data in the index bundle instead of hashing all keys again

[simple ravine]

In visual studio, you have different profilers, the one you showed would be the checked one

[golden bane]

I'd use VS (Community) if it had support for Rust

Superluminal is paid, but I guess I'll buy it once my trial runs out since it works really well

[rapid pagoda]

for a while my GGPK toolchain was actually written in perl, but that's best forgotten

[golden bane]

How does binary data parsing in Perl look like?

Do you just use regex like for everything else?

[rapid pagoda]

https://perldoc.perl.org/functions/vec, https://perldoc.perl.org/functions/unpack

[simple ravine]

API surface for DAT reader looks a bit neater now

[golden bane]

Looks close to what PoB does

[simple ravine]

my end-goal is to code-gen the dats tho, so you don't have to hand-write it for general consumption stuff

[empty venture]

Yesterday I asked here about any source of 2D item assets used in the game. Just for your information, I've went ahead and asked support if ggg is ok with me using assets from Bundle and their reply is no. Kinda sucks but well ¯_(ツ)_/¯

[simple ravine]

Their answer will always be the default no

[golden bane]

Contrast that with Chris' "We like Path of Building" response

[simple ravine]

if you ask them "Are you cool with us data-mining the .dat files" their answer will be no too 😄

PoB and many other tools would not be possible without the reverse engineering of that said data.

[empty venture]

But I guess using actual assets is kinda different, no?

[golden bane]

PoB does that too, for the skill tree

[empty venture]

so are u saying I should just go for it?

[simple ravine]

It Depends ™️

[empty venture]

lol 😄

[golden bane]

Don't get too attached to your account, but yes

[simple ravine]

I guess intent is key

[worthy cape]

[golden bane]

That too

[empty venture]

Tbh when u think about it, it isn't that much different from using their official API and resources at web.poecdn.com

so they shouldn't be bothered too much

[worthy cape]

Another member in the rejection club, we meet on Tuesdays. 😉

[empty venture]

count me in! 👍

[frank drift]

pretty much as a matter of policy their answer has to be no

there's no good way to say yes without it causing them a bunch of work

[worthy cape]

Everything from reflecting negatively at them when the file formats change, to interference with the patching or runtime via locking, or misbehaving when misinterpreting the formats.

Shipping baked data is much more reliable, all the pain is with the tool developers.

[simple ravine]

when they say yes, implicit expectations that they'll support your endeavors and not cause breaking changes, etc

[rapid pagoda]

what I'd like would be some sort of gentlebeings' agreement with GGG in terms of unreleased and recently released content

I suspect they're much more bothered by data mining spoiling the existence of new content/mtx, than data mining revealing obscure numerical details of how some mechanic works.

[simple ravine]

if they'd had to choose indeed.

but pandora's box and all

when the djinn is out

[worthy cape]

That indeed. I've seen some stuff in the files for sure which I've mostly kept my gob shut around, and intended to mask out from tools.

Like WIP or ready hideouts before they're actually launched.

Don't mess with the revenue stream.

[simple ravine]

yeah, but we know that there'll come along someone who won't abide by that

and i believe has been in the past as well

[velvet fog]

I removed almost all .php entry point

[rapid pagoda]

Is that a wobble of agreement? 🙂

[compact isle]

I don't have a nodders anymore :(

[vapid pulsar]

Morning people, as requested @worthy cape
https://www.youtube.com/watch?v=XNN0RLIfg-Y

[worthy cape]

If it's not too much of a problem, I'd love to have the captured footage to avoid double-compression of some of the rougher videos.

[vapid pulsar]

Sure ill upload the raw file now too

[vapid pulsar]

https://mega.nz/file/lY4TjKCJ#s1QdDZopvuGcQnwUPYaXqV2z5xuQw4eYfUIg6AnLWbQ

[paper jetty]

Hi

may i speak to the maker of POB

[velvet fog]

@paper jetty you can directly submit issue or request to https://github.com/PathOfBuildingCommunity/PathOfBuilding/issues

[hazy fiber]

it depends about what, if its an issue or a request for somthing then this github chuanhsing linked it the ideal place, if its about somthing else then you can pm me and I can forward you to the correct person

[velvet fog]

@hazy fiber may I ask monster critical strike multiplier, is it fixed to 130%?

[hazy fiber]

PoB doesnt do monster crits, for game in general?
I thought they had the same base 150% we have (with 5-6 base crit chance) and then increased by things like map mods

I was going to implement monster crits into PoB at some point while working on my calc defences rework but I have been neglecting it for a while and need to get back to it

[velvet fog]

I saw it is 130% in pob and wiki

[oak estuary]

It's 130% for minions, @hazy fiber I'm sure openarl got that number from the wiki, though, so if that didn't have a source I'm not sure where it came from.

[hazy fiber]

its very possible monsters have 130%, its been a long time since I tested it (pre 2.0?) and dont think my testing method was very accurate back then

[velvet fog]

it seems wiki use fixed value for all monster

[hazy fiber]

if its not in the game files it makes it a bit of a pain to figureout

[velvet fog]

that's why I ask this question :p

[hazy fiber]

changed in 2.3 and I never updated my notes, its 130%

A monster's base Critical Multiplier is +30% or atleast is stated as +30%
https://www.pathofexile.com/forum/view-thread/1667259 (under enfeeble nerf)

[velvet fog]

woo..thanks

[hazy fiber]

old old enfeeble use to give like -40% crit multi and I swear you needed atleast 25% curse effect to cancel it out pre 2.0 (my old test method) but clearly that mighta been flawed

[oak estuary]

That means people could use enfeeble with reckless defense and reduce all damage, or can crit multi not go negative to reduce damage?

[hazy fiber]

not go nergative

but also enfeebles damage cap was broken for a while and you could on a cursebot reduce it all the way to 100% less damage delt by monsters

so that worked anyway

[oak estuary]

Yeah I knew about that one.

[gilded tree]

I think the profiler should work if you attach to the process via the c++ debugger

But not quite sure, never tried that. But it should work with the CPU profiler

[timber path]

Want to know a funny thing? After some digging, I found out that assigning a CellStyle when adding the column headers, causes a huge performance loss. The cell style I'm adding is quite simple though:

Style style = new Style(typeof(DataGridCell));
style.Setters.Add(new Setter(ToolTipService.ToolTipProperty, new Binding($"{columnName}_Tooltip")));

Just a tooltip thing, but without it, it seems to perform just like before 😮

[golden bane]

In Qt, you can call hints that you're about to change the layout/styling, and also temporarily disable some of the background reformatting/restyling. Together with lazy data population, you get good performance this way. Maybe your tech has something similar

[simple ravine]

I saw that there were some that hid the headers while updating it, and then added it back

[simple ravine]

but do you need that much casting? a lot of boxing/unboxing that will happen

actually just one now that i look at it

but yeah

[worthy cape]

Either my code is bad, or my storage/server is bad 😄

[simple ravine]

oof

[worthy cape]

UPDATE pack_entry SET manifest = (SELECT manifest.manifest::bigint FROM manifest WHERE manifest.id = pack_entry.manifest_id);

Denormalizing my database, live 😄

[timber path]

Probably casting & some other areas can use performance boosts, but that's not the root cause it seems. When completely disabling CellStyle for the columns, it runs decently enough to be workable.
Also regarding the headers, the issue isn't in the initial loading, but once it's loaded, the scrolling is hell. That's my first concern. The longer loading times (likely caused by other things such as boxing/unboxing etc...) is an issue on its own and shouldn't contribute to the scrolling issue.

[simple ravine]

Soon ™️ we have WinUI 3

there's also MAUI that's interesting, especially for the things you're doing

[worthy cape]

I’m still bummed out that Dear ImGui’s tables cap out at around 64 columns.

[timber path]

I've already spent too much time optimizing this thing, I don't want to spend much more time looking at other frameworks for UI controls :/

[empty venture]

is there any data regarding unique items in the bundle? at least name + artname?

[hazy fiber]

iirc all data related to uniques was removed, but I think the art is still in there

atleast its on poedb before we know the uniques

[empty venture]

yeah, art is still there

but with their internal naming

[worthy cape]

Not sure if there's anything linking the ItemVisualIdentity and BaseItemTypes with the name.

[empty venture]

baseitemtypes got a name in it

but thats for base items only, no uniques in there

[sharp lark]

is there an alternative to currencycop? i want the thing that lists all my items by price, and total net worth. can't get currencycop to work anymore

[worthy cape]

Kids these days seems rather fine with Exilence Next for networth estimation.

[gloomy palm]

The way PoB updates its files... It seems very foolproof, but I wonder if any developers know if it's actually a good design?

[worthy cape]

Biggest hurdle is when it's installed somewhere where it needs more rights to write to stuff.

Don't think we've had too many bugs outside of that (black console window sticking) over time.

Hashing the local files had a significant hitch in CPU when starting up in the past, but we optimized that by using better sha1 code.

[gloomy palm]

Do you just manually need to run something that generates the manifest file each time a new update is made?

[worthy cape]

Yeah, it's a semi-manual step as part of cutting a release.

[gloomy palm]

Thank you for the information.

[worthy cape]

Recovery from a botched update, should we ever push one, might be icky and require a reinstall. If any fatal flaws in updating are found, a reinstall could also be needed to move onward as well.

Not naming names, but a recent release didn't include two Lua files in the manifest and blew up nicely. 😄

https://github.com/PathOfBuildingCommunity/PathOfBuilding/issues/3115
https://github.com/PathOfBuildingCommunity/PathOfBuilding/commit/504889f25ffafc2138139b485ef4e785ec8261d1

[gloomy palm]

I only took a brief look at the code, but it seems like it should be able to recover from any state as long as the files on the server are fine, and the updater itself is undamaged.

[worthy cape]

Program needs to make enough forward progress to actually complete the update check and do the staging. Some failures may blow it up earlier.

[gloomy palm]

Ah, checking for the update isn't the first thing, right...

[worthy cape]

Most updates are done in-place by PoB itself. The separate Update executable is only used when it needs to replace things like runtime files that cannot be replaced in use.

For that it stages it ping-pongs back out.

[gloomy palm]

Cool

[worthy cape]

The update check is done as a subscript as it can take a while to hash/download/etc. things; before we optimized it it could take minutes.

[dusk coral]

had GGG commented on ever making an addon system like WoW's Lua addons. PoE could co what royale games did to FPS games if they drastically improved QoL, theres so many activities compared to diablo3

[frank drift]

multiple times, the answer is no

Chris went over it again in the podcast yesterday

[dusk coral]

ill have to suscribe, im new to the game (just got to endgame). The inventory mgmt is killing me because im OCD about it but lazy

[golden bane]

Yes, this file: https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/master/update_manifest.py

[frank drift]

https://www.youtube.com/watch?v=88N4s2qxRIM no idea about the timestamp though

[golden bane]

Please not more Lua, PoB is enough already 😄

[frank drift]

tl;dr is Chris wants a consistent UI experience and doesn't want the testing complexity/exploits

[dusk coral]

that makes sense, but I hope they still prioritize QoL, if only just inventory mgmt

like id be happy for a whole league to be just QoL

[frank drift]

QoL doesn't make revenue sadly

[dusk coral]

mtx whales go on strike for QoL

[empty venture]

I'd say Lua is quite popular for any modding/addon interfaces. Not so much for writing stuff like PoB 😄

[golden bane]

The language is alright, there's just no tooling or libraries

[empty venture]

Just in case someone ever needs it, I managed to find a way to get "unique item name" to "art name" mapping from the Bundle. The key here is UniqueStashLayout.dat which you need to join with Words.dat and ItemVisualIdentity.dat

[worthy cape]

Sneaky.

[empty venture]

@timber path, created a pull request in your PoeAssetUpdater project with getting that mapping. No idea if you need it tho. (Was using the project to extract base item names / art already)

[timber path]

Thx for the PR. PoE Overlay (which I use the asset updater for) doesn't actually require them, it mainly looks at the base item types and the unique item name is more as-is "flavour" or rather "search" (plain) text. The other PR related to art names will be obsolete, since we can't use path-based art image retrievel anymore starting next league, so it has to be ripped out and replaced with images obtained from the static poe trade data api (rather than compose them using the client files)

[swift beacon]

Nice. I haven't had a way to generate the models for uniques because I couldn't map the names to the 3D files easily, but this might be the key.

[empty venture]

"we can't use path-based art image retrievel anymore starting next league" - wait... what? :x

got any source for that? it really sucks if true :/

[worthy cape]

#tooldev-general message

We're in deprecation league right now.

[empty venture]

But that refers to some "official" API hosted by GGG, right? I've thought of using assets bundled with the game which currently have the same structure.

[worthy cape]

Yeah, the thing discussed there is constructing CDN paths based on the internal names you see in game files, which is going away in favour of URLs with richer encoded content and a bit of hash for the lulz.

Not familiar with what WhiteFang's thing does, but I assume that names used for game data will remain names used for game data.

My parse is that they'd have to go from using the CDN with the easy paths to using the CDN with paths harvested from trade, rather than go the possible but icky path of using game files.

[simple ravine]

.dds -> imagemagick -> .png

this is the way

[empty venture]

yep, thats exactly what I'm doing

[worthy cape]

Requires shipping more assets tho, or hosting your own CDN.

[simple ravine]

well, if you're fed-up with all the lulz, then...

[worthy cape]

Did a bit of housecleaning in my DB, 70.3G to 3.63G on-disk.

Turns out that deleting 100G of binary data helps.

[simple ravine]

that ought to make your queries slightly quicker

[worthy cape]

The big entry table is still big tho, so unless all that blobby data was somehow fragmenting things, it shouldn't have affected it much.

At this size I can afford backing it by SSD too or put it out in the cloud somewhere, which is nice.

[simple ravine]

I'm not a postgres expert, but (yes, i hate people starting out liket his, but give me some credit)... storing that data will mean that your rows use more than 1 page

if you fit your row in less than 1 page, you'll be good

and if your primary key is not linear, you'll get fragmented stuff, because I think pg stores it in some kind of tree structure

[worthy cape]

Distinct table tho, file (hash, data) and pack_entry (path_id, manifest_id, data_hash)

The former had all the bulk data, the latter has all the directory entries.

[simple ravine]

¯_(ツ)_/¯

[empty venture]

I have to admit I know nothing about CDN ¯_(ツ)_/¯

[worthy cape]

My toy one has the best accidental feature where nothing has mime types and as such, it’s largely useless for hotlinking.

And also, no filenames.

[empty venture]

all the 2d item icons have 250MB in total, the ones I actually need somewhere around 60

I think it shouldn't be too bad

I probably could even cut 2/3 out of it by resizing

[rapid pagoda]

@worthy cape At some point I'd like to get an archive of the Data/ directories you've captured. No rush

[worthy cape]

Been kind of hoping to get them into a mini version of the big dedup repo so I could zip them up and put somewhere.

Don’t think I have any handy atm.

[rapid pagoda]

Might also have some queries about hashes in Audio/Dialogue, but less likely to need a dump of that one unless there's something really weird that popped up for a short time

[timber path]

The poe asset updater exports image paths that are later used by poe overlay to construct a url to ggg's cdn to display currency icons, fragment icons, map icons etc... In the price check window and trade notifications. But that "url construction" part will change next league, so I will change the asset updater to accommodate this change on ggg's end.

[empty venture]

that sucks, I'll have to move to something else to get those paths then

What else is there for dealing with dat files? PyPoe and anything else?

[simple ravine]

Depends which language and what you're looking for

If you're looking for a tool that will allow you to view dat files, then you have SnosMe's dat-viewer which works in the browser, really nifty

WhiteFang's application can view dat stuff as well

ItThatBarks's pogo exports things

I'm working on PoeSharp, which you'll need .net 6 preview 7 or later to use

[worthy cape]

Bleeding edge, not surprised 😉

[simple ravine]

I'm a masochist, I confess

[worthy cape]

andersen here already playing Chris' Hard Mode 🙂

[simple ravine]

Not gonna lie, I was pretty happy when he mentioned it

[empty venture]

need something to read dat files, process and export to json

[simple ravine]

as a library? or a program?

[empty venture]

or a standalone thing, doesn't matter really

[simple ravine]

you have RePoE that has already-exported stuff

SnosMe's dat-viewer allow you to export dat files to json

[empty venture]

RePoE is missing unique items and thats what I mainly need

[vapid pulsar]

Unique items no longer exist in the client game files. They are all handled server side now.

You best bet will be to use the wiki query tools

[empty venture]

their names, visualIdentityKey and appropriate asset do

[vapid pulsar]

That is true. IF that what you want then use SnosMe's tool

[simple ravine]

https://snosme.github.io/poe-dat-viewer/

is there a programmatic way to get the current patch number?

[velvet fog]

yes, from patch server

[simple ravine]

ah yes, now that you mention it, I think PyPoE has some (incomplete?) patch things, right? I could get inspiration from that

or if you have any other reference?

[velvet fog]

https://github.com/OmegaK2/PyPoE/blob/dev/PyPoE/poe/patchserver.py#L521

it still works for me

[simple ravine]

ok

I think I will implement something like this as well, for completeness

[vapid pulsar]

You have to use the path server's binary protocol

[simple ravine]

yeah, I think I tinkered with this a couple of years back and got it working

[worthy cape]

Assigning patch versions to builds is the only thing I don't have a way to do with my thing.

[empty venture]

How does that thing work? I enter 3.15.0, press import and get an error: "you may need to adjust the patch version"

[vapid pulsar]

3.15.1.2

is the current version

[empty venture]

oh, I see

[worthy cape]

Been correlating it with the wiki list of versions and their timestamps, but it's super icky.

[vapid pulsar]

You can get the patch version easily from the Client.txt

[worthy cape]

Couldn't find anything in the executable that said what its version was, just the incomplete release string in the lower right corner.

[simple ravine]

It ought to be somewhere... likely in the ggpk or somewhere no?

or does it just get the latest and do a diff?

[vapid pulsar]

The version is hard coded into the exe

[velvet fog]

can't find any version in ggpk file

[worthy cape]

I only found this one, but I'm inept at grepping for binary data and the version expressed as bytes is a bit short:

[velvet fog]

seems possible to find this string using strings2 before

[worthy cape]

This string, yes, but it doesn't fully express hotfixes and whatnot.

[velvet fog]

ya..

[worthy cape]

In any way, the executable manifest is not always updated in builds either.

[vapid pulsar]

There is a full version like this in the client

[worthy cape]

Like this one, only touching data and macOS:

[velvet fog]

and they add encryption after protocol version 4, it's version 6 right now

[vapid pulsar]

You can still query with the old protocol version

Last I checked anyway

[velvet fog]

nope, they hide Bundles/ and Bundles2/ in version 4

[vapid pulsar]

Ah darn

[velvet fog]

dir list in version 5 and version 6 are different, seems different encrypts

[timber path]

You're still free to use my tool, since it also combines dat reading & visualisation of them. Maybe you can create a separate project within the solution, or keep your own changeset; maybe create a fork of the repo or something... Or move the things you need to a completely separate method and json output file. I'm fine with ignoring certain outputs and commenting methods not used for poe-overlay's asset generation. 🙂

[empty venture]

just fork it, stupid me 🤦‍♂️

[worthy cape]

In this house, we adhere to the Waterfall Model 😉

[worthy cape]

The SVN rev (or whatever the tail end is) seems very new, it's not present in most versions I have.

It was never present in the pre-bundle era at least, I don't have any EXE datasets since.

Rumor has it it's Friday today, hoping to dig into some things this weekend.

I'm likely to use Steam buildid as the primary identifier for releases and annotate them with approximate version (from tags, from wiki matching, whatever).

[velvet fog]

Plan to resurrection my Diablo 2 website this weekend

[worthy cape]

Like something like this:

[simple ravine]

wait, so there is no way for us to download the bundle files programatically anymore?

oh wait, we can use Steam to get things no?

or do u need to authenticate with steam?

[velvet fog]

steam download need auth

[worthy cape]

You need an account with a free license.

[simple ravine]

I guess that's not a showstopper for some workloads

[vapid pulsar]

You can still download bundles from the CDN

[simple ravine]

but you need to acquire the patch number, and if I understood things correctly, it's not possible anymore?

[worthy cape]

DepotDownloader loves to ask me for a SteamGuard code every now and then, not sure if I'm holding it wrong or if the sentry files have a finite lifespan.

[velvet fog]

version number is not encrypted

[simple ravine]

is it some proprietary encryption, or just TLS?

[velvet fog]

tcp socket?

pypoe still work on protocol version 4 to get patch url and version number, just miss Bundles/ and Bundles2/

[vapid pulsar]

Quick and dirty python script to get the cdn URL from the patch server
https://gist.github.com/Eingin/c248dfe8fd2224d09975acacf911caf5

[simple ravine]

this is one of those things I'd probably want to make into a serverless function, which grabs the latest and commits it to some repo so it's available to anyone who needs it

[vapid pulsar]

Yeah, would be pretty simple

[simple ravine]

struct.unpack("<b8Ib", data[:34])

what does the 8 signify there?

8 ints?

https://docs.python.org/3/library/struct.html#format-characters this list is confusing me

[vapid pulsar]

Yeah 8 ints there. there is a 256 byte value there, always zeroed out

[simple ravine]

I'm receiving 201 bytes from the server, hmm

[vapid pulsar]

the packet has a structure like

opcode = 1 byte
hash = 32 bytes
primary string length = 2 bytes
primary string = primary string length * 2 bytes
backup string length = 2 bytes
backup string = backup string length * 2 bytes

[simple ravine]

I'm sending the bytes 1 and 6 to the server

I'm guessing that's wrong then

[vapid pulsar]

That is correct

the request is two bytes

opcode = 1 byte
version = 1 byte

[vestal pond]

This is my code to get the patch server url

[simple ravine]

wait u're sending 1, 4

[vapid pulsar]

You can send version 4. its a much more simple protocol

but will still respond with the header

[simple ravine]

strange that I only receive 201 bytes then

[vapid pulsar]

That sounds right.

ah the hash is 256 bits not bytes

[simple ravine]

ah 😄

[velvet fog]

opcode = 2 byte
hash = 32 bytes
primary string length = 2 bytes
primary string = primary string length * 2 bytes
backup string length = 2 bytes
backup string = backup string length * 2 bytes

opcode
01 xx: request server info, xx is protocol version
02 00: response server info
03 00: request folder info
04 00: response folder info
05 00: request patch notes
06 00: response patch notes

[simple ravine]

ah there we go

[round hazel]

@worthy cape I guess i am really dumb on using non-GUI. What should i do to output as a text file for your bun's "list-files" function?

[worthy cape]

list-files prints on standard output. You can say bun_extract_files list-files blargh.ggpk > biglist.txt

[round hazel]

ok

[worthy cape]

Similarly, extract-files accepts a list of files on standard input or on the command line, as well as the possiblity to use regexes to match files.

So if you make a list with list-files, cull it down, and pipe it into extract-files, it does the right thing.

[round hazel]

thx

[worthy cape]

You can thus extract the whole pack with either one of these:

bun_extract_file list-files Content.ggpk | bun_extract_file extract-files Content.ggpk C:\Temp\poe
bun_extract_file extract-files --regex Content.ggpk C:\Temp\poe ".*"

[simple ravine]

https://raw.githubusercontent.com/poe-tool-dev/latest-patch-version/main/latest.txt

https://github.com/poe-tool-dev/latest-patch-version

there we go

automatic update of patch version to git

[simple ravine]

@worthy cape Do you by any chance know if there's a libooz/oo2core_8 linux version?

[worthy cape]

My stuff builds fine on debian buster.

As for Oodle libraries, heaven knows.

[simple ravine]

How do you read bundle files on linux then

[worthy cape]

With Bun, as it builds fine 😄

I mean "Oodle libraries" as in official RAD/Epic ones.

[rapid pagoda]

oo2core is the official library. ooz is an open-source reimplementation of the decompressor

[simple ravine]

oh wait; for some reason I thought libooz was using oo2core

[worthy cape]

All you need to build my fork is libunistring, libsodium, cmake, a c++ compiler and competence.

[simple ravine]

hm ok, that sounds like a job for a Dockerfile

reason is I'd like poesharp to be cross platform compatible, and right now, I am just bundling, no pun intended, it with the libooz.dll file

so if there would be a libooz.so file as well, that'd make that possible

is https://github.com/zao/ooz the source you used to help us kindly compile the libooz.dll

[worthy cape]

That's the repo, yep.

It's gonna be liblibooz.so, for amusing reasons 🙂

You probably don't need the deps if you're just building the ooz library, but not sure if there's a knob to omit the rest.

Nothing a bit of # can't solve in CMakeLists.txt tho.

[simple ravine]

yea doing the other things directly in .net

is that a good idea?

[worthy cape]

Use CMake, don't use any old stale vcxproj from the previous project.

[simple ravine]

I thought visual studio supported cmake nowadays?

[worthy cape]

It does, but you should open the directory so that it does the CMake thing.

There's stale old project files in the directory from the project I forked, before I added CMake.

[simple ravine]

ah oh wait, building this builds an ooz.exe

[worthy cape]

It's not super obvious in VS, but the way you use CMake is to "Open a local folder", not "Open a project or solution".

[simple ravine]

ah

I found C++ for Linux in the visual studio installer as well

adding that to see if I can get away with using that

ah nice, it built a libooz.dll

but it says 'x64-Debug', should I worry about getting a 'x64-Release' configuration made?

[worthy cape]

Don't forget to change the configuration to x64-RelWithDebInfo or x64-Release, unless you want it slow.

Release is more optimized than RelWithDebInfo, but lacks debug symbols and stuff.

[simple ravine]

yeah, I'll make a new build configuration

there we go, 105kb instead of 800kb

I see there are now options to add linux using gcc or clang

yeah it wasn't liking that much

haha great, thanks for the detailed information

[worthy cape]

You can drop everything below like line 38 in the CMakeLists

I'll probably do something like -DENABLE_BUN=OFF in the future if I can remember.

I want the OOBE to build Bun tho, at least until the day when I figure out how to move this to another repo.

[simple ravine]

well I'll be damned

nice, thanks for the help

[worthy cape]

[simple ravine]

It'll probably be some headache to figure out if there will be signature differences between the two and stuff

460kb tho, does that look right?

[worthy cape]

Probably. Two megs here, 674k stripped, all in Debug.

-rwxr-xr-x 1 zao users 462K Jan 8 2021 liblibooz.so

Release.

[simple ravine]

ok

hmm, the one I had before from you was 549 kb (dll)

[worthy cape]

The set of binaries I have are at: https://zao.se/~zao/poe/

-g suffix corresponds to a git commit

[simple ravine]

I see ooz-g...zip which contains ooz.exe

[worthy cape]

Those have the full suite.

ooz/libooz/libbun/bun_extract_file

[simple ravine]

hmm, im extracting Data/. (410 mb, 2770 files) in about 7.5-8.0 seconds

I'm not super happy with that

[worthy cape]

So do you select specific pathreps that map to that directory, or do you expand all paths and then group by bundle?

[simple ravine]

I do a full index of all the files and then find the /Data/ folder

that takes about 700-800 ms

but after that it's indexed in an efficient tree structure

do you have a similar way to extract with your command line?

[worthy cape]

bun_extract_file extract-files --regex src_dir dst_dir "^Data/.*$"

Or if you're silly, ... list-files ... | findstr Data/ | ... extract-files ...

[simple ravine]

ah, that did all the languages

[worthy cape]

You'd do [^/] instead of ..

Didn't bother implementing alternatives to regexes like * and ** globs, because they made my head hurt.

[simple ravine]

3.1 seconds

that's fast

[worthy cape]

Does a full expansion of each path-rep and collects the paths that match any regex, IIRC.

[simple ravine]

I'm gonna re-evaluate my career choice now

[worthy cape]

Builds no directory structure nor keeping the full set around.

https://github.com/zao/ooz/blob/master/bun_extract_file.cpp#L143-L167

[simple ravine]

I wonder if it's slow because I am P/Invoking the ooz.dll

[worthy cape]

You should pay for that once per bundle, shouldn't you?

Or are you going once per block?

(I keep forgetting what's in bun and what's in ooz)

Should be enough work done to not really matter.

[simple ravine]

per "get me the file in this bundle-file (i.e. Mods.dat", it'll determine the proper bundle file from an index (dictionary), read the header and decompress the blocks it needs, and return that

[worthy cape]

But do you bundle related files together, to share the decompression cost of the same block?

[simple ravine]

nah

it's pretty naive that way

[worthy cape]

That's a big win in not re-decompressing things, even if they're in file cache.

[simple ravine]

time to attach a profiler

hmm

I don't cache the compressed bytes in memory

and re-reads it from disk every time I want to extract a new file

[worthy cape]

I guess it's a bit ickier if you're a general purpose lib and not a dedicated batch extractor.

[simple ravine]

yeah

[worthy cape]

Maybe indeed cache blocks and try to order your decompresses by bundle, or make a batch interface.

[simple ravine]

do you keep the compressed stuff in memory?

or just be smart about each bundle file?

I suppose I could do that actually

or hmm, no

[worthy cape]

I gather all candidate file records referencing a bundle, decompress the whole bundle, and slice out the parts to files.

I could also be smarter there and decompress the exact blocks covered.

But it's all one-pass, I never touch a bundle twice.

[simple ravine]

gotcha

yeah I have to think about this

[worthy cape]

Even in my real applications, it's either bulk decompression up-front, or few files on user actions as they work an UI.

Bulk ingest/exports really benefit from batching.

[simple ravine]

yeah, library development isn't really my strong suite heh

[worthy cape]

I'd like something a bit more async for real apps, as blocking sucks.

[simple ravine]

i need your ELI5 sometime on the path reps though, because I think there's smarter ways there

haha I thought about async today too

however, async is really good at killing perf

[worthy cape]

Given a full path to a file, you can locate the path_rep to the directory it's in.
Given the full path to a directory that directly contains files, ditto.
You can unfortunately not reliably get a recursive tree for a directory that only directly contains directories.

[simple ravine]

like git in that sense, no leaf (file), no dir

[worthy cape]

I forget if there's any overall order to path reps apart from the hierarchy, so you may have to strip off components from the directory until you find a parent rep you can filter.

Even worse, no direct child, no dir listing.

Consider this tree:

foo
|- bar.txt
`- baz
   |- qux
   `- quux
      `- mu.txt

There's no path_reps for baz or qux, as they contain no files, even though they contain directories that eventually contain files.

[simple ravine]

hmm

lol...

dotnet run -c Release
Opening bundle from GGPK...
Opened Bundle index. 00:00:00.8145348
Extracting...
00:00:12.1203941
Done.

when single-threading it

this is terrible

goes looking for jobs at mcdonalds

ok, time to do some wrangling

[worthy cape]

I've been setting up dedicated VMs on a different home infra host for serving data now.

I need larger SSDs 😄

The DB VM has 32+32G disk, the dedup VM has 32+600G.

[sour perch]

whyt cant i stream yet lol?

my boi wants to see my build in action

[vapid pulsar]

This is a channel for discussing 3rd party tools, not issues with Discord.
But to answer your question, see #🔊voice-help-and-rules🔊 Rule 7.1

[simple ravine]

Opening bundle from GGPK...
Opened Bundle index. 00:00:00.7924271
Extracting...
00:00:01.8576342
Done.

that's better.

[simple ravine]

@worthy cape my trick is a bit cheesy though

I wrote this:

public class SimpleCache<TKey, TValue> where TKey : notnull
{
    private readonly Dictionary<TKey, TValue> _cacheDict = new();
    private readonly Queue<TKey> _invQueue = new();
    private readonly int _maxCached;
    private readonly object _lock = new();

    public SimpleCache(int maxCache) => _maxCached = maxCache;

    public TValue GetOrAdd(TKey key, Func<TValue> whenMissing)
    {
        lock (_lock)
        {
            if (_cacheDict.TryGetValue(key, out var cachedValue))
                return cachedValue;

            var val = whenMissing();
            Add(key, val);
            return val;
        }
    }

    private void Add(TKey key, TValue val)
    {
        if (_invQueue.Count >= _maxCached)
            _cacheDict.Remove(_invQueue.Dequeue());

        _invQueue.Enqueue(key);
        _cacheDict.Add(key, val);
    }
}```

I feel a little dirty

and then I'll just do:

private SimpleCache<IFile, byte[]> _compressedCache = new(16);
private SimpleCache<uint, IFile> _bundleIndexCache = new(256);

[simple ravine]

For all who develop using Rust, I stumbled on something: https://github.com/microsoft/windows-rs

I'm not sure how useful it is, but that's at least something

[golden bane]

It's great that Microsoft is investing in the Rust ecosystem, but at at least for me, I prefer cross-platform GUI. Right now, I just use Rust to extend Python due to lacking GUI support

Also I don't know how much sense these bindings make if they don't provide safe abstractions over the Windows API. But true, it's better than nothing

[simple ravine]

yeah, there's WinRT bindings for Python as well available

[civic crane]

@simple ravine

Automatically updated Patch Version for Path of Exile
how? don't see any ci/cron

[simple ravine]

Azure function. https://github.com/poe-tool-dev/poe-patch-update

runs once a minute

[civic crane]

ok, just document it in readme so noone ask same question 😄

[simple ravine]

good idea, will do

[golden bane]

If anyone's interested, the Python WinRT bindings are tucked away here: https://github.com/Microsoft/xlang/tree/master/src/package/pywinrt/projection

[bronze kestrel]

What specifically do you extend using rust tough?

[golden bane]

A GUI application for GGPK/Bundle/dat parsing and reading, mainly

[bronze kestrel]

Just as a UX consideration?

[worthy cape]

The UI story in Rust is somewhere between nonexistent/immature and "use some stale fork of Dear ImGui on top of a graphics API".

[bronze kestrel]

Well this is true. Although they have a few electron-like projects. 🙂

Lemme find the one I had found before…. Not that you may have use for it but just a nice find of mine

[golden bane]

Yeah, kind of. I just couldn't get below 10s startup time on a hot GGPK

[worthy cape]

It's one of the driving forces that pushed me from GL+ImGui for my format playground and model viewer to C++ for my Hideout Editor.

On GGPKs, the world is so much nicer now with loose bundles in a Steam directory.

Can't wait to transition my local tooling to the world's jankiest data cloud tho.

[golden bane]

You might be thinking of Tauri here

[bronze kestrel]

^ bingo

I found it a nice little toy for certain uses.

It’s not the typical native binding ui kit though

A lot of “magic” there

But it’s decent compared to alternatives

[golden bane]

I use Python for Qt bindings, general instrumentation of the Rust library and user scripts

[worthy cape]

I found out the other day that Qt stopped doing non-commercial releases of 5 recently.

Guess it's time to focus on Qt6 for anything new.

[bronze kestrel]

Not even sure why I’m chiming in, I’m very much a CLI proponent. Heh

[golden bane]

Yeah, although not much changed. The jump from 5 to 6 is much smaller than from 4 to 5

[bronze kestrel]

If it can be done in one command, why have a UI sort of deal. Although there’s inherent value in making poe tools as accessible to people as possible.

[worthy cape]

The move to an integrated package manager seems like a major hurdle for the packaging project I'm involved in, nothing good come from more intelligence 😄

[golden bane]

Integrated package manager? Please elaborate

[bronze kestrel]

nothing good comes from more intelligence
Amen

[worthy cape]

One of the big things about Qt6 is that it is way more modular and piecemeal, integrating Conan or something which means that it's pretty much impossible to provide a monolithic good build of Qt on a cluster.

[golden bane]

I see. I despise these "inner systems"

Good that I can just get PySide6 from PyPI 😄

[bronze kestrel]

^ QT account let’s you make the best use of our services.

Smells like a lot of funky bs

[worthy cape]

Must've gotten bought by Fandom 😉

[golden bane]

Looks like someone's career depends on lots of people signing up

[bronze kestrel]

Just say “we don’t want to remain a FOSS project anymore” instead of beating around the bush. Yeesh

Might actually save face more that way

[golden bane]

They actually have to, due to the agreement with KDE

[bronze kestrel]

Well have to and want to are a bit different I’m wondering if they actually “want to remain” that way

[golden bane]

Short rant: They have some new feature flags behind importing stuff from __feature__, mimicking Python's own __future__. They must have felt very clever coming up with this. Wow, it's like Python, so pythonic. I could not imagine anything more unidiomatic. The "double underscore" namespace is for Python and Python only. The developers purposefully resigned themselves to such an ugly naming convention to not mess with their user's code. And what does Qt do? Copy them. Since the Python code developers are nice people, there probably won't be any __feature__ module in the future.
TLDR: Qt devs are namespace thiefes, because they don't understand the language they provide bindings for

Slightly OT but I had to tell someone 😆

[worthy cape]

Reminds me of the dude that insisted that he should be able to use reserved names in C and C++ because they ensured that he didn't collide with third party code.

So close, but yet so far.

[worthy cape]

TBH, considering that Qt kind of is a dialect of C++ already with MOC and signal/slot extensions, they probably already consider themselves the platform vendor of a Qt-enriched flavour of Python.

Rules are for people who don't define the platform.

[simple ravine]

nothing good comes from more intelligence
what do you mean?

[golden bane]

Sadly, that's true. First-party Qt support is a huge boon for Python. I just wish it was done a little better

[golden bane]

Probably just "Don't be too clever for your own good"

[worthy cape]

The further a software strays from a boring straightforward build which can be configured and adapted from the outside, the worse it is to package and deploy in settings that the vendor didn't expect.

Consider vendoring of libraries. A build that downloads libraries on the fly and builds them internally, that means that if you have a need to influence which compiler or flags are used from the outside, you're greatly inconvenienced.

[simple ravine]

reminds me of the S.O.L.I.D principles. they're pretty applicable here as well to some degree

[worthy cape]

A build that uses a weird "non-standard" build system has similar problems.

Software does these kind of things out of convenience or to enable some of their userbase, but often ends up making the work of others much harder.

[simple ravine]

especially O (Open/Closed) - Open for extension, Closed for modification

[worthy cape]

Another example is electing to deploy software in containers or Anaconda repositories, instead of having it properly buildable from source.

[golden bane]

Don' remember the original quote, but it's something like: "If debugging code is harder than writing it, and you're writing code as clever as you can, who's going to debug it?"

[worthy cape]

In my work world, there's two core requirements for non-binary software:

• it should build with whatever compiler and flags you tell it;
• it should use externally provided libraries as far as possible.

[simple ravine]

isn't that derived from the complete opposite of intelligence, lack of understanding and knowledge?

[bronze kestrel]

Sometimes trying to make things too smart only results in them causing issues or some sort of abstraction that’s neither good nor useful for the sake of being clever. My interpretation

[simple ravine]

gotcha, yeah I agree with that

[worthy cape]

Another aspect of "smartness" is builds that try to repair or adjust the environment to what they think is "sane".

[simple ravine]

relatable to dunning-kruger 😄

[worthy cape]

Bazel is horrible in this, as software like TensorFlow does its very best to scrub most of the external environment away.

[worthy cape]

@rapid pagoda Just checking before I go grab way too much, did you care about foreign Data or just English?

All Data dedup since tech patch is 41G, just English is 29G.

[civic crane]

@velvet fog can tell me which dat file has the most number of rows

[rapid pagoda]

Just English is fine. Can also skip datl/datl64

[velvet fog]

Do you know any way to convert bc7dds into png in linux server?

[worthy cape]

Not off-hand, I use g-truc GLI or DirectXTK (on Windows) to load the textures.

It has functions to copy regions and save to disk, so you could probably cobble a tool together.

[clever thistle]

imagemagick can't do bc7dds?

[velvet fog]

https://github.com/ImageMagick/ImageMagick/blob/main/coders/dds.c#L1996 seems no for dx10 bc7

[swift beacon]

I use texconv, which might work on linux.

https://github.com/Microsoft/DirectXTex/wiki/Texconv

[simple ravine]

MIT license, nice!

[simple ravine]

finally

it's alive!

[worthy cape]

I spent most of today on fussing over my DB schema and rewriting my ingest code.

[simple ravine]

I need to pick your brain about the path spec stuff

I'm annoyed by the initial 800 ms to index the bundles

Oh, zao lol

btw

I sped up the bundle extraction thing

in a very cheesy way 😄

I cache things hehe. I wrote a SimpleCache<TKey, TValue> that works fairly efficient

[golden bane]

Is this with mapping all paths to their file or without?

[simple ravine]

It's mapping all files and adds it to a tree structure (directory/files)

[golden bane]

I see. I get 426ms without the tree structure. That part is still in Python since trees are tricky in Rust

[simple ravine]

I have a gut feeling it can be lazy loaded a little

but 800ms for the whole thing is fairly aceptable

what bothers me a little is that it becomes slightly big in memory

but hopefully people who use it is smart enough to get what they need and dispose of it

[civic crane]

No caching, getting right from binary on demand https://github.com/SnosMe/poe-dat-viewer/blob/master/lib/src/bundles/index-paths.ts

[golden bane]

It would be cool if you could build a folder hierarchy from the bundle without having to index all files. But oh well

[simple ravine]

yeah, that's what I want to do as well @civic crane

I'm pretty sure I can get down to those numbers then as well

I am uber-impressed by the perf on your app btw, it's splendid

// Art dir is special

what's up with that?

[civic crane]

https://github.com/poe-tool-dev/ggpk.discussion/wiki/Bundle-scheme#bundle-file-format
you can quickly get child content by reading payload_recursive_size . Unfortunately for Art dir payload_size == payload_recursive_size or smth like this, so we need to fallback to more slower way of listing Art dir content @simple ravine

[simple ravine]

yeah that part confused the heck out of me

[golden bane]

That's exactly what I was looking for. I kind of forgot about payload_recursive_size, as my first iteration of the code didn't make use of it.
Your code looks very clean btw, very nice. Have you thought about using Rust for .dat analysis instead of C? It has native WASM support and the strong type-level features would go well with TS

[civic crane]

I'm not against looking at rust code and understand it quite well, but writing in it - no 😦

[golden bane]

What do you not like about writing it?

[simple ravine]

@civic crane Do you work professionally with programming? If not, you should consider it 🙂

[civic crane]

I don't like borrow checker, all these things about ownership, ugly extra template parameters for lifetimes. I feel like language attacks me and says: you are dumb, what are you doing 😅 I feel very unproductive writing it compared to just typed language

[golden bane]

The borrow checker is your friend. Don't fight it, let it help you write better code 😄 I don't bother dealing with lifetimes yet. When in doubt, just .clone() stuff and benchmark if you really need to do something more involved

[worthy cape]

Rust is great when the structure of what you're trying to do fits the language worldview. When not, it sucks.

I find my code a bit write-only without IDE annotations for each intermediary state of iterator chaining, said IDE tooling loving to just colour the whole screen red with squigglies and some vague error.

[golden bane]

What IDE do you use? I use Pycharm with the Jetbrains Rust plugin and get type annotations for everything

[worthy cape]

VSCode with rust-analyzer currently, I think, but it's a bit hit/miss, especially if in an async fn.

I couldn't use JetBrains, the debugger variable inspection is pretty much unusable.

No intelligence, only shows raw internals.

[golden bane]

I didn't have to debug my code yet, but look here: https://blog.jetbrains.com/clion/2020/05/whats-new-in-intellij-rust/#improvements_to_lldb_support
This should fix your problem I think. Paid IDEs only though

[worthy cape]

https://cdn.discordapp.com/attachments/839978057452617729/868688483810697306/unknown.png

Not sure if it's something specific to using a MSVC toolchain or if it's universally butts, but this was just the other week.

Seems to align with the screenshot of simple enough types in that blogpost.

[simple ravine]

I finally stopped paying jetbrains for dotUltimate

their IDE Rider was not that great.

couldn't find a reason to use it over visual studio

[worthy cape]

Kind of hard to motivate using something that's not the first party ecosystem for .NET

[simple ravine]

yeah, their ReSharper (R#) plugin can be pretty useful for some, but after like 20 years of .NET, it's kinda just in the way and slows the IDE down

it has some nifty refactoring tools, but that's pretty much it

[golden bane]

Do you have

[profile.release]
debug = true

in your Cargo.toml?

[worthy cape]

Don't remember what profile this was targetting, but probably.

Don't have anything JB installed since paving the machine either, might give it a go in the future again.

[simple ravine]

automation works!

[polar comet]

im looking for my programmer duo Faust

has anyone seen him

we write the code through API framework meta scaling CMD processing i-core computing mainframe index stationary trilinear bi-code. It's pretty legit.

[civic crane]

so 10 vaal orbs, ok GGG

[gilded tree]

I just hope there will be a rust analyzer plugin someday. Prefer it way more than the intellij implementation, it somehow feels better.
For example Procedural Macro’s weren’t working if I remember correctly. But seems like there is an experimental feature now which fixes this, maybe I should try it again.

[empty venture]

Do we have any PoB guys in here? What is the deal with <PlayerStat stat="SkillDPS" value="table: 0x17b94140"/> in the xml?

[golden bane]

I doubt that will happen. They roll their own plugin so that they can possibly sell a new IDE in the future

[gilded tree]

RustLand

[grave wren]

RustCharm

or maybe RustJ

[oak estuary]

What do you need to use SkillDPS for?

[empty venture]

I'm parsing the xml on my website and displaying some relevant info. If I'm not mistaken SkillDPS is the dps breakdown for specific active skills?

[oak estuary]

That being in the XML is just a side effect of how we use that table internally. Probably should just be excluded, tbh

[empty venture]

I'm currently skipping it obviously but I think I would be cool to have that breakdown in XML

[oak estuary]

The problem is this logic is fairly generic for all of those stats. Modifying it so the output makes sense just for one stat isn't very appealing from a maintenance standpoint:

for index, statData in ipairs(self.displayStats) do
        if statData.stat then
            local statVal = self.calcsTab.mainOutput[statData.stat]
            if statVal then
                t_insert(xml, { elem = "PlayerStat", attrib = { stat = statData.stat, value = tostring(statVal) } })
            end
        end
    end

[empty venture]

I see

[empty venture]

@oak estuary As you seem to be in the topic, got another question. When my res goes negative, XML doesn't contain XResist stat. Is it intentional?

nvm

my parsing was fked up :x

[simple ravine]

More abstraction/indirections solves every programming problem 😄

[empty venture]

I'm checking if a string is parsable to float but forgot those can be negative as well... only checking all numeric + dot 😅

[simple ravine]

most languages/frameworks have built-in stuff for that.. what are you using?

[empty venture]

Python. There is an isnumeric method but that only checks if all characters are digits. At the end went for the brute force method of try-cast-catch 😄

[golden bane]

That's the most idiomatic approach anyways

Easier to ask for forgiveness than permission

[empty venture]

according to some SO answer, checking first has better performance

[grave wren]

Never preoptomize

[empty venture]

I know, just in case someone is curious 😄

[worthy cape]

I'm post-optimizing \o/

Running content ingest on old GGPK packs and everything hurts.

[golden bane]

@empty venture Also check out my library for parsing PoB builds 😉

https://github.com/ppoelzl/PathOfBuildingAPI

[grave wren]

If only you had this when I started my bot 😄

[golden bane]

It's never too late so switch 😛

[grave wren]

Sure, pull request welcome

[empty venture]

already got most of the stuff coded anyway as well

[golden bane]

If I do, you're obliged to accept it 😄

[empty venture]

what really annoys me is that PoB XML doesn't have base item stats

like dmg on weapons or armour/evasion on other stuff

[grave wren]

Repoe has that data if you need it

https://github.com/brather1ng/RePoE

[empty venture]

but then having to join the two together, such a pain

I ended up starting a headless PoB and getting items that way

[grave wren]

It'd be nice if we had a version based unique repository too

[golden bane]

Maybe this is something my library should do 🤔

[grave wren]

Wanted to make a tool to quickly compare patch notes as static site but parsing unique data per patch is kinda annoying

[golden bane]

PoB's items are versioned, is that not good enough?

[empty venture]

I'm kinda worried that my approach won't work on an actual server, haven't test it yet :x

[grave wren]

POB has no internal names so getting images is pain

Are gems versioned as well in pob btw?

[golden bane]

No, and for uniques only legacy versions are kept. It's not a time machine, so it might not fit your purpose that well

[empty venture]

there is some purpose of legacy uniques, not so much for gems

(standard league)

[grave wren]

Yeah unsure if it's even needed tbh gotta think about what I want to display a bit more before deciding

[empty venture]

u can get unique name -> internal id mapping from the Bundle

if that is what you're looking for

[grave wren]

Yeah it would just be nice not having to go there as well. I am also not sure if we can still use the CDN from Poe or if I'd need to selfhost the assets, do we have any official infos?

[empty venture]

I believe someone said they are moving from "easy urls" to "urls with hash"

you will have to use url provided by them

won't be able to create url based on internal data

[grave wren]

Oof

[worthy cape]

#tooldev-general message

(previous screenshot was misleading, the digits are of unknown source)

[grave wren]

Cool thanks for digging for it

[simple ravine]

https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-6/

You know they're serious about performance when they start talking about emitting smarter assembly/IL code and JITASM

[worthy cape]

twiddles some thumbs

With legacy GGPKs once indexed I'm getting somewhere around 5k DB inserts and 5k disk writes per second.

[simple ravine]

on spinning disks?

[worthy cape]

DB and target filesystem on SSDs, but both over network.

Source data is on harddrives over sshfs or NFS.

[simple ravine]

ok

I think I need to take a look at that code

and steal it for Unicode

that might speed up my dat reads

CreateReadOnlySpanFromNullTerminated employs a vectorized search for the null terminator, making it typically much faster than the open-coded manual loop.

mm love me some simd

[worthy cape]

Bravely assuming that you're reading legit data 🙂

[simple ravine]

I mean, if it's not legit, it's fine to crash

[worthy cape]

My context tend to be schema exploration and adaptation, so incorrectness is kind of assumed.

[simple ravine]

right, so this is where it's assumed the schema is defined and correct

I have a RawDatFile class which DatFile implements form, which can be used for exploratory things

I was thinking a little bit now I that I rewrote it... the fourth? time

this time, I think I might be happy with it

[worthy cape]

Progress bar of sorts:

Started ingesting below the topmost blob this evening.

[simple ravine]

1 pint per bar?

[worthy cape]

^_^

[simple ravine]

shaving nanoseconds hmm

it's getting hard

[vapid pulsar]

Not sure about C#'s capability but maybe there is some SIMD instructions that could be used for speeding things up

I've seen this for speedy utf8 strings https://dirtyhandscoding.github.io/posts/utf8lut-vectorized-utf-8-converter-decoding-utf-8.html

[lyric umbra]

is there an item list for this game in text format or something?

having to grab everything individually from the wiki is a nightmare

[rigid oxide]

Hey, does anyone know what is the typical response time for [e-mail] these days? Dropped a message there yesterday and am eager to be able to do requests to do some real development. I understand if there's not a lot of time for manual application review, but would love to know what to expect at least

[rapid pagoda]

What kind of items? Base item types, specific types of gear or currency, unique items, something else...?

[lyric umbra]

unique items in this case

[frank drift]

was pretty quick a few months ago but who knows, GGG could be pretty busy

I think Novynn is still doing those, he reads this channel once in awhile

[rapid pagoda]

What information do you need about the items? Just names or more?

[rigid oxide]

Thanks, guess I'll poke some existing tools in the meantime

[velvet fog]

in patch 3.15.2, Baranite Thaumaturge reduced the maximum cast range for long-distance skills
compare MonsterVarieties found nothing, but in skill spell maximum action distance +% [-40]
https://poedb.tw/us/Baranite_Thaumaturge
seems they use skill range as monster aggro range

[rapid pagoda]

That'd make sense. The "sniper attacks" were all from monsters using skills at long range, not monsters noticing you and walking into range to use skills

[velvet fog]

I wonder how to implement this in mysql: spawn_weights.tag in ('armour', 'gloves')
tags weights example:

gloves 1000
default 0

[swift beacon]

Got videos working over the last couple of days. When selected, it extracts the .bk2 and then converts it to .mp4 so it can be played.

So now I have all "viewable" file types finished and able to be displayed (except for that one pesky .xls file).

[worthy cape]

I'm in the worst place to be, code is fast enough to make forward progress, but slow enough to make me wait... file ingest progress starting yesterday dinner to the right of the blob on the left:

[rapid pagoda]

JSON array in a string field. You can use https://dev.mysql.com/doc/refman/8.0/en/json-functions.html to manipulate it

[rapid pagoda]

lol, I'm pretty sure that one snuck in by mistake 🙂 there's been a few other spreadsheets in older versions

[velvet fog]

it's ordered compare, can't directly search tag
ex: input tags armour, gloves
compare list:

armor 0
gloves 1000

and

gloves 1000
armor 0

first will return 0, and second return 1000

[rapid pagoda]

Oh, that is tricky. You might be able to do something with JSON_TABLE() but it might be easier to just do the heavy lifting in PHP (or whatever)

[velvet fog]

just wondering how wiki's cargo can easily do that

[rapid pagoda]

oh yeah, btw, so I did this https://github.com/oriath-net/pogo/commit/a23de8f56a0ca11ff5f3ca1874f6876f1a281eb9

I enjoy editing it less, but maybe other people will~

(also holy crap that diff stat!)

[worthy cape]

Looks like you're a highly productive developer 😉

[worthy cape]

Ran this query on my partial set of files:

SELECT substring(filename FROM '\.[^.]*$') AS ext, COUNT(filename)--, ARRAY_AGG(filename)
FROM (
    SELECT DISTINCT filename FROM pack_entry
) AS dist_files
GROUP BY ext;

https://gist.github.com/zao/8eeb0df482b0d25068a7bddc27f69ff1

Good case hygiene, only .BK2 is wrongly cased thus far.

I wonder what the .bat files are for 😄

I'm starting to regret this megatable, it's taking a literal forever to query against.

Metadata/Terrain/Tools/Batches/TidyTiles.bat
Metadata/Terrain/Tools/Batches/TidyTilesInFolders.bat

[rapid pagoda]

fun tip: create an index on REVERSE(filename)

then you can search REVERSE(filename) LIKE REVERSE('%.bat') 😄

[worthy cape]

Seems to be an 1.0.0 remnant, but my dataset is still quite limited:
https://gist.github.com/zao/feadfc8cc3ce3afa1c90961b8ea15c56

Super elegant file access: ^_^

zao@poe-serve:~/mnt/contents/data$ less 5a0/5a0ed395a0fcde022baef326c13434852b530f22822f52154fe485a7f00264e4
zao@poe-serve:~/mnt/contents/data$ less bed/bedf9e4ea90a4e48b9ccf38127db75f4abf3a173434abb9745657683557fd703

[simple ravine]

Yeah, the internals are vectorized nowadays already for things like span.IndexOf at least for Span<byte> and Span<char> IIRC

[grave wren]

whenever your database is ready i expect it to randomly bring out dates instead of data @worthy cape : https://journals.plos.org/ploscompbiol/article?id=10.1371%2Fjournal.pcbi.1008984

What is not funny though is having genomics spreadsheets suffer from auto-conversion of gene names like SEPT8, DEC1 and MARCH3 into dates, a problem first characterised in 2004.

[simple ravine]

That is one random problem

[worthy cape]

Postgres assumedly is slightly better than Excel.

[grave wren]

too good, i need my 30% error rate

pls add

[simple ravine]

mssql ❤️

[worthy cape]

Best I can do is probably incompetent ingestion.

[grave wren]

fine but it better be really incompetent

[rapid pagoda]

strange question time: can you hide pantheon souls with a loot filter?

[broken cloud]

The Hide loot filter command does not work on quest items (just tried with a watchstone)... but if you really want them hidden, you can set them to completely transparent (SetTextColor|SetBorderColor|SetBackgroundColor 0 0 0 0)

[rapid pagoda]

Yeah, that's one of the other item classes with that flag set 🙂

[wooden magnet]

Hello GGG and fellow Exiles!

I've got a small suggestion.

**make it so that you can choose your loot filters directory. **
Currently I am travelling a lot and on this work laptop that I'm using I need to access a vpn to access the folder "My Documents" where my lootfilter is located. and by connecting to said vpn i get 300ms ingame

So currently I've got two options, either play with 300 ping, or play without a loot filter.

Using the sync filter will not work either as that saves a temporary file in the documents folder when you're using that filter.

[worthy cape]

While a neat idea, it's unfortunately nothing that a third party developer (like us) can work around.

[frank drift]

wrong place sadly, you need to send that to GGG

or to your employer

I'd be interested to hear what they think about you playing PoE on a locked down work laptop

[worthy cape]

If you have enough rights to redirect your Documents folder, you could do that. If not, welp, separate user profile for gaming?

I haven't tried it on interactive games, but I've got a local (admin) user toor on my machine which I use to install badly behaving software.

[wooden magnet]

I do not have them rights as that's the map we use for our cloud servers aswell

[worthy cape]

(for a while, probably still, AMD's drivers refuse to install if you've got any user profile directories redirected, so I had to cheese it)

[wooden magnet]

I need to be physically connected to the vpn to have rights to access that folder

@frank drift Sorry, didn't think of going to the website, silly me.

[worthy cape]

Something like add a new user to the system, Shift-Rightclick the program and "Run as different user". Super convenient when you need to test things with a fresher profile.

UI isolation may screw with you if you run trade tools, not sure how it behaves between multiple same-privilege users on a system.

AMD's drivers are amazing. They block installation if the current user has redirected any directory, including Downloads, as some of their runtime software might want to record into some profile dir like Videos 😄

[simple ravine]

NativeAOT compiled .net 6 console app:

that's actually pretty solid

[civic crane]

@velvet fog some inprogress items leaking 😅

[long sky]

You should be able to see the Cargo query on the wiki that does it, right?

[velvet fog]

I saw cargo code, but dont understand how it work. That should be ordered list for tags, use simple json query cant archive

[rapid pagoda]

don't miss ItemClasses 62, 63, and 64 🙂

[rapid pagoda]

finally got this interface working again 🙂

[rapid pagoda]

Huh. Well, this is an unexpected discovery. The Tencent client uses a v2 Content.ggpk... with bundles

[simple ravine]

v2 as in yet another format to reverse engineer?

[worthy cape]

Nah.

The old 1.0.0 to tech patch GGPK format was version "2". Everyone thought it was a chunk count.

Tech patch changed that number to 3, and I forget if macOS is 4 or not.

Quite convenient for tooling as you could just read like 12 octets from the file to tell if it was a legacy or bundle GGPK.

[simple ravine]

ah

[rapid pagoda]

Alas, NPCTextAudio doesn't contain transcriptions of the talking pet dialogue, even in the Chinese data files. And so my quest ends for today

[rapid pagoda]

...man, the Chinese i18n team really had a field day removing unnecessary blood effects from everything

"How do you take the blood out of a Blood Elemental?!"

[worthy cape]

Challenge accepted 😄

[simple ravine]

Crimson Elemental?

[rapid pagoda]

I can't tell exactly what it'd look like, but there's a bunch of .pet (particle emitter?) files for various gory effects which are empty or much shorter in the Tencent Content.ggpk

[grave wren]

Beer elemental

[rapid pagoda]

… someone in croatia is trying to download all the audio files off oriath.net one by one

my dude, you can literally just download the game and have them for yourself. there's no need to hammer my server to get them all

[golden bane]

Slip them in some fake/corrupted ones

[rapid pagoda]

too much effort, just IP banned them

[worthy cape]

Sounds like a croative person.

[bronze kestrel]

They’re using the unreleased PoE hardmode irl

[worthy cape]

Gah, the reverted builds keep ruining my neatly ordered history:

id,data_manifest,time_updated,win_manifest,mac_manifest,change_number
3901890,5332233849470628839,1560065390,,,6396747
3900167,855611360925272798,1559980247,,,6394243
3900326,4218960254236259784,1559979490,,,6394226
3900167,855611360925272798,1559965297,,,6393763
3900024,1202362578451579012,1559960553,,,6393531

[simple ravine]

Figuring out a very efficient way of generally working with these pathreps etc is enervating

[worthy cape]

I'm starting to like this DB schema tho.

Like 170ms to query for all filenames that are contained directly in Data and have extension '.dat':

SELECT id, full_path FROM path WHERE parents[1] = 'Data' AND extension = 'dat';

[simple ravine]

There needs to be some kind of inferred "rule" to how this is set-up etc, but I'm not sure

[worthy cape]

What about it is confusing, apart from how Art has the weird recursion bit?

[simple ravine]

It doesn't seem to be sorted by its tree structure

I want to efficiently create a lazy-loaded tree structure that's fairly quick to navigate

recursive_size would go all the levels down, and not just 1 level down, right?

[worthy cape]

Right, that's what it does everywhere else afaik.

[simple ravine]

so there are path_reps that are represented in the list, that also is represented if you use the recursive_size

[worthy cape]

Ooh, I finally found out why I had more GGPKs than extracted builds.
Some of the GGPKs are from the awakeningbeta branch, not public.

Been wondering if I had some subtle error somewhere, but nah, just over-restrictive soup sieving.

[empty venture]

A quick question to PoB guys: is it intentional that nameSpec is sometimes empty for skill gems? In the export string/xml.

It seems to be the case for item-provided skills. However, that value gets populated when loading/importing a build. It seems to be empty only just after adding such item to a build.

[velvet fog]

The Quest Rewards description in patchnote is confusing

Storm Burst, Storm Brand, and Wintertide Brand are no longer offered to the Witch for completing The Siren's Cadence.
Is it also applied the purchase from vendor?

The old and new patchnote about quest rewards are quite different

[lyric umbra]

im pretty sure there is something extremely scummy going on with certain item indexers in the game

remember when they added a 5 min delay to the stash tab API unless you're whitelisted? well poe.trade sure is sloooow in indexing things and people are getting messaged literally 1 second after they put their items up for sale

the only conclusion I can draw is that the site owners got it whitelisted, delayed the results for everyone else and are sniping all the items themselves

[worthy cape]

I seem to have misplaced my roll of tinfoil...

[lyric umbra]

I guess that's what happens if you're too fucking lazy to develop a good trade interface and let the community do all the work for you

call it tinfoil, it would be exactly what I would do if I ran the site

[empty venture]

u sure those messages aren't coming from the official trade site?

[lyric umbra]

yes because said items arent even appearing on official yet

they're already gone by the time you message, for minutes

[empty venture]

anyway, why come to this channel with that?

[lyric umbra]

mainly because I've been working on my own indexer to beat these fucks, but I see now that it's a futile battle

i wasnt even aware of the artificial delays because I see everyone always hate on private indexers because they're so fast, yet that's not my experience whatsoever, until I did some testing and found very odd results

I mean you can test this right now. put an item up for sale, see how long it takes between official and the 'whitelisted' poe.trade

[empty venture]

well, I can't

[worthy cape]

I never got around automating my trade delay checker, and it was largely irrelevant once the big delays were solved back then.

[lyric umbra]

scam link, hammer please

[worthy cape]

Auto-handled, no need.

Server admins are actual magicians.

(or very keen-eyed)

[civic crane]

official trade shows item as soon as I change zone, while poe.trade showed my item after 5 mins. It shows listed 7 mins ago, so poe.trade is still whitelisted by ggg and it's just an artificial delay, and also means poe.trade isn't lagging rn

[hazy fiber]

iirc the owner is XYZ and he doesnt even have a character in expedition?

nor could I see him doing that, or giving others the ability to do that

its very possible its just trade being weird

becouse im pretty sure the official trade site indexes it first, before even poetrade etc get it with their "no artificial delay"

[compact isle]

https://github.com/grindinggear/skilltree-export/tree/royale 👀

[pseudo ocean]

Hey i've been getting 403 on the public-stash-api since like the 9th of august

It ran like non-stop for months on end without issues, but now peeps keep copmlaining at me my ladder doesn't work 😦

[gleaming monolith]

I see discussions around online status no longer being shared in the trade api, but was it also removed from the ladder api?

[velvet fog]

both csv export and ladder api removed online status

[candid ridge]

@velvet fog crafting bench page seems broken https://poedb.tw/us/Crafting_Bench

used to be a helpful mod of bench crafts now it's an entry for the crafting bench hideout object

used to be able to go to https://poedb.tw/us/CraftingBench, but it's redirecting to the above url

[worthy cape]

I've always done https://poedb.tw/us/CraftingBenchOptions

[candid ridge]

that doesn't work either right?

yeah

[worthy cape]

https://poedb.tw/us/Crafting_Bench#CraftingBench is a good link these days.

[candid ridge]

oh it's the second tab

[frank drift]

yep

[pseudo ocean]

test

[velvet fog]

moved to first tab

[grave wren]

Discord py is no longer maintained interesting read: https://gist.github.com/Rapptz/4a2f62751b9600a31a0d3c78100287f1

[simple ravine]

Halfway through this, and I think there are... other... companies who can learn from this.

[frank drift]

not sure what there is to learn

Discord thinks they're big enough they can force devs to play by their rules and they're almost certainly correct

[simple ravine]

did you read the thing?

[frank drift]

awhile ago

of course the maintainer feels that things will be catastrophic, they are catastrophic from his view

[simple ravine]

I'll read through everything over breakfast before I say more I guess

[frank drift]

Discord probably won't even notice

not saying Discord hasn't been the typical dumb social media startup its entire existence

[simple ravine]

alright, let me go make some breakfast and get some caffeine, this was longer than I thought

[frank drift]

but they've scaled to the point where they don't have to pretend to care about 3rd party devs

[simple ravine]

well, this does in fact sound a little bit messy

of course it's their own platform, they do as they wish, but from what i gather, a bot will not be able to read the contents of messages as previously done

[bronze kestrel]

Won’t that just shift to discord’s privileged message access meaning the bot once approved will be able to read messages via their own api and only for a limited amount of servers?

[simple ravine]

Not sure, I've not followed the discussion

[bronze kestrel]

Everything else will have to go through the anal bureaucratic slog that is discord’s approval process

Still. Sad that he quit. Hope someone forks it. It’s a nice lib.

[grave wren]

sometimes i wish more people would just throw in the towel

so companies feel the issues with just shitting on people providing lots of utilitiy

[frank drift]

like I said, good for him, but doubt Discord feels anything

[grave wren]

which is not the point

[simple ravine]

What I question a little bit is the favoritism and not listening to feedback. I do do understand that they need to take privacy very seriously as a growing platform for user generated content, especially considering a lot of users on Discord are juvenile.

[bronze kestrel]

There’s always two sides to a story. Without being on both sides of the fence and personally witnessing the convo and reasoning behind changes - all you’re left with is speculation. Sure, sometimes companies do stuff in misguided ways that hurt the community. But sometimes what they do might be required for one reason or another and the community refuse to accept it on the basis of “big company just being a dick”.

¯_(ツ)_/¯

[grave wren]

which is also not my point, but if a company relied heavily on the community and seemingly disregards their input sometimes it wouldnt hurt to adjust completely

it's good that they made their point by saying thats it for me

[bronze kestrel]

They relied on it until they gained momentum. After that it’s an anchor weighing their future decisions down

Seemingly

I’m not for it or against it. I’m just sad he quit. But totally understand why.

[grave wren]

same, i liked discord.py

it was a better tool to integrate with than many other libs i use professionally

[worthy cape]

Reminds me a bit of the fall of the Twitter API.

[grave wren]

and thats from a person that's not even a coder by trade

[bronze kestrel]

I’m currently making a bot using it. Makes me wonder how soon will my bot fall over and need a refactor. A bit annoying.

[grave wren]

it was pretty stable for the last 2+ years for me

[worthy cape]

For Twitter third party clients and interactions are great until they're not, then the intended developer demographic is metrics and advertisements, you know, things that directly make them money.

[grave wren]

but we'll see

oh, @regal mural is 3 years old already

oof

[simple ravine]

Twitter licensing Firehose to a single entity is a bit awkward

[bronze kestrel]

If they just gave us a secure api that’s not something we have to wrap a client lib over and it works as advertised it would solve lots of issues. But from what little I heard discord lacks that in spades.

[simple ravine]

eventhough it's their full right to do whatever they want, it smells really bad

[grave wren]

i actually really enjoy the client libs if done right such as discord.py

[bronze kestrel]

Saves us work in the end. A bit anyway.

[grave wren]

depending on the api sometimes even a lot 😄

it's amazing how bad the average api is i use

[simple ravine]

When it comes to bots reading contents of messages, is this only for bots that join 100+ guilds, or is it for all bots?

Because that seems to be the big problem

[grave wren]

from reading it seemed like for now its just for the big bots

but maybe i missed something

[simple ravine]

Something is happening April 2022

are all bots going to require some kind of "ok" from Discord to read contents by then?

[grave wren]

maybe i'll finally have a reason to learn go (if that client lib works)

[simple ravine]

I mean, moderation bots is a big chunk (I am imagining) of what bots do

by bot that I made, scans all messages for a PoB link and spits out a summary

[grave wren]

its only for verified bots > 100 guilds

[simple ravine]

it'd be inconvenient to make a slash command for that

[worthy cape]

Haven't read up on it, but feels like they kind of want to limit interaction to call-response /roll 2d8 types of commands.

[grave wren]

i'd link the article but i cant find a way to change language

[worthy cape]

Less arbitrary consumption, more toys.

[simple ravine]

Huge communities need automated moderation though

[grave wren]

https://support-dev.discord.com/hc/en-us/articles/4404772028055

there we go

In April of 2022, access to message content will become a Privileged Intent—like presence and guild member data—for developers building or managing verified Discord bots and apps.

This change affects only verified bots, bots that are in 100 or more servers. Unverified bots in fewer than 100 servers are not affected at all.

[worthy cape]

That would mean accepting that there's a moderation problem and that the built-in tools for it are inadequate 😉

[simple ravine]

Well, different guilds have different needs

[grave wren]

there is no moderation problem, all is well on the discord front

[worthy cape]

There are no bad actors in Ba Sing Se.

[grave wren]

welp thought i'd fix a bug in my bot but turns out pob doesnt export minion accu

[simple ravine]

Reminds me, calculation engine library

it's equally compelling as it is daunting

[grave wren]

would be fun but yeah definitely not starting one up :p

[simple ravine]

I'd start one up, but I'd want to have some interest and commitment to a community contribution and drive

[grave wren]

oh fun, crit mods also miss on the xml

i think the big issue with any calc engine is to even start with a community

i dont think we'd have pob if we tried a community one at the start

[simple ravine]

Well, given the amount of interesting applications that we would have if we had an open and more approachable library that could calculate all the aspects of a character etc..

[grave wren]

absolutely, but i can only imagine the fun of deciding on a common starting point

[simple ravine]

If you don't try... 🙂

[golden bane]

What you intend to do here is basically to rewrite PoB. Currently, I'm still working on a new exporter, but after that... we'll see 😉

[wicked patrol]

Hey guys sorry i might be in the wrong room i need help with the chaos recipe enhancer can anyone help with that? or point me in the right direction??

[simple ravine]

Yes, and no... My intention is to make something that can be more used more broadly. It would enable applications similar to PoB to be developed with the use of it

[wicked patrol]

i was told to go to this group to ask for help with the issue im having. is this the right group or not, sorry for the trouble i am just looking for help

[empty venture]

"PoE tool development discussion. NOT IN GAME DEBUGGING. Not how to use tools" - I guess its a wrong room

[wicked patrol]

thanks sorry for the trouble

[golden bane]

What I mean is that you still need to put in a comparable amount of effort. I understand that you can do more with it when it's composable

[empty venture]

How many ppl would want to make a tool similar to PoB tho? Even with some calculation library?

[simple ravine]

Right. With comparable, I'd be brave enough to say that one shouldn't take the aggregated amount of effort from inception to now, as iterations and refactorings has happened, and so on. It wouldn't be a simple feat of course.

A lot of knowledge, understanding and effort has been poured into the PoB project, and it's an unbelievably important tool that should continue to thrive.

[golden bane]

If you truly want to create such a calculations library yourself, I'd suggest contributing to PoB first to get a better feel for the problem space

[simple ravine]

Well, that'd require me to understand the architecture of PoB, after learning enough Lua.

[golden bane]

What's the alternative though? Would you want to start completely fresh and ignore the work that went into PoB? That doesn't seem wise

[simple ravine]

Well, it's one thing to read and trying to understand the code, and another becoming good enough to contribute back to it.

[worthy cape]

It's warty and structures computations the way it does for reasons. Knowing those helps when designing an improvement.

In particular, what benefits and drawbacks is there to the table+flag kind of setup it has for mods and what applies when, how does it balance between parsing and having more logic and checkboxes to get the Correct Behaviour, how does it handle future changes?

[golden bane]

I'd say that if you don't know the code well enough to contribute, you also don't know it well enough to rewrite or adapt it. I'd even go as far as to say that you'd need a better understanding of it

I know I'm not entirely impartial because I'd like to have more regular contributors. However, I think contributing to PoB is a great first step for any future work in that problem space

[grave wren]

i find dealing with lua too annoying to do more than what i did so far, fighting against the language and learning the codebase isnt fun in combination

[simple ravine]

I understand where you're coming from, and I am not intending to compete with PoB. I'll take it into consideration. I think however doing Lua is not what I would enjoy. If I don't enjoy it, then there's pretty much no reason left.

[worthy cape]

Point kind of being that it's hard to design some "computation engine" without quite knowing the boundaries of what to compute - like how many actors, what kind of scalars there are, everything that's codified in the best implementation there is currently.