#tooldev-general
1 messages ยท Page 86 of 1
youre misinterpreting the colors
orange is for urneachable, yellow is for reachable reward nodes
the benefit of having the button is that it artificially adds depth to the tool
im just following ggg design logic
looks good
debatable
i think im onto something here but nobody is interested in this stupid mechanic
https://i.imgur.com/p8dgaMH.png im going to add this
and make the assumption toggleable
since finding conclusive information is next to impossible
actually theres nothing needed to code, this is the most efficient blocking method hands down: https://zedor.github.io/poeNexusBlocker/?00A00B0B00B000B00B0B0B0B00B0B000B0B00B0B00000B0B0090000000000700B0B00000B0B00B0B000B0B00B0B0B0B00B000B00B0B00800100
only the average path needs to be recalculated, but its 1.25 actually
its impossible to brick if you always take care of your fragments
you can get away with it with only 8 4way nodes aswell
intredasting
ps4 don't have public stash api
but it has the forum
anyone knows for how long the item lingers in the trade api after it's gone?
this one is a bit more balanced on the types of fragments used to do the blocking
alot of options though to accomplish that layout
so i set it up
actually i just finished collecting t16 pieces for the mem farm but
i tried the ultra block before and it takes me 3-4 fragment completions to spawn a reward
and for some reason i havent gotten a reward while farming mem fragments for a long time
so it currently stands at 39/40
and there are still plenty of places that should be able to spawn rewards
specifically, 6 more
pardon me, 5
so im going to spam some more quarries to see
what the heck is going on
(i placed the 3 pieces around nexus to see if it would speed up reward spawning)
Does the ladder api have a way to sort by depth?
@covert reef you can use sort=depth or sort=depthsolo in the query url
Thanks a lot
how to install pypoe? different pages show different command lines:
pip3 install -e .[full]
pip install -e .[full]
- what are pip and pip3? also it says ".full should either be a path to a local project or a vcs(...)"
pip is a Python package manager. https://packaging.python.org/tutorials/installing-packages/
got it, my bad. i thought brackets meant "optional key", but it was just a symbol. "pip install -e .[full]" worked
is there a way to export data from "BaseItemTypes.dat". I want to get file with something like
(item name) (item width) (item height) (item icon)
. is there a way to write something like sql query for this or smth?
@chrome topaz a chinese POB fucked with my bot, after fixing i noticed your pob.party also does not accept it and returns a 500: https://pastebin.com/GLBmDf9w
sent payload: https://gist.github.com/FWidm/25f0297e13fb5877d20badb9f70c8164
where can i find web.poecdn.com link to .png file ?
for example, for vaal regalia it should be "https://web.poecdn.com/image/Art/2DItems/Armours/BodyArmours/BodyStrDexInt1C.png"/ But i don't see anything similar in BaseItemTypes.dat
@hazy fog something like it setting the encoding to windows made it work
@storm wedge i know that's hardly an ideal solution, but you could make a search and look at Network tab in the browser console
and then filter by "art"
actually no, there is a better way
@timid crest thanks, i managed this by doing two things:
- downloaading base_items.json from repoe
- writing script to "parse" this json
oh i see, its in
"visual_identity": {
"dds_file": "Art/2DItems/Armours/BodyArmours/BodyStrDexInt1C.dds",
"id": "BodyStrDexInt1C"
}
how was this json made btw? when i used pypoe json export i got different result
or this is the exact reason why REpoe was made?
but there is also yet another way btw, you can get items of one category from https://poe.watch/api with get
and then get the icon field from there
hm ,interesting site.
another question - is there description anywhere of possible keys for web.poecdn.com link? like scale=1 etc.
๐คท
as i see, scale=0 ignores h and w parameters
@grave wren does it work with normal pob? I'm just wondering if I fix it and it turns out it's not even supported
Didn't try it I just noticed the encoding issue. Can try in half an hour or so and report back
the chinese pob seems to load yes
Does anyone know if the poe.watch dev hangs out here?
@deft jolt
yes that would be me @frigid nova
@compact isle What is the best place to give feedback for the trade website?
the forums
Thanks!
press escape, graphics options -> display mode -> fullscreen to windowed fullscreen
(or regular windowed mode)
@hazy fog this lad above needs some help
semi-related to poe (more about just strings parsing) question: how can i "pull" item base from magical item name?
template is: (prefix) (base) (suffix)
suffixes are easy - they starts from "of "
but how can i tell if item had prefix at the first place?
You can find all the names of all the mods here: https://raw.githubusercontent.com/brather1ng/RePoE/master/data/mods.json
in the "name" field
yeah that's what I was going to say, you can't just know unless you have a lookup list of one or the other to compare against
thanks
def find_longest_substring(string, substring_list):
contained_substrings = [substring for substring in substring_list if substring in string]
assert len(contained_substrings) > 0, "no substring found"
return contained_substrings.sort(key=len, reverse=True)[0]```
Undoubtedly not the fastest way, but there are few enough base items that it shouldn't matter
Yes fair, I assume that there isnt inclusion, which is incorrect. In that case you can just return the largest found substring
@earnest radish fixed my post
Could do some better error handling to detect when there are two substrings which are both in the string and neither strictly contained in the other
Any news on an official oauth registration system? If there's going to be a turmoil/mayhem race next month I'd love to integrate oauth instead of using an IP system + google recaptcha to filter out bad actors from my zone tracker
just message @compact isle if you require basic user auth
maybe he doesn't allow pms from non registered discord accounts? maybe phone 2fa required? I dunno
i'll try 2 factor authing real quick then
@polar island thanks and will do
@compact isle whenever you're available, please message me so I can talk about possibly getting Oauth for my site
anyone know why i cant place memory fragments?
@coral moon try asking in #1โgeneral, this isn't a 3rd party tool question
any folks with hideout sites here's the default haunted hideout export
https://zedor.github.io/poeNexusBlocker/
how much should I contain the capabilities of the tool? should I look to account for challenge completion (length of path, connect to) and various similar things (like spawn weights, things that are harder to test and get proper sample sizes) basically expanding it into a nexus simulator, or keep it relatively simple as-is?
or is that a matter of personal preference
im adding last spawn rules that i tested for, both reward and boss related, and visualization of boss spawns, but then all it comes down to is code cleanup and optimization so it looks nicer
im afraid of going in to things that are much more complex while having dubious returns (nexus blocking itself has yet to be tested for efficiency anyways so effectiveness of the tool is limited)
I'm looking to take a crack and handcrafting a model to predict watcher's eyes prices. Does anyone here have all of the data for watcher's eyes posted since the start of Synthesis league?
If not does anyone know of an open source project which parses the river which I could set up on a vps to grab the data myself?
Ideally I'd like all watchers eyes that have shown up for sale at any point in the league, as the one's that have disappeared are a much better representative of price than the ones that haven't sold yet
fair, do you know of some lightweight river parser I can use to grab it myself?
Do you have a concept of how much bandwidth it would take to parse the river from the start of Synthesis League?
(order of magnitude)
I see, that's a shame.
@frigid nova 348573750-360714321-340380017-391278531-368564817 from 2019-03-08T21:22:19Z
(via the glorious search mechanism this channel has ๐ )
@worthy cape I appreciate it but it would seem I don't need it because the river doesn't contain sufficient data for what I'm looking for
Can anyone post ctrl-c data of item with both enchant and implicits? Trying to parse ctrlc data into object with javascript
you can get that from official trade site
there is a 'copy item' button at left coner
@earnest radish I believe he just took the data from the current listed items through the trade api
If anyone who actively parses the river would be interested in starting a small side project with me to do some hand tailored models for small market price prediction hit me up. We would start with something like watcher's eyes and track all listings throughout the course of a league to gather enough data to make a reliable model.
@queen oriole feel free to flick an email to roryv@grindinggear.com with your details / requirements
is there anywhre in ggpk info "uniq item name -- icon image for this unique"?
cors.io is down. RIP me
Time to learn how to do that
But not right now probably cause it's getting late and I'm tired right now
Tomorrow will be fun though
don't sure the site name is?

well, if we don't have the site name, can't add it into blacklist.
@hazy fog Thanks I'll talk to my ad provider.
anyone by chance know why the ArcaneSurge GrantedEffect has an invalid/excluded ActiveSkillType of 32, even though it seems like most active skill gems with that type still works in game with it? Seems like the only thing in the GrantedEffect table that invalidates the 32 type, for now just manually excluding that type from support gem matching on arcane surge
ActiveSkillType of 32 is TriggerableSpell?
probably, it seems weird that it'd be in the invalid active skill types column for SupportArcaneSurge though
i've been using it where if that ActiveSkillType was in the ActiveSkills row, you couldn't link it with that skill
but it seems like that's not how it actually worked
[ 33, 36, 26, 37, 32, 74 ] is in my invalid/excluded skill types column for ActiveSkillType id SupportArcaneSurge
also, how do you normally track ActiveSkillType? there's no id in that dat file so any time they remove an index it completely shifts everything and I had to refigure out every type's new position. I used to have a mapping for each number to a definition but gave up when they refactored twice or something
You know, there is an open source application that does handle the skill type stuff properly, mainly because the person who wrote it (Hi! :D) also wrote the current version of that code in the game itself
I'll save you some time by pointing out this particular file:
The last 3 types should give you a hint as to what the heckaroni is going on in Arcane Surge's types
That explains how you figured those out so quickly ๐
๐
@timid hemlock thank you very much, I guess I'll need to pull from the github from now on
For using custom sounds in your filter, can i have multiple sounds for 1 item where there's a chance it will make this or that sound ?
Like it's not always the same but changes
No, that's not currently possible with the filter system
what is this for then ?
You could detect file access to the sound files by a program and repeatedly swap them though
You can have multiple sounds, but not for the same rule/item
hmm because i could of sworn that when using the mathil sounds every time i dropped a unique i would get different sounds, not always the same one
Must be different rules/categories of uniques then
wierd, so only 1 sound for 1 item
1 more question if i have a sound for a specific item and i have a sound for ''currency'' which sound will it choose?
(the specific item is a chaos orb for example)
If you'd like more variety, I'd suggest manually setting drop sounds for specific uniques or maybe unique item bases
Depends on what's on top in the filter file
i was more thinking of having different sounds for when i drop a chaos orb or alchemy
It works on a fall-though basis, the first match it finds gets chosen
okay let me go more in detail. So in filterblade you can choose a specific sound for an exalt for example, and then you can choose sound styles for frequently used sound and one of those are ''currency sounds'' which one of the sounds will it choose ?
this
so i have a specific sound for a chaos, or fusing but i also want to have a different sound for the rest of the currency items
Idk, download the filter, search for both in the text file and find out what's on top of the other
But I'd guess individual items go on top
I think you'd need to manually edit the filter for such advanced/complicated rules
ugh each item individually ?
Or make more styles/groups/whatever
The wiki has good docs on it: https://pathofexile.gamepedia.com/Item_filter
i guess this could be a cool feature in the future then
for the website
but thanks anyway
btw can i have spaces in between the cutomsound files?
like chaos orb.mp3
or does it have it be chaos_orb.mp3
Yes, but generally, having spaces in file names is just asking for trouble
i like causing trouble >:))
@fickle yew Poe.Ninja api for currency is returning garbage data
exalted orb listed at 85c on synthesis.
Get'em while they're hot!
looks like large scale price manipulation
hm apparently poe.ninja eats fractured mods @fickle yew
which also happens in the pobs from there
Fun. I'll take a look at both.
Anyone by chance know how the game evaluates which buff to choose if you have multiple instances of a non-stacking buff? ie, someone has a level 21 hatred but is supported by a necromancer with level 1 hatred, would both technically apply/be active (since the level 21 is higher but the necro buff gives attack speed and stuff)? I thought about just taking the highest stat value between the two buffs and making them both active, but then I realized that wouldn't work for certain buffs like Sapped where the lower value should technically take priority (stronger saps = lower value for sapped_damage_+%_final). also in game, it'll show only one buff at a time (and generally the stronger one in the case of stuff like Rallying Cry). Not sure if there's a flag in the Stat.dat table to possibly say "prioritize lower values for buffs" or something like that.
To my knowledge: For auras it picks the one with the highest level, regardless of values of the stat. I don't know how it works for debuffs like Sapped.
@frigid nova so that means if you had a higher level buff than your necromancer, you wouldn't get the attack/cast speed bonuses per aura for that aura right?
I believe so, ill test in game right now
Thanks, I don't have a good way to multibox and test all this stuff in pvp. Still trying to perfect my buff evaluation for a lot of stuff
As a side note: I have a google doc which has begun to reverse engineer the buff.dat flags if youre interested in helping with that.
Sure, I can take a look. I've worked of mostly PyPoe's list for BuffDefinitions but maybe something I have is modified
alright aura effect > level > higher stats (by order of stat keys in BuffDefinition) is probably what's happening, unless level downscaling changed everything
any link for current work?
@grave wren Reprocessing builds with fractured mod support now. So they should show up later today.. o r maybe tomorrow.. It's reprocessing 100k+ chars through pob..
Wow apparently 360k pob builds that it's recalcuating.. Zzz.
Just cloud all the things!
is openarl's list of mods here: https://github.com/Openarl/PathOfBuilding/blob/master/Data/3_0/ModItem.lua the complete list of mods that can show up on non-jewel equipment?
@long edge You can find all the data for the mods currently in the game here
https://raw.githubusercontent.com/brather1ng/RePoE/master/data/mods.json
thanks nick_
is it possible to parse enchant, implicits and explicits from ctrl+c item data without having ton of lists (like all helmets enchants list in all languages and so on?) i guess no, but maybe there are some tricks
@fickle yew would be interesting to know avg stats for builds like processing time - https://poe.ninja/challenge/builds/char/Shantom17/Landscape_Apeiria?i=1&search=skill%3DMirror-Arrow%26sort%3Denergy-shield seems to not yet be updated btw
@grave wren it should be done. It might be missing from the actual frontend of the website actually. But isn't it included in pob now?
That I can check in a bit. Gimme few minutes and I can check it.
pob code is updated ๐
Alright. I'll fix the frontend. That doesn't require a rebuild.
Thanks for testing @grave wren

is this the right place for noob ahk questions?
soo.. i wrote a macro that is supposed to invite the last player in chat on scrolling down my mouse wheel (just for testing and getting familiar with AHK). The problem is that it works on all windows except for POE.
and if i let it run only on poe window it doesn't work at all
is there some command i have to use to make it work on this window or something?
@fickle yew Got any chance to add curses to count as supports for bane on poe.ninja/builds?
@frigid nova Yeah I'll take a look. Been a bit busy lately ๐
No rush ๐
Vacation coming up though, hope I can catch up a bit.
someone here familiar with ahk programming?
@fickle yew not sure why, but I just noticed poe.ninja is.. eating crafted mods somehow
all of the mods shown on that bow are missing from the PoB import (it shows as a 2-mod bow), and they're also not factored in to the DPS calculation (16% of cold as extra chaos, but zero chaos in the DPS breakdown)
it needs to eat ๐
@grizzled seal already fixed. It's recalcing. Got lost the other day when I fixed fractured mods a bit too quickly.
anyone know what the ratelimits for websockets are on the trade website? @compact isle
Hello, is there any way to know if the rolls from an item are either good or not? Regardless of the tier and the iLvl, as an example with a ring, if I get 30 maximum life but I know the maximum explicit roll you can have is 50 (made up max, not sure if this is the reality), I could get a fitness which tells me this item is 60% of the best possible roll.
@cursive ferry i suppose you can check the max by consulting the mod list (and adding up hybrid mods aswell case youre also taking them into account and not specific explicit-related values)
checking https://raw.githubusercontent.com/brather1ng/RePoE/master/data/synthesis_implicits.json , is there a specific reason for the underscores ( _ ) appearing at the end of implicit names? for example "SynthesisImplicitLife4_" vs "SynthesisImplicitLife3"
@cursive ferry If you're trying to value something, you should note that mods are tiered and that you can spend divine orbs to reroll explicit values within a tier.
There's some latent intrinsic value to a low top-tier roll vs. a high roll on a lower tier.
Depending on the overall value of the thing of course, you don't divine a 3c ring, but you would probably divine the heck out of a few-mod 7exa chest.
i want to make file with each line:
<unique item name> <path to icon of this unique>
2 questions:
- where in ggpk can i find this data?
- is there an easy way to export ggpk data to text files like this? (atm i figured that i can select table in ggpk pypoe vggpk viever, select "copy" from context menu, then make new text file, paste there, then import this new text file as csv into mysql and run queries in mysql)
However, unique item 2D art seems to be in ROOT/Art/2DItems/
I guess it would be easier to use the wiki API instead though
thanks. are there at least item names in ggpk? idealy i want to have "translated" list to figure item icon url for non-english names (atm logic is: knowing uniq. name figure this uniq english name, knowing english name figure icon url) probably wiki has only english names, right?
Some are named, some are named differently than ingame, some are numbered ยฏ_(ใ)_/ยฏ
@frigid nova thanks, this looks like what i wanted
thanks @long edge and @worthy cape , I was thinking of implementing a feature to tell if a rare was worth to buy but the complexity to get the answer is kind of high
contrary to popular belief, there's just too many permutations, and there's not enough data to cover all these permutations to a statistically significant degree, I would say
especially considering you'd have to normalize, remove outliers etc, because the pricing data you have is solely going to be 'wanted price' and not 'sold for price'
anything that's based solely on API list prices will run into two big problems: 1) there's no way to distinguish between an item that's sold and an item that's removed because it didn't sell 2) items which do sell will disappear from the trade API quickly, items that don't sell will hang around longer
I don't think it's an issue of "processing power" per se -- more a matter of, we don't have the semantic information about what items actually get sold for what prices
hence my comments as well
but if you take a position of assuming the normalized pricing is to some degree true
you could infer a pricing model, especially when cross-matching different combinations
also, the trade API never sees items that get sold straight to vendor ๐
well, those aren't interesting anyway
so there's an implicit bias in the data
I think the second problem is solved by picking the market you want to analyze and keeping all data for a league. I think the first problem isn't that big of a deal, prediction can handle noisy data. The biggest worry, imo, is weeding out people intentionally abusing the data (unlisting and relisting the same item, and more sophisticated variants of that...)
I don't think the amount of data is that much of an issue -- you just need to build a model which isn't trying to generically analyze items naively living in a huge dimensional space indexed by all affixes. This is not outside of the realm of possibility.
Keep in mind many people price items based on the data on the market currently, clearly there is enough information there to derive somewhat accurate pricing schemes, regardless of whether items have sold or not.
the price structures come from combinations of modifiers with a synergy
and there are an increasing amount of modifiers added to the game, and the amount of possible combinations is getting pretty big
Undoubtedly, so trying to price all items in the game is daunting. But starting with small markets seems reasonable to me
this is what a human brain can do much more efficiently than a computer, making those assumptions
For example jewels should be very approachable - they have 4 total affixes and no crafted mods
and if you put your current knowledge into an algorithm, you're biasing it
Yes there is always a bias variance trade off isnt there
You are correct that there is not enough data to just black box ML these problems. But building a biased model based on preconceptions should require less training time, data ,and more accurate predictions.
again, the amount of permutations vs items on the market is likely not enough, especially considering it's a temporal problem as well
has anyone tried to hand craft a model for a small market?
prices shift quickly in a league, same with jewels
they are more valuable a bit later into the game, meanwhile quite cheap in the first couple of days
I don't think the temporal dimension is all that intimidating
right, i think you'll have to categorize things into specific synergy-categories
temporally speaking, very early on in a league prices are all over the place because the market is starved for currency
that could help a little bit, but far from solving the issue I think, considering there are combinations that pop up, and are deemed valuable due to new build ideas
preconceptions would miss all of those items
and that is where your algorithm to make a determination, as well
Are we really that discouraged from mispredicting new trends as they appear? Why not just worry about mean prediction error
well, I can't speak for anybody else, but I would not treat a model very useful if it wasn't catching things I couldn't catch myself
A live model will react live to meta shifts, and there are ways to take existing league data and transition into new league information.
meta changes with new leagues
like 3.7 will clearly likely have a bias towards melee for an example
I don't think that affects underlying prices nearly as much as people predict ,and like i said, you take old data as a starting point for the model and you heavily weight newly incorporated data.
that might invalidate quite a bit of the pricing structures from past leagues
The goal of the model is prediction not forecasting
and yeah, I feel like there is a lot of work to be done in terms of recognizing patterns of value on items
I mean, people have tried. I encourage people to try, but I am just stating my perspective of it
Has anyone tried with full data? not just currently listed river data
the one that's integrated with the AHK ItemInfo service... what's it called again?
A simple tool to price check your items in path of exile by copy and paste
and they've, based on my knowledge, are the ones who've gotten farthest
I would really love a crack at this with jewels, but I would need someone who is actively parsing the river to collect all data on jewels. I don't have the money to set up a server for a whole league just to extract jewel information.
in my experience, it's decent at telling you what items are totally worthless, but not very good at judging value of anything else
It doesn't seem like they comment on their methodology anywhere?
they have, they're segmenting the MLs on base-type etc
link?
don't have it handy
the common pricing strategy of "list everything at 5c and see if anyone bites" is likely to become a pain point too
even if you would come somewhat close with jewels, I think it'd become exponentially harder with a chest piece or a ring
yet again i think the key is only using data of things that have dissapeared (aka not using a snapshot of the current river)
you might be underestimating the number of affixes on jewels
have you seen the jewel DPS evaluator thing someone put together recently?
@frigid nova ok, sir
๐ฟ
jewels are a bit easier in that they're generally only used for EHP and damage
(rare jewels at least, uniques are another matter ofc)
i bet for a lot of basic rares you could combine the life+res value into some generic EHP value
is that interesting though?
you could easily hack an evaluation method to do that in a day
you wouldn't add much QoL with such app either way, as you still need to evaluate each item manually unless you don't care about the non-basic rares at all
maybe i'm wrong... maybe people are looking for a thing that tells them "hey, this is worth more selling than vendoring, ship it in your 5c tab"
the solution to true price prediction is very far from that, technically, so you wouldn't be "on your way to a better solution" when developing such solution
hence my skepticism
Perhaps one day we can gather the appropriate data, set up some hand-tailored models and see. Until then its just speculation and me getting overly worked up about it ๐
@earnest radish Yeah, that's the one!
has anyone tried learning people, not items? i mean, if there is someone who puts many expensive items that disappear later (at least for a while so you can consider them sold, being used in some build and relisted by another person later), you can consider this player as experienced. and then you can parse all his listings to figure meta combinations etc. good thing that this "experience player detector" can be carryied from earlier leagues, right?
As soon as the experienced player finds out they are being tracked, this would be wide open to manipulation. Consider that experienced players that craft for a profit are the most likely to manipulate the market in their favour.
- how can they know?
- "experience player" (in the way i used this word) is not label that will follow someone forever. but rather something like flag "this player put most expensive items in the past day, what did he do before?" it's more about the past, not about the future. kind of narrowing set of items that can be used as example
that's just some thoughts, idk how useful they can be
however even if this idea will perfectly work, it can help only with top tier items (since sample criteris was "high price" in the first place)
is there anywhere a list of possible data you can get from official site?
so far i know about
pathofexile.com/api/trade/data/stats
pathofexile.com/api/trade/data/items
Something I wish I could do to price a rare item (=> well, more like knowing if an item is useful or not) quit well would be to (somehow) use PoB with the templates from the top players your can find on poe.ninja. You could then "easily" tell if the item has a significant impact on the popular builds. At least it would make you quickly know if an item is either useless or useful I think. I really like poe.price but I wish there was a client side (some library anyone could run on their own computer) that will tell you quickly (matter of ms) is a item is useful or not
interesting idea but how do you decide which are the top build and which are outliers
clustering on the leaderboards I suppose
isn't the idea to replace a given item with your item to see how it compares? then you need the rest of the items, how do you then deicde just which build to take the items from?
everyone gets as much damage as they can while getting enough hp and attributes and resist on all their gear pieces you can't combine 20 builds and say well their average ring has 34 fire res 42 lightning et... etc
so, i'm trying to build list (array of arrays) of all item bases on different languages
idea is that, for example en[100] and de[100] would contain text for the same item base.
i parsed https://(langcode).pathofexile.com/api/trade/data/items
but found a problem - different languages have different array length.
specificaly - in german it seems that Saintly Chainmail and Holy Chainmail translated by the same word (Heiliger Kettenpanzer) - i checked by trade search: https://de.pathofexile.com/trade/search/Synthesis/eGeJwsL
https://de.pathofexile.com/trade/search/Synthesis/E3XzmZS5
in json file "Heiliger Kettenpanzer" only listed once. this means i can't match translations the way i thought (by index)
any ideas how can i get what i want?
that sounds almost like a bug to me
items json is weird. in all languages except russian there are 2 entries for Titucius' Span (306 and 307)...
i guess find out which specific items have the same translations and hardcode those ones?
yes, this is an option. i wanted to have something that builds this list automated though, to do this easily from time to time (every patch with new items)
that shield seems to have been renamed at some point
so most languages have both names
that explains, thanks. unfortunately, this means that every time any item will be renamed parsing should be checked again for possible new hardcoded exceptions...
what are blue and red nodes in Path of Building?
the colors appear when you select "show node power" -- red represents offensive power (boosts to DPS), blue represents defensive power (life, ES, armor/evasion/dodge/etc)
the brightness of the color corresponds with the strength of the effect
Ctrl-c info usualy has "blocks" delimited by - - - - - - - -. But for fractured items with no other explicits what should be "explicit" block is the part of previous one. Does ingame ctrl-c acts the same (i checked trade site)? Do you think this is a bug?
since there isn't any technical other than this ->
I just noticed something, all the gateways are now using cloudflare so using winMTR isn't possible right now (to check your routing towards the actual gateway)
If you try and get an ip from ingame, it will just show you an proxy ip
so how are the users suppose to find out if the routing is their issue (ISP) or GGG now?
ยฏ_(ใ)_/ยฏ
trade-off though, between being able to handle DDoS attacks, or having non-critical features
yeah, but they should setup some dummy ip's with a different subnet but same location so people can still figure out their routing issues
that still opens up attack vectors
and wouldn't be representative of the actual load or up/down-status of the realms
It will(would?) still be representative of the routing to the datacenter.
Yes, but a misrepresentation for the vast majority of users ,who don't even know what routing means
Even if Amsterdam is closer to me, Frankfurt is normally better suited because of load
Frankfurt is closer to me but Milan is was nicer 
don't think he is
there's a couple, yeah
One is called Exiled Presence and the other has a bit more generic name
Heya guys, any developers here who wouldn't mind me running some silly question by? General, any development. Game. Web. Software. I've got some questions I'd like to discuss re: setting up a DB that would be queryable based on an objects attributes. similarly to how PoB figures out, based on equipment, which nodes would be a offensive upgrade, defensive upgrade, or not an upgrade. Thanks in advance!
You can use headless pob to run an item through pob, not sure what you mean by database unless you mean running the entire trade api through pob then querying the results from that
That's a very big topic with not a lot of specifics... Not sure where to start, or exactly what you mean, @pseudo ocean
Node power is pretty straightforward. PoB calculates what your character's stats would be if the node was allocated. Offensive power is DPS, defensive power is some composite of life, ES, and various defenses -- not sure on the details
@pseudo ocean u made me inspire myself to build a lil demo that feeds the poe trade api through pob
Thats awesome man, I totally want that functionality haha!

there's no way I could ever make this a public website a fairly simple build has a throughput of 24 items/second with a threadripper 2950X
maxing all cores on a*
headless pob? how to obtain it?
I wonder how bad headless PoB would run on the Knight's Landing nodes... 272-some Atom-like SMT4 threads... I don't suppose that PoB can leverage AVX-512? ๐
I once threw one of those nodes at a chess engine once. Did about as well as a single consumer GTX 1080.
They're generally meh, particularly if you have any form of serial code.
Ours are pretty much unused now and no-one wants to target them for development.
The nice thing about not knowing which product lines that Intel intend to murder ๐
How would you avoid hitting the rate limit when making requests? Is it as simple as putting a delay in your code between each call or is there a better way?
If youre making xhr requests the respone header contains the rate limit and your current status both by ip and account. I believe this is in the pinned information
Yeah I see what the rate limit is I'm wondering how to avoid hitting it
Ah thanks
https://pypi.org/project/ratelimit/ this for python is pretty nifty, used it before for APIs
@velvet gate - In NodeJS I used either Lodash debounce or velvet module - think debounce worked best though.
https://www.npmjs.com/package/bottleneck I like this for NodeJS because it has clustering support.
I had a terrible idea last night. Hook headless PoB up to the trade api stream and throw it into a generic algorithm to make the "best" (most dps) build for a given skill gem
I think that br4ther1ng'/emmit's code could be a better fit for that once it's modularized a bit further
We even have a generic algorithm for skill tree optimization already (not connected to our computation engine yet)
is there a good app for quickly transferring stuff from one tab to another during trade etc?
or tool*
nop
Nice
I've got a rough draft of some stand-alone javascript code I wrote to take fetches from the official trade api and render them as the official trade website does. Would anyone be interested in a module like that?
@deft jolt poe.watch is your site, right? I think the end date listed for synthesis is a month off (wiki says june 3, not july 3)
relevant (better) source: https://www.pathofexile.com/forum/view-thread/2484587 (bex says June 4th 10AM NZDT)
I choose to believe the API is wrong >:[
Yeah
Huh, that's weird
@compact isle Can you check the above? Poe league api has incorrect end date for synthesis
So today I had this idea to create a tool that reads your stash tabs, looks for fractured items and then combines all of them in every possible way to generate all possible outcomes and allows you to search them for specific mods.
However I failed quite hard because it's taking really long to compute the item permutations, even with multithreading.
Just my 44 fractured chest items are over 80.000 possible combinations which takes like 30 minutes to compute.
I guess there is a faster way to do this, maybe someone cleverer than me wants to pick up on that idea.
I'd cull all items with non interesting mods
Well the code that I wrote doesn't really know what a "mod" is, it's just text for it.
What I did was download the .json with the synthesis mods from poedb.tw and create regex patterns from then and match them directly against the mod list from the poe stash api.
80k shouldn't take 30 minutes that's for sure
I made a crafting simulator that did 100000 iterations a second. Strange that it'd take so long
also wouldn't it be 13k combinations not 80k?
how I'd do this is assign every "mod template" a unique integer (could be done dynamically at runtime), so e.g. "+# to Accuracy Rating" would be mod 1; then for every item, parse its mods into a list of (mod_id, mod_value) so if an item has "+123 to Accuracy Rating" it would be parsed as (1, 123), make sure it's sorted; then bruteforce all combinations (for your case that would be C(44, 3) = 13k) and calculate possible mods by summing just with a single linear loop (since mods are sorted)
btw I had no idea how synthesis works before reading this wiki page 5 minutes ago so dunno if this makes sense
Not knowing "what a mod is" is actually beneficial as IIUIC it's counting modified numbers regardless of source, so if you have multiple things on an item doing +maxlife, they all count together.
Are there any semi reliable tools that can create an outline/table of contents out of a pdf you know? My best idea yet is to try to find a tool transforming a pdf to any tree like structure that contains font information and then try to cluster by font size.
Kinda ot but why not ask here 
is the lutbot script using /Exit or something else to get to main loginscreen?
it kills your tcp connection
You can squint at the impl at http://lutbot.com/ahk/lite.ahk
Seems to leverage CurrPorts' command line interface?
@graceful fable care to share your code?
@frigid nova It's not really ready yet, but the suggestion from @chrome topaz really helped. I removed the multithreading and it's so much faster now doing it linearly.
By the way, does anyone know if there's a library that can parse the input of item mods (for example +(8 to 10) to maximum Life) and return the corresponding Mod ID and the values? So +(8 to 10) to maximum Life would evaluate to SynthesisImplicitLife1 with values ( 8 and 10).
I'm currently using raw text search with regex to match mods to their "ids" but that doesnt really work on all mods.
Is pypoe maybe capable of doing that?
Thanks for that link, but I saw that earlier already and as far as I understood it generates json files from specific data from the content.ggpk, right? But it doesn't actually do the conversion from the mod to the mod_id, does it?
@graceful fable note the mapping from mod_ids to stats is many-to-one
if youre restricting to synthesis implicits the mapping might be one-to-one, as @polar island pointed out all the data needed to construct this mapping is in repoe (stat_translations, synthesis_implicits, mods)
Feel free to pm me if you have any questions! ๐
Sure, thanks alot ๐
where do i go search for tools, that can help me autoreply Thanks!, click hideout etc?
Check MercuryTrade
MP: why did removing the multithreading speed it up? If itโs because of lock synchronisation, prebatching the processing and only syncing up infrequently (every few seconds or so) might be faster.
@graceful fable put t that on GitHub and I can give you a c# code review
@simple ravine I took a peek at your C# lib for interface inspiration a while ago... it's one of the more readable C# codebases I've seen ๐
(my Rust interface doesn't look like it, of course, went way more low-level)
there are 2 versions of it
check branch new for a more simplified one (not done yet)
I'm going to convert it to .NET Core 3 for everything so I can optimize away some of the .NET Framework stuff, meaning more concise code and perf
@worthy cape do you have your code on github? always wanted to take a little peek at Rust
@simple ravine https://gitlab.com/zao/poe-rs
Currently smashing my face against .arm files, there's a lot of variations on them.
Usual caveats apply, first pass implementation so it's not overly idiomatic or nice ๐
Yeah.
I tried using MemoryMaps, and in Windows it allocated all the memory until I released the memory map, but on linux it was nicer
Used files for a while, but it was rather error-prone with all the seeking.
If you go look at the actual allocation
I get something like 50 megs private, few hundred megs working set, 22G virt.
Which is perfectly fine, as it's backed by the file.
yeah, that's a problem ๐
oh hmm
well, my new way of reading it hierarchically got pretty fast actually
building up the tree immediately, rather than reading all the records then reconstructing the tree
With a hot cache, I spent something like 9 seconds to build my scaffolding.
This time around I went for hitting the file on the fly, exposing raw offsets to the caller as "unique id" of the entry.
Didn't profile why I'm having that cost, but I'd like to be able to use my library in one-off programs like poecat and poels.
my library's intended purpose will be my 'overlay' program later on, which will be a replacement for PoeItemInfo/PoeTradeMacro and MercuryTrade
with a bit more richer info and UI
and use the actual GGPK/DAT information for the item data
Goal I'm working slooowly toward currently is 3D hideout viewer.
Sounds nice.
Not that I need my project for anything in particular, but it's a great driving force in learning the language and libs.
Yeah
For mine, it takes 88 ms to open tthe GGPK and then get the Data folder files
like so:
var ggpk = new GgpkFileSystem(ggpkPath);
var dataFolderFiles = ggpk.Directories["Data"].Files;
Ah, right, you're incrementally populating the tree.
I've got a hard time with that as interior mutability is hard.
well it just stores the pointers
finds the root and gives you the pointer to the first directory
this way, initial load will be fast enough for a desktop appllication
and a desktop application like this likely will never need the entire tree anyway, just a small fraction of the 300,000+ files in the ggpk
I toyed with the idea to have an acceleration cache, that had pre-parsed the structure of the GGPK.
But that blew up as I couldn't tell when the file had changed, might've been able to rely on the 32-byte digest-thing of the root.
One nice thing about Rust tests is that they run in parallel. One less nice thing about my cache logic was that it raced hard ๐
(had used disk-backed sqlite3 with insufficient transactions)
hehe
476383 files in GGPK
Opened in 3035.6666 ms
(OS cached GGPK)
full enumeration in 3 secs isn't that bad on a cached ggpk?
Same magnitude as I had, so fine.
Must be nice for you to be in a world where UTF-16LE is the text encoding.
I've got UTF-8 and a lot of conversion ๐
(beats C++)
Some C# can actually beat C++ nowadays because of super-smart JITing etc
if you write it properly... obviously takes some computoring knowledge, but possible
And obviously not for all cases, but some
The primary reason I don't do C# is that I hung too much around the SlimDX people ๐
I wouldn't know who those are
but if you feel inclined to become indoctrinated with some proper C# sometime, let me know ๐
.NET bindings for D3D9-11, went rather deep into working C# and C++/CLI without excessive GC churn and making the runtime tick.
I spoke with Tanner Gooding, whose on the corefx team
He said he wanted to make all the DX stuff work properly in C# natively. We'll see how that'll turn out once they get to it
but that'd be pretty cool
yes, even specific hardware intrinsic if you'd like
but there are also higher-level SIMD operations using Vector<T>
i'm checking items json for french language and found thing that i don't understand:
Sacrifice \u00e0 minuit (in json) --> Sacrifice ร minuit (in browser)
but
Sacrifice \u00e0\u00a0midi (in json) --> Sacrifice ร midi (in browser)
why \u00e0\u00a0 transforms into "ร "?
\u00a0 is a non-breaking space
thanks. my bad, thought it's doubled 00e0 (idk why, maybe tired)
@deft jolt the times on the league API are sometimes off, we just fix them before it becomes relevant
@velvet fog @queen oriole sent emails out with OAuth deets, sorry for the delay
thanks
POB will be localized into Korean?
questions does anyone know how to bind a key to make a photo pop up. like presing f8 and i can get the pic of where fossils and who does what in syndicate
I use sharex which has a boat load of features like that
pressing Print Screen or Alt + Print Screen will copy a screenshot to your clipboard
I printed my cheat sheet for Betrayal after getting tired of having it on the other screen.
technically pressing print screen by default copies a screenshot to your clipboard
windows snipping tool is pretty nice for me
alt-gr + print screen for active window as well
could swear someone already wrote a ahk script to do that
Win shift s for snipping shortcut but it sometimes breaks
noone is asking for snipping he wants to press a hotkey and have a chart be displayed 
yeah it was a tangent
Not exactly tool dev but if you guys had a 3D printer what would you print PoE related?
@bleak valley โ
We already banned popsicle sticks. I really don't want to ban rubber bands too.
can i figure from mods.json (repoe) what mods are excluding each other?
don't believe so
@storm wedge you can. the main one is group: mods with the same group exclude each other. there's also adds_tags, which interacts with the spawn weights. see https://github.com/brather1ng/RePoE/blob/master/docs/mods.md for more info.
but for example, incursion temple mod (weapon flat ele dmg+pen) and jun pen. craft have differrent types. does this mean i can craft ele pen on temple item?
@polar island i think increase stats with the same id are additive. not sure if stats with different ids can be additive, probably yes if they resolve to the same stat when evaluating the condition.
temple: LocalColdDamageAndPen; jun: LocalAttackReduceEnemyElementalResistance
you probably can. i don't know of anything else that would influence it
does any1 has a nice sheet for investments?
like where i can put how much sextants used etc?
Anyone know why I get this error when I try to run RePoE
PyPoE.poe.file.specification.errors.SpecificationError: <ERRORS.RUNTIME_ROWSIZE_MISMATCH: 3002>: "GrantedEffects.dat": Specification row size 93 vs real size 94
because it depends on PyPoe that is out of date for recent version
ah ok, thank you
qq, does PoB have a way to handle the Arcane Surge from Arcane Blessing/Ascendant->Heirophant, or should I just tack a L1 gem to the end of the link to simulate it?
@stuck reef If the text is red, the mod is unsupported. Just tack it on
kk thanks ^_^
Is it legal to use the poe stash api commercially? I can't find info on it.
commercially how?
Some sort of trade or other QoL site with paid accounts/donations for the site?
I don't know how yet, not there yet. I just know that the tool I'm working on eats way too much cpu to be supported on ads.
are there any decent gem planners out there like https://poecraft.com/gemplanner used to do?
@waxen kernel You can do that in PoB just fine
@golden bane PoB can tell me where I get each gem?
don't think so
@waxen kernel I'm sorry, I missunderstood your question. Recently, I made a website that offers such functionality, as long as you use PoB alongside it: http://cinnabarit.pythonanywhere.com
Path of Exile Leveling Helper
@golden bane awesome thanks, I will try that out
@velvet fog on this page, can you add a column for the item that the mod appears on? https://poedb.tw/us/mod.php
better than clicking each item page individually
Does Poe-Trademacro reads Memory? or how does he gets the informations out?
@ashen plank You copy the items you want price-checked to your clipboard with Ctrl+C. Reading from memory would be against the TOS
But how does he checks where my mouse is on?
Free keyboard macro program. Supports hotkeys for keyboard, mouse, and joystick. Can expand abbreviations as you type them (AutoText).
No need to check mouse positions from in-game, you can do that through OS APIs
i dont get the point how he really gets the informations^^
ctrl + c on item gets item stats on your clipboard... so ctrl + c can get chararacter information if you do it on you char too? if you add the code ofc
I don't think I understand your question
What character information are you talking about?
Its ok, my bad english^^ do you mabye know, whats the name of the Function is, in trademacro? Where he gets the Informations to Clipboard.. to much Code to find it^^
Strg+F > "clipboard"
@ashen plank Pressing Ctrl+C while hovering over an item copies item information to the clipboard. This is a core functionality of Path of Exile, not one that's added by AHK or AHK scripts
What scripts like PoE Item/Trade Macro do is add extra behaviors after this information is copied to the clipboard, like parsing the item text to determine mods and displaying it on screen.
But if you press Ctrl+C in PoE, then open Notepad and paste, you'll see what it's working with
Ok thank you very much! thats what i wanted to know.. never testet it without trademacro lol...
The other place you can legitimately get information out of PoE is parsing the Client.txt log file for area transitions, chat, etc.
yeah, i saw that too from labcompass, very interessting how many informations you can get out without memory reading ^^ Is somewhere a List of the possibilities getting Informations without Memory read?
From the top of my head: clipboard, Client.txt, ladder API, character API, public stash tab API, OCR (pls don't)
OCR is allowed?
I can see the argument both ways but I always thought it wasn't. or that they had never explicitly allowed it
Yes, but it's usefulness is very limited as you cannot take action on what you OCR'd (otherwise it would obviously be gameplay automation), only display information to the user. Probably not worth the effort in >99% of all use cases
right, I get that
but you could, for example, have a script that would alert the user if it detected they were in a reflect map
with some limited reliability
Yeah, you could very well do that
It would be nice to see someone take an open source OCR package and tune it properly to read map mods reliably
With the user configuring game window resolution/position and language
However, I think it would be much easier to have the player Ctrl+C maps to the clipboard and analyze that instead
yeah thats the only application of OCR I can think of that would actually be pretty useful. and I imagine doing OCR on a known font, with relatively little noise, would be pretty accurate
If the player is conscious enough to ctrl+c every map before they put it in Im not sure they'd need to be warned of anything
at that point you can just use stash search to find certain mods, which would probably be more efficient
Ctrl+C will not display sextants, scarabs, and the like.
also true. and zana maps
Good points from both of you
OCR could prevent false positives from reflect map + anti reflect sextant
unfortunately such a tool is less useful now that you can hide the map mods, you'd have to have them always showing for it to work
which can be a bit annoying
hello, anyone from ggg here please? I need to ask a question about a tool I've made
Btw. Just ask the question. Youre probably not super likely to just get a specific GGG response. If anyone else can answer the question they will or if someone from GGG can answer it theyll see it.
it's a license question, I don't want to share the tool unless I'm sure it's ok with GGG licenses
Might have a go with emailing support.
ty
fyi, I've published a simple static web page which is showing a calendar of the currently running leagues: https://poe-leagues-calendar.gitlab.io/. It's using the PoE API, so it's only showing what's availlable in the API. It's completly free ofc. You can find more informations there: https://www.reddit.com/r/pathofexile/comments/bn0ot6/poe_leagues_calendar/
anyone wanna help last minute test some changes i've made to the race zone tracker?
Hey guys, I try to use the poe-racing overlay but it can't find my character.
I'm playing in Flashback Softcore SSF and have selected the correct league when downloading the script, but it just says "Could not find: ..."
Still isn't changed in PoB, not that people care, but next league they might do.
is poe trademacro updated for the event?
no, but there is a fix here https://github.com/PoE-TradeMacro/POE-TradeMacro/issues/970 to get flashback leagues to show up
Wondering if anyone who has more experience with datamining can give some advice here. I recently started using datamined gem tags to determine whether I should list the 'speed' stat from PoB as "attacks/sec" or "casts/sec":
https://github.com/aggixx/PoBPreviewBot/commit/21822f182bb7fc592dbfdcdcaa56500a674bb8c2
Ideally, I would like to do the same thing with minion skills. However, the skills minions use are not gems and are not in RePoE's gems.json
As far as what I have to work with, this is what PoB XML gives me to work with:
<Gem enableGlobal1="true" enableGlobal2="false" enabled="true" gemId="Metadata/Items/Gems/HeraldOfAgony" level="21" nameSpec="Herald of Agony" quality="20" skillId="HeraldOfAgony" skillMinion="HeraldOfAgonySpiderPlated" skillMinionCalcs="`" skillMinionSkill="2" skillMinionSkillCalcs="1" />
So I have what is, as far as I can tell, a unique identifier of some sort for the minion. I do not have a unique ID for the minions skill, but I guess if I could find that monster HeraldOfAgonySpiderPlated it would be its skill with an index of 2? or 1?
but I'm not sure where I can find info on this HeraldOfAgonySpiderPlated. And this is just for example's sake, ideally it would support all minions (Raise Spectre being especially complicated)
Id guess its in some monster data that isn't exported in RePoE?
in theory I could definitely get that info from PoB's data files
https://github.com/Openarl/PathOfBuilding/blob/master/Data/3_0/Skills/minion.lua#L783
but that seems like a less than ideal solution
Messing with PyPOE, I think I see how I can export spectre data and minion skill data. Not sure about minions themself yet (eg HeraldOfAgonySpiderPlated)
looking through PoB source trying to find some clues as to where it would be , the only mention of HeraldOfAgonySpiderPlated in exporting is here
https://github.com/Openarl/PathOfBuilding/blob/c2de224a278867da4c82c8341062d99ca7a575b6/Export/Skills/act_dex.txt#L492
But I can't find the code that datamines the minions in minionList
I had a very productive weekend. Managed to parse all my v14 and v15 .arm files to the end. Struggling a bit with v16's apparently arbitrary sequence of trailing numbers, but damn this is fun to do.
Much easier to start with old versions than to go for the v29 hideouts I started with.
any document or code could be shared?
@velvet fog I try to document as I go, but a lot is in gasp analogue form on inked paper ๐
A lot of the effort spent has been on getting my GGPK library into enough shape that I can brute-force apply a parser to all 18k ARM files easily to rapidly test it.
Probably a bit overkill understanding all ARM files if all I actually need is the hideouts, but hey, it's a challenge.
thanks
hmm hypothetically, if one was to make a macro that picked up all of the perandus coins with one buttonclick per perandus coin drop
would that be against ToS?
Yes
one server action <-> one user action
o what i meant was, lets say that i have a htokey bound to like k
every time i press k it scrolls over and picks up one of the stacks of coins
so i just have to spam k to pick up the whole pile instead of click on each one individually
so one press of k = one mouseclick for server
still not one user action
anyone who could help me get trade macro to show flashback prices instead of synthesis?
@quick wedge Answered earlier - https://github.com/PoE-TradeMacro/POE-TradeMacro/issues/970
If you're uncomfortable editing the files, one of the later comments link a modified file.
Is poedb down?
Seems so from here.
Posting my question here at the advice of Faust.
It's really more of a math question than a tool question, but it's the basis of what I'll need to create a tool down the road so I need to know how to proceed.
I'll repost my little essay that may include needless exposition because yall are familiar with PoE
For reference here, I'm testing things in a game. In this game an item can have up to 6 affixes on it -- 3 prefixes, 3 suffixes. For this experiment's purposes it has to have a minimum of 4 affixes.
As you may realize, there are combinations of ways to get these.
For example - if it has 4 affixes, you can have 3 prefixes 1 suffix, 2 prefixes 2 suffix, or 1 prefix 3 suffixes.
There are also weightings for prefix vs. suffixes. What I want to determine is a rough formula that correlates this weighting ratio of prefixes to suffixes to the % chance of getting 3pre1suf, 2pre2suf, or 1pre3suf for example. 2 pictures of data collected to follow along with what I think I'm supposed to do.
This is the result of 500 "rolls" on an item that had a prefix to suffix weighting ratio of 1:1.39. As you can see, it decently weights towards having more suffixes than prefixes. I noticed this and decided to experiment further.
https://cdn.discordapp.com/attachments/472867163330576384/578301940237074434/unknown.png
The most offset weighting I could find was a ratio of 1:2.61. This is the result of 500 rolls of that item. It's CLEARLY heavily influenced (3% of the 4-affix results were 3pre1suf compared to 71.5% being 1pre3suf)
https://cdn.discordapp.com/attachments/472867163330576384/578302129119428610/unknown.png
So what I think I should do to approximate this is find the extremes for ratios on both ends, gather data at maybe 10 total points with equidistant ratios, and graph how many of each prefix-suffix combination occur for each. From there, I could approximate a trendline.
So the main question is, is that an appropriate way of going about this? If yes, how many total ratios should I test? If no, how should I go about this?
This is a question I've thought a lot about and would be happy to discuss with you.
There is a lot to be said so I'll just pm you directly.
- I believe it's simply the
spawn_weightin combination with probabilistic theory. - Mods are added to the item procedurally, eliminating any
mod_groupconflicts. - Bear in mind that there are also tags applied for some mods which "modifies" the
spawn_weightof mod N+1
yeah @simple ravine had a very long conversation with nick about it. Think we figured it out for the most part.
i appreciate the input
actually, do you know of any mods that have tags that modify the spawn weight other mods without completely eliminating them besides the caster/attack groups?
not on top of my head, sorry. those are the two main ones.
but i believe there are some added ones the past couple of seasons
np. figured id ask while you were here. i'll take a look into that
I was thinking about creating some sort of interactive leveling guide, offering both generic default guides fit for all classes, but also specialized guides for each ascendancy and multiple builds, including the ability for people to actually create and share their own guides.
I'm however rather inexperienced in coding or modding of any kind and don't know if there are any resources I might use? Much like Path of Leveling for example, the guide would require a library of gems and perhaps gear items too. Does such a library already exist in a why that I might incorporate in, say, spreadsheets?
@compact isle Hey Novynn, sorry for poking you directly,
I've been thinking about upgrading our lil' website www.filterblade.xyz . Currently it uses a anonymous online storage system and a local storage system to store user filter presets.
I've seen that one website (PoEMap) is using OATH2 to identify their users and imo that'd be the ideal way to go for filterblade as well.
Is there any chance we could connect filterbalde to the PoE-OATH-System to identify our users?
@hushed relic feel free to flick an email to roryv@grindinggear.com with your details / requirements
there are multiple Rory's, I'm usually just "Novynn"
So I've made a most likely exceptionally shitty python script for pulling counts of mods from a text file that had several thousand rolls of items stored in there. I have literally never touched python and am trying to learn, so this was mostly made through google fu and trial and error.
Would appreciate it if anyone who actually knows python could tell me
- Why it is bad
- How to improve it
- How to implement something like determining whether the item rolled 4, 5, or 6 affixes
- How to implement something to determine how many prefixes and suffixes the item rolled.
I suspect I could do 3 and 4 by adding prefix/suffix counts that reset with each loop maybe? Not quite sure though. Ty for any help
update: i have slightly improved my shitty tool
Can you also send the 4000.txt? I'll show you an improved version of the code afterwards
@pseudo ocean
@livid fable i think youre looking for something like this for 3.:
# define pattern between chars, in your case: new line, &&&&&&&&&&&&&&&&&&&&, new line
def parse_item_string(file_name, delimiter='\n&&&&&&&&&&&&&&&&&&&&\n'):
with open(file_name) as f:
content = f.read()
# get all items in a single string, split at the delimiter to get the single item string
return content.split(delimiter)
def parse_item(item_str, delimiter='\n--------\n'):
item_blocks = item_str.split(delimiter)
# items consist of multiple blocks delimited by newline '--------' newline
print(item_blocks)
#get each item's mods line by line (mods are always last, thus -1)
item_mods = item_blocks[-1].split('\n')
print(item_mods)
if __name__ == '__main__':
#obtain item_string list from 4000.txt
items = parse_item_string('4000.txt')
#parse the first item from the list above
parse_item(items[0])
Output is:
python/playground/poe/demo_2.py
['รฏยปยฟRarity: Rare\nPain Refuge\nSplintered Tower Shield', 'Chance to Block: 24%\nArmour: 10 (augmented)', 'Sockets: B R ', 'Item Level: 1', '10% increased Armour\n1 Life Regenerated per second\n+6% to Cold Resistance\n19% increased Stun and Block Recovery\n']
['10% increased Armour', '1 Life Regenerated per second', '+6% to Cold Resistance', '19% increased Stun and Block Recovery', '']
for 4. you have to estimate whether its a pre or suffix depending on available rolls, nothing like that is 100% accurate besides having the infos GGG has with advanced tooltips
I just went with his dataset of clipboard items
@livid fable Some suggestions:
- Variables are written
snake_case, constantsCAPITALIZED_SNAKE_CASE. For further information on styling, refer to the almightyPEP8 - For "initializing" integers for use with augmented operators,
= int()is the clearest approach - When dealing with resources, context managers like
with open() as f:clean up after you, e.g. close files automatically once they are no longer used - Redefining
linesaves no memory in Python, it sacrifices clarity instead - For case-insensitive string matching, use
string.casefold()instead ofstring.lower() wordsshould be namedwordinsteadstring.find()is for finding the index of a substring. If you just need to know if the substring is contained, use theinkeyword instead- For maximum reusability, split your code into modular functions
if __name__ == "__main__"only runs your code when you execute the file directly, allowing future library use
@golden bane I will look into what all of this means when I get a chance, I appreciate it.
python doesn't have any scope like using for auto-cleaning up resources like C# does right?
would be a pretty good addition if not
Most of the time it doesn't matter, but if you need something like that you can always just define your own context manager
in C# we do
using (var fs = new FileStream(fileName))
{
// Do things here
}
// fs will be closed and disposed here,
// even if exceptions are thrown inside.
and it's pretty common to do stuff like this:
using (var fs = new FileStream(fileName))
using (var sr = new BinaryReader(fs))
{
// Do things here
}
because this is messy:
try
{
var fs = new FileStream(fileName);
var br = new BinaryReader(fs);
// do stuff..
}
catch (Exception ex)
{
// handle stuff
}
finally
{
br.Dispose();
fs.Dispose();
}
In Python, you would do something like this for other file objects than files as well:
from contextlib import contextmanager
@contextmanager
def open_file(path, mode):
the_file = open(path, mode)
yield the_file
the_file.close()
files = []
for x in range(100000):
with open_file('foo.txt', 'w') as infile:
files.append(infile)
for f in files:
if not f.closed:
print('not closed')
contextlib/contextmanager is built into python or an external thing?
stdlib
I presume that's something that comes with python?
hmm yeah that works, could make interesting things with that I presume
However, in Python you still have to handle exceptions
how would that work in async contexts though?
Can't do it in the context manager, as it doesn't do anything between the calls to __enter__() and __exit__()
from contextlib import asynccontextmanager
that's one of my biggest issues with most popular languages, that the async story is a bit of an after thought
to be fair, python is a grandpa with a great come-back story
oh wait, 1990... I thought it was older
Python async support is pretty good, it even has it's own keyword
The asyncio standard library brings stuff like async def and async with keywords
still feels like an after thought, being something you import
for C# for an example, it's syntactic sugar that the compiler then takes care of
e.g. it's first-class citizen in the language
Python imports are costly and async / asyncio draw in a lot of them
Plus almost nobody writes async code in Python
I guess it's apparent that I don't fancy Python much heh. I have grown some respect for it, but it's still in my opinion an inferior language/environment. But there's a lot of interesting built with Python
Well, there is nothing inferior about it. It doesn't even try to compete with C#
Right. I don't intend contesting or argue the case, and I fully respect people who develop with Python ๐
yeah, I think both Java and .NET Core will handle threading better, which is surprising considering its popularity with data scientists.
but hey, it's better than PHP at least ๐
is there a "pypy" and "pypy-stm"?
ah ok
Also, numpy is one of those things that you really want to use if you're calculation heavy, I assume?
and then I've stumbled upon 'pandas', but I presume that's more for data scienty stuff
iirc, numpy handles broadcasting and tensor operations really well because it pushes all of iteration down to the level of c.
Writing some tensor operation in python versus just having numpy broadcast it has pretty wild timing improvements.
yeah I can imagine
Ultimately though python is still pretty slow at some things - I struggled a lot with trying to get fast pseudo random number generation in python
its wildly slow compared to C counterparts
one thing I like with .NET Core is that they're hard on pushing performance stuff, utilizing hardware intrinsics where possible, and because it's JITed, detecting features is free
isn't that numpy?
@earnest radish yeah that was going to be my next attempt was to use cython to import the random number generation directly.
or you mean, write your number crunching operations in c your self?
i'll stick with c# ๐
whenever I had a problem with python being too slow and tried pypy, it was even slower...
yeah I've stumbled across that before xanthics, the general moral is numpy.random is much faster for generating single randoms, while numpy is better for doing batch generation.
@earnest radish which classes?
is it pseudo-random or crypto-random?
its just using mersenne twister iirc
at work, I draw too many boxes with lines between them while talking to customers
but that can be fun too, i guess
personally I've always really enjoyed teaching, but it's definitely a different set of tools to industry coding. What kind of frontend experience were they looking for?
heh yeah, mine are a little different, but hey.. to boxes!
i played around with the new css grid thing... looks pretty cool
Selenium is a headless browser thing used for e2e teseting
or UI testing
should be easy enough to learn if you have a good foundation, I guess
just put it on your resume, they're only gonna ask you to invert a binary tree in the interview anyway
Na, they ask you to do FizzBuzz these days
we consider inverting a binary tree as a leaked interview question here, lol
where?
@compact isle Can we register for oAuth Provider anywhere at the moment?
I run several Discord Bots that use POESESSID (primary one is stash-tab-runner to output stash tab modifications to discord)
The other one used to use it as last_online was available on the website and we did cool graphs with user data but GGG seemed to of removed that field from being available in 3.6
@elfin flume flick an email to roryv@grindinggear.com with your details / requirements about OAuth
cheers
wasn't sure if this had been posted yet
can't import links yet, but still, pob in a browser, hellz yeah
@tidal nova I believe some of the responsible people hang here ๐
@worthy cape plz point me in their direction! i play on a mac pro via GeForce Now, and it's the only way I've been able to use PoB successfully!
Judging by search, it's our good buddy xyz?
rip passive skill trees
there are jewels that can corrupt the tree lol
sounds like a pain in the ass
wonder if we'll get support in pob
Ziggy's video says each jewel has a seed which changes its effect on the tree?
it shouldn't be too bad depending how they are in the dat files
I would recommend watching Baeclast #40, Jonathan shows some debug overlays, commands, and an attack animation preview tool ๐
huggles his single-frame-step key in the video player
Every jewel can end up scrambling your passives tree a certain way. The clue is a particular โseed numberโ when you read the jewelโs description. It would say something along the lines of: โA battle occurred in year <X>,โ where X is the seed number itself. Each seed has different effects for particular sections of the passives tree. Iโve been told that there are thousands of possible results, and youโll just have to trade to get the type of effects/seed number that you want.
the "historic" mod line is one aspect
i'm assuming the base is another (including help text)
so long as you can see the exact values via alt text and it's not some modulus there could be a way
Looks like the base might limit the mods on the jewel
Like the one annouced might only work in certain sockets
there's new socket specific code?
Because of the stats it modifies
oh you mean it'll only be effective in certain sockets
one for each legion faction
Yeah
i hope the PRNG algorithm gets released
all it's going to be is determining how each seed affects nodes (ie, is it a stat transform, a random roll, the order each node gets assigned something). also going to be a pain to see what happens when two overlap
ggg isn't exactly known for its transparency in game mechanics
hopefully it's not something you have to figure out by using a debugger on poe, haven't used an assembly debugger in forever
with those jewels, I imagine their effects are different depending on the jewel slot too... so all those 1000s of combinations are then multiplied yet again
but it wouldn't be poe with the need for an external site/tool, right? ๐
"I found a jewel, let's see what it does and where it's good before pricing it"
"I want a jewel that does X, let's see what seed number matches that need for slot Y"
@queen oriole is it known if two can overlap?
Lol ๐ I was like whose pinging me lol.
@frigid nova have to wait until we can see another jewel, historic might be the same mod for all legion jewels and all are limited to 1
i'm starting to think historic is going to be the same mod on other jewels
Also inb4 they just hardcode 9999 seeds into the .dat file ๐
at least 5173 seeds
for the vaal one
unless they don't start at 1 for whatever random reason
just because the number is 5173 there doesnt mean there has to be a 5172
i could understand starting above 1 but do you really think GGG is "random" enough to make a mod that rolls numbers in ranges that start at 5173?
no, but I think there is a possibility that not all numbers between 1 and 5173 (or whatever the max is) are "rollable"
@simple ravine , that would be really weird then because the mod system generally works by rolling between number ranges, and there's no functional point to have a range of 1000-2000 and then 3000-5000 etc if it's producing a random output
if there was some deterministic reason to separate the ranges it would be much better to just put as another mod
or by jewels
eh
a simple mathematical formula would suffice to achieve that
like, prime numbers, sin/cos/tan
divisible by 3 or whatever u want
so what is the point of having ranges
multiple ranges that is instead of a flat range
what is the point of the whole number thing in the first place?
to manipulate the tree by assigning random properties to nodes
because if you do get a matching seed you could actually know what the outcome is
i give up
are you assuming some pattern is gonna exist based on the values itself?
instead of it being actually random
what i wanted to lead you to is
why would GGG want to present a seed number to the player?
to confuse and make something mysterious obviously
what
not to provide a strange yet deterministic item
well i'll disagree with that then
if they really want to confuse players just hide the seed
would make it just as confusing and mysterious
even more actually since you wouldn't actually be able to make tangible comparisons
at least without memory hacking to get the actual mod value
either way, assume there are 1, 2, 3, 4, 5 - 5173+
1, 6, 11, 17 etc could be the same thing
or something like that
51 could be one seed, and 73 another
that would go against their statement saying there are thousands of possibilities
if it's a modulus then there aren't actually thousands of possibilities
i mean it could have been a lie but i'm not assuming that right now
well there are other permutation coefficients
like tree placement, base item type etc
then the seed would matter
and not the same thing
wait by what you mean
then it's still not thousands
yes
the number of slots on the tree is like 21/22 jewel slots
you would still need at least a couple hundred combinations
i'm trying to explain it to you in multiple ways
whatever
how many jewel slots are there?
21/22 something around there
5 (base item types) * 21 (jewel slots) * 25 (seed variations, example assumption) = 2,625
wasn't it thousands of possiblities per jewel?
like specific legion types had thousands of possibilites each
yeah so it's at least 100 seed variations regardless like i said, still hundreds
well 96
since 2000/21 to qualify as "thousands"
i think it'll be easy to know for sure, someone's going to buy a ton of these jewels
and you can probably divine them
if you really think that the number of unique mods for a single jewel base is low, then eventually you'll pigeonhole into the same jewel
same effective jewel modifications
Jewel Nยบ 5 - best modifier for the situated exile
Oh divining them will be very interesting ๐ค
It sounds like the seed is based on the nodes that the jewel can affect
It has to be a somewhat deterministic since the client will need to know how to change the tree
hm, doesn't a jewel affect every node in radius since all the icons change?
i guess it'd be nice to have been shown a node aside from the frenzy charge getting changed since we don't know
could just be visual
it appears that everything in a radius is converted to x node type
actually, I think I am wrong about that
some of the notables have been converted to one node and some to another, same with the pathing nodes
right now based on that post, i'm assuming it's going to be some PRNG transform function on the graphId/nodeId/some new uniqueId on each PassiveSkills entry, and the transform is completely unrelated to what the node actually currently is. also assuming the mod pool for transformations of a specific jewel type is a new table, with a different pool for keystone/notable/small.
it looks like it only takes into account what type of node it is
keystone/notable/normal convert, mastery & jewel socket don't convert
pretty much, i hope it doesn't end up requiring memory debugging/decompiling to figure out the algorithm
there's one place the algorithm might leak out without needing that but i'll keep silent about that
It seems like they want to keep the algorithm a secret as long as possible
also, if this is on the website, then it should be easy to figure out
easier*
yeah that's what i was guessing, the passive-skill-tree page might leak it
even if it doesn't have it directly, you can run all the combos through the service giving the information back to the website
you'd still have to equip stuff in game to update the page right?
depends on how it is implemented. It is possible that you would have to do that
I am not really a fan of attaching a debugger ยฏ_(ใ)_/ยฏ
me neither, but it might be the only way
Is there a comprehensive list of poe.ninja API endpoints for item tracking? I found bits and pieces of them through Googling, but I still have no idea how to retrieve item overviews for Hardcore Synthesis league, for example (i can do it for standard, hardcore, and softcore synthesis)
Upside of 4k monitor and not being arsed figuring out DPI scaling - you can fit a lot of information in a tool screen :D
https://i.imgur.com/Z1zt3g5.png
hideout stuff?
Aye.
Started looking at the hideout.arm files and yakshaved into parsing every single one of the 18.5k ARM files.
Working my way up through the versions of the file format, got some idea what version 29 (current) entails, and have gotten through v14-v21's structure.
Now alternating between parsing additional versions and trying to figure out the meaning of the k-entry fields.
The first two fields are the width and height of a larger tile that the K-type entry covers. k_1_by_1 describes the shape of the k_x_by_y section.
nvm, I've analyzed google chrome's devtools network tab to see the exact http requests being made from the poe.ninja API (example with currency overview for hardcore synthesis flashback: https://poe.ninja/api/data/currencyoverview?league=Synthesis%20Event%20HC%20(SRE002)&type=Currency)
n cells are placeholders that a k entry covers, s seems to be some sort of empty space. f I haven't figured out yet.
could be connection points?
I ran some statistics over all the fields of K-entries and they seem to lump up in groups of four, so I'm guessing some sort of adjacency.
The plan now is to visualise them and see if they make sense by eye.
I know, from one of chris' past talks, that they have a map of every connection point, but it was 3 on each side
Hard to get a feeling for them in the original textual representation as they're 23 fields wide (24 in later versions).
Once I have vis up I can also start comparing ARM files corresponding to known locations in game and see if there's any hints.
Hideouts, boss arenas, etc.
I've also got a tool that runs in the command line shotgunning my parser across all the files in the GGPK to spot files that don't parse right.
Right now I've run everything against a Content.ggpk snapshot from 2019-03-31, but I've got the preseeds torrent GGPKs from Betrayal, Synthesis, 3.2.0 and 3.0.0 lying around.
Do you know if there were anything significant missing from those pre-launch GGPKs?
You see lines 12-15 in the ARM file? There are 2 * width * height such lines.
Yeah, they try and remove anything that chuanhsing will post
I'm suspecting those are borders/connection entries of some sort.
Err, not width * height, the dimensions on line 8.
So probably not borders at all, I had that wrong.
in your screenshot or the gist?
a_upair and c_uipairs in the screenshot.
ah, I see 0, 0 pairs
I used to think a_upair was the dimensions of the K-block, but that only held for the first fistful of files I parsed.
in the gist, line 25, 26
k 1 2 1 0 1 0 1 3 6 6 1 3 6 6 0 0 0 0 0 0 0 0 2 s s
n s s
are those meant to be on 1 line?
They are verbatim from the ARM file, they're supposed to be separate.
They make sense if you group them up like:
(k 1 2 1 0 1 0 1 3 6 6 1 3 6 6 0 0 0 0 0 0 0 0 2) (s) (s)
(n) (s) (s)
The understanding I have there is that the k-entry there is a 1-wide 2-high tile that also covers the n below.
Across all my files, that seems to hold.
ah, ok cool
the fact that 27 and 28 are clumped together is weird though if they are split between 2 lines
27 starts with a 1x1 k-entry, 28 starts with a 3x3 k-entry.
k.s.s
| . .
n.s.s
. . .
k.s.s
. . .
k-n-n
| | |
n-n-n
| | |
n-n-n
this tool on your github?
haha np
The older parser is up on gitlab, but butchered it a bit to be able to use it as a lib for the vis.
I do intend to continue writing up my format specs.
Still a lot of guesswork and placeholder names all over the place ^^
I wonder how much of my animation format work I need to scrap now with 3.7.0's new animation system ๐
probably most of it lol
I really like that the asset preview tool that was shown in the Baeclast seems to use Dear ImGui for the interface.
Time to hit the sack soon, thanks for the discussion.
Sure thing
@worthy cape: they started deleting .dat files from preload ggpk at some point
and they are afaik based on some earlier build before the release, not necessarily the one that will be playable
So they might be some changes still between prerelease ggpk and the actual patch
Final progress shot before I actually go to bed ๐ https://i.imgur.com/SY9OU0p.png
Visually rendering the k s s and k n n stuff?
Not sure I see anything in that kind of just a blob
Yeah, just wanted to see how much I could get out of a UI grid before I start rendering things properly.
That video was a lovely re-find, I had forgotten completely about it.
Got me spelunking in the data dirs, there's both master.tsi as some sort of tileset info, and tiles.tst specifying a lot about the tileset, referencing a ton of binary tdt files, which have strings referring to et/gt files.
What a web ๐
Yeah, I like that video a lot. Their terrain generator is amazing
I see later on that the connectivity arrows are not always centered in the ForestWall set. Interesting.
The terrain generator has a reference to an .et file when they place the wall spline too, so they are at least inputs into the generator.
Anyone able to help with a problem i have with PyPoe?
Why do I even keep this 128 megabyte JSON file around... all it does is crash VS Code.
@gloomy idol Probably not, what ails you?
@worthy cape i tried using both the ui and cli and i keep geting things like "ValueError: Did not find data magic number in "Mods.dat"" when i try to view a file.
Odd. What game version are you looking at? The magic number is present in my 3.6.5 file at least, haven't tried pypoe.
Do you need the stand alone client version and not steam?
Mods.dat is identical between my Steam and Standalone packs.
do you use latest version of Content.ggpk?
How can i tell it's latest version?
Run PathOfExile.exe once
Just installed PyPoE here, loads Data/Mods.dat after a long long wait.
Yea, i got PyPoe to work on another computer. Gave not been able to open Mods.dat yet as it makes the UI unresponsive. But i can open the other files at least.
Trying to get a useable Object list for affixes. Tried using RePoe with it's json. But it didn't have mods set up good enough to use to convert POE stash API affixes to objects with teirs and all that.
If anyone knows how to map the mods and stuff from PyPoe or RePoe to "+35% to Lightning Resistance" hit me up.
@timid hemlock #PathOfBuilding in path of building some watchers eye rolls are still missing and 3 stat watchers would be nice. Additionally in the config section a checkbox with arcane surge would be nice.
Is acquisition working well for others?
Price and everything gets updated on my forum shop but trade sites don't show any change
I fixed a bug this morning that was affecting the GGG site, but it won't have impacted poe.trade or anything
Im a nobody but just wanted to mention that in the small video clip uploaded to PoE official YT channel 1 hour ago has clips with Bottled Faith in inventory. It has been announced Bottled Faith will not be in 3.7 onward? So redo of the video maybe?
Delete that if you feel it shouldnt be here. Just not sure who to mention to incase the clips actually do need looking at
you can always post any feedback to support@grindinggear.com and it'll filter down to the correct people
definitely a better place to mention it than #tooldev-general
Ok thank you. Sorry about that
@obtuse citrus I tried setting up the module but could not get it working. Mind showing a simple example of how to use it?
I was trying things like pprint.pprint(tf.reverse_translation("+312 to Accuracy Rating")) but couldn't get it to work.
Seems like there was a problem specific to strings starting with values
I fixed that
@worthy cape What are you working on for a tool? ๐
@frigid thorn Ultimate lofty dream goal - Custom hideout viewer (in VR?).
Right now, figuring out the ARM files that describe hideouts and bigger tiled set pieces.
And along the way, a Rust library to grok all the file types there are in a GGPK.
Oh, sounds lovely!
Finally finished getting all the skill trees I have up and running
http://poeskilltree.com/?v={1.1.0->3.6.0}
you can add &c={1.1.0->3.6.0} to the end and compare the trees (i.e. http://poeskilltree.com/?v=3.6.0&c=1.1.0)
known issue: maximum skill points will always display 121 and maximum ascendancy skill points with display 8 even though they have varied over time.
If you have data from before 1.1.0 let me know, and I will add it. I have a lot of json from pre 1.0, but I don't have any assets.
https://github.com/EmmittJ/SkillTree_TypeScript/tree/master/data
nice tool, awesome being able to look at the old trees
@chrome topaz just learned about pob.party, awesome site you made. Just added a link to it in PoBPreviewBot replies
did you make the reddit bot?
yes
ah cool, @hazy fog one tooldev please


