I also let a few mobs beat on me for a while to verify it 😛
#tooldev-general
timid hemlock
obtuse citrus
You mean unknown6 dont you
timid hemlock
Nope, Unknown5
timid hemlock
Yeah 😃
timid hemlock
Also, in MonsterTypes. Unknown2, Unknown3 and Unknown4 appear to be Armour, Evasion, Energy Shield as a percentage of some base amount
I don't know how to calculate the base for armour and evasion, but for ES it's 40% of base life
obtuse citrus
Armour and EVA are in the defaultmonsterstats
so it probably just multiplies those
velvet fog
Unknown2 maybe BonusStr
timid hemlock
As for Armour, it's not in DefaultMonsterStats
obtuse citrus
What do you think stat4 is then? I always thought it would be armour but wasn't able to verify it
timid hemlock
I have no idea
I've done a bit of analysis on it and came up empty
It doesn't scale the same way as anything else
Including the armour from Zombies in the spreadsheet
velvet fog
Stat4 marked as Life, Stat6 marked as AllyLife for me
AllyLife for summons, Life for normal Monsters
timid hemlock
Oh? Interesting
obtuse citrus
interesting indeed
timid hemlock
Makes sense, I guess
velvet fog
Stat4 was confirmed by some tools before
obtuse citrus
yeah I was about to ask, someone told me you could get monster life from memory reading on the client, which I didnt want to do since I dont want to get banned for just trying to figure that out
mortal bone
@timid hemlock every time you add a feature I get asked when I am going to implement it lol
timid hemlock
😛
torpid mesa
@timid hemlock pob doesnt let me set stack size for explosive arrow
like 1 stack vs 5 stack, just always assumes 1 stack
side feature request/ tell me im an idiot and dont know how: single skill use hit damage+ ignite would be nice
seems to only have ignite dps + hit dps
cursive bluff
is the dev for poe.ninja in here?
seems i am having some issue with staying on HC Legacy. whenever i click on a different category, it changes back to Legacy
same with other leagues.
seems category links always to regular challenge league url
timid hemlock
@torpid mesa EA fuse count is a bit tricky to handle; I don't really have a suitable way of scaling the explosion damage, aside from adding a hit damage modifier (which is a bit ugly)
But there's some changes to the program's modifier system that I'm planning to make soon which will make it trivial to do
So you might see something coming along in the next few updates 😃
torpid mesa
:> sweet
even more complicated is that stuff like added fire from boot enchants applies to the ea explosion
and gets scaled by fuse count
timid hemlock
Yeah, that's one of the main problems
I can scale the base damage easily, but I don't have a way to scale the combined damage correctly without adding a modifier of some sort
rotund ember
how do complex mods appear in the api?
for example: (2-3)% chance to Freeze(12-16)% increased Freeze Duration on Enemies
are they like 2 mods or 1
rotund ember
ok thanks
undone hull
hey fellas
i was trying to import currency stashs i have into google sheets
i was able to import poe.ninja exchange rates using an importjson function
i was just wondering before i went through hours of labor if anyone knew of something similar or had done something like that already
velvet fog
@compact isle Is it possible get different translations from forum api (/character-window/get-stash-items)? https://redd.it/65ts1v
hollow oar
hey guys, is the GGPK defragmenter still useful ?
velvet fog
for disk saver, yes
unique night
3.0 gonna be a lot bigger
compact isle
@velvet fog replied
odd aspen
@chrome topaz Is there a way to accurately search for aggregate resist rolls on a ventor's gamble in poe.trade?
No matter what I try to do, it doesn't seem to take into account negative resists (presumably since -#% to res is a different mod than +#% to res)
I search for a range of between 5-15% total Elemental Resistance, but it shows rings like -18 fire, -24 cold, +14 lightning
Would be really cool if it would take into account negatives
(here's the search for this example btw: http://poe.trade/search/esaharikutodub)
mortal bone
+/- res are the same mod. Total elemental res is being calculated correctly; if you want each res to be positive then you will need to add them each with a min of 0
chrome topaz
yeah it's not supported atm
obtuse citrus
if the api gave you the stats then that would be not a problem D:
Given means of translation, you could also show it in all leagues poe supports
velvet fog
@desert needle Forum Tracker Bot only shows
"...(more)
for some posts
torpid mesa
most*
delicate ore
@timid hemlock enchantments don't show up for triggered spells, you have to select All Skills to see them
timid hemlock
Actually, it was only picking up the first active skill in each socket group
That's fixed in the next update
desert needle
yeah that happens with the tracker on some quoted html that starts the posts because it only gets the first like 128 chars then strips the html, I should have it pull more, strip the html, then post the first 128... but really it just needs more work on parsing the forum quotes
crystal hamlet
So I've got my extended filter syntax tool into a releasable state now. Made a windows installer for it and added something to help ease of use for filters that are released to other players 😃
https://github.com/skaufhold/poe-superfilter if anyone's interested in trying it out
pseudo ocean
what ever happened to the guy doing a price check program? last i heard, he got as far as jewels
torpid mesa
@crystal hamlet it looks real nice. I'll prob play with it.
rotund ember
in the api how are leagues named?
i found only Standard and Hardcore
do those include the challange leagues?
rotund ember
no I use this: http://www.pathofexile.com/api/public-stash-tabs
mortal bone
Oh, you will have to use "next_change_id" to continue on
mortal bone
you will eventually hit the challenge leagues
rotund ember
and I only see Standard and Hardcore
mortal bone
that is a river of every league
rotund ember
I dont know how they are named
rotund ember
even old leagues?
mortal bone
old leagues won't be found
you won't get the name scheme that is
you shouldn't need to know how they are laid out though
if you come across a league you haven't seen add it to your list
and move on
rotund ember
ok thanks
compact isle
@rotund ember the league names should be the same as in the http://api.pathofexile.com/leagues?type=main call
just for future reference
mortal bone
@compact isle he was looking for past leagues as well. I gave him that link earlier and that wasn't what he was looking for I guess
compact isle
ah
velvet fog
@compact isle Passive Skill Tree json still not fix 😫
compact isle
fixed locally so it'll be out at some point
mortal bone
It's generally fixed at patch time
sly oracle
@mortal bone what tools do you work on?
mortal bone
PoESkillTree, MTX Finder, and http://poeskilltree.com/compare are ones that are published
I mess around with a lot of other things though
Do you need help with something specific?
chrome topaz
@wind garden i restarted some stuff so maybe it's fixed now
sly oracle
Just curious.
mortal bone
Ah, ok
unique night
i would guess spawn relate to the chance of dropping and the generation for chance of specific mods to appear on the item
obtuse citrus
no, it's just adjusting for existing mods on the item - to make spell caster or weapon mod wands/daggers more likely and reduce the amount of "mixed" mods. It's just a second column so it can be multiplicative with the other weighting, so it doesn't mess it up.
brisk flax
@pseudo ocean I was working on a price check program, tried with jewels, then with accessories, and couldn't get the data I was working with to produce consistent results
brisk flax
And the more I thought throught he problem, the more pessimistic I got - ideally I'd only price items that sold, but determining a rubric for "this item sold" required making huge assumptions in one direction or another
brisk flax
They're insane, yeah
And so at the end of the day, the dummy estimator (guess the mean every time) had mean squared error of ~17 chaos, and the best estimator I put together could produce MSE ~15 chaos
pseudo ocean
lmao
desert needle
yep, I started down a similar path and ended up giving up for the same reasons, even trying to get a clean training set is extremely difficult
the variance in pricing is just crazy
and assuming something taken out of a stash = sold just throws things off even more
brisk flax
yep! I think if I really, really wanted to produce a reliable estimator, I'd enlist experienced, reliable players with a good track record selling rare items at a profit (ideally people who spend a lot of time flipping rares), and have them price items
which is absurdly time-inefficient, but I'd trust the numbers more than what stuff is listed for in people's stashes, without any indication of whether it sold
desert needle
even trying to predict pricing for unique items with ranges gets really complex because sometimes a 40% abyssus is 10x a 41%, sometimes it's 2x, etc., you have to trim so many outliers - and there's a ton of good data there!
I think a lot of rare pricing is like dark voodoo + luck + timing
brisk flax
yeah, I tried for rares first because I knew it was a difficult problem (relative to uniques), but... yeesh.
desert needle
hard to take elasticity into account as well (desire of seller to make a profit vs sell the item, flipping, popularity of builds, etc.)
brisk flax
also there's no downside other than opportunity cost for listing a rare that doesn't sell, which leads to a muddier data set
brisk flax
there's not like a 1alch "listing fee" or whatever that leads to (in addition to the lack of item variance) much more consistent and predictable pricing in World of Warcraft auction house listings
and we're never (under current design philosophy) geting a true auction house in PoE, so I gave up.
desert needle
if we could actually know that an item was "sold" it would be more meaningful, but it's still so difficult
like if I find a perfect ring for my build I don't GAF if it's 1c or 10c I'll just buy it, doesn't mean it's worth 10c though
brisk flax
my guess, based on comments Chris has made re: auction houses, is that solving item pricing would harm the game's economy, in the view of GGG - the market is already too efficient, in their view
desert needle
yeah, I think that's the case
it bums me out because I think instead of taking on the challenge of solving that problem they are just ignoring it
brisk flax
so I don't foresee any changes that would make it easier to quickly and accurately price highly variant items - that would only help people get upgrades even faster
desert needle
I feel like there are a lot of different theoretical ways to create an instant-style transaction system without the problems of the Diablo/WOW auction houses, especially since we know what the main ones are
brisk flax
I mean, I can accept their argument for what it is - "we balanced difficulty around inefficient trading, and already that rascal xyz has made trading much more efficient than we planned on" - and I think there are solutions (grifts, honestly), but I get why they might not like those solutions
desert needle
and it would be awesome if GGG were the company to solve that, huge win for them
ya
one of the common arguments I've had at work in my career - if users are doing something in a different way than you expect/want, en-masse, then they aren't doing it wrong - you, as a developer, are. Adapt and support them.
brisk flax
for all of d3's flaws, the greater rifts system is awesome for giving people who feel like the game's too easy an outlet - obviously you start to run into "okay now it's mathematically impossible for me to progress further", but it allows for easy upward scaling
yeah. the same thing comes up with flasks - the reason people make macros is that it's the most efficient way to play, given how flasks operate in the current environment. and I'd bet that 3.0 nerfs flask uptime significantly to handle that, so that the incentive is gone
but I don't think there's a good way to remove the incentive to trade efficiently, so either they'll have to make it less efficient (charge "listing fees" to put stuff up for sale), or reduce the harm they see in efficient trading (by providing more difficult experiences for advanced players, so that they don't make inexperienced players who don't trade face an absurdly difficult experience)
I typed a lot about game design philosophy when I was trying to say "nope, I gave up on that estimator", ha
desert needle
it'll be interesting to see what happens in the 3.0 beta
the Awakening beta was f'ing terrible IMO because trading was nearly impossible, so you basically played everything SSF
the content balance was totally wack
that's why you had stuff like the Malachai fight getting so tuned down, then at the first release so many people struggled because it was early league so they nerfed it more, now it's like an AFK fight if you have any sort of twink gear while leveling/etc.
I remember being like level 80 with 50% resists and no life struggle bussing my way through an EQ build with like a 300dps 2h heh
"this skill sucks"
unique night
add auction house and add durability to all gears without the ability to repair and add orb of durability that remove the durability making it permanent so only the good shit get to last forever and you rebuy trash items. so this prevent the economy flatlining in a week in a fresh league and inflation.
lethal flume
Making trading less efficient with listing fees will only lead back to good ol' forum shops, unless item linking on the homepage is also removed at the same time.
sharp minnow
I think, people would use an item until it was broken and them scam others. Like leaguestones charges.
unique night
uh it would make sense to make it untradable/disappear once the durability is zero. and about the leaguestone charges, blame the buyers for not reading
unique night
ohhh i see. yea ggg already solved this by requiring to hover each item and cooldoown if item is removed then readd. again people are impatient.
wind garden
@timid hemlock how do you do self-update in PoB? It seems like the app updates itself without ever restarting. Is that just how it looks or do you actually manage to update without a restart?
mortal bone
Well, it is in LUA, so I would assume that he can reload the scripts without actually having to reload the program
sly oracle
Yeah, it's pretty simple to self update without a restart.
You just design it from the start
wind garden
ah LUA
was thinking it might be C#
which makes things more difficult :/
@mortal bone does your tool update? or does the user have to download a new installer?
mortal bone
It updates
But it updates by installing the new version
Basically, you initiate the download, start the installation, close the current application, finish installing, and then reopen after installation
wind garden
k, pretty standard
timid hemlock
Emmitt is correct
All the scripts and data get reloaded, but the window and renderer stay running
The only time that doesn't happen is if one of the binary files needs to be updated, in which case you'll see the window close, then reopen
But that doesn't happen very often
vestal axle
Anyone know which API i use to crawl through forum like poe.trade?
mortal bone
Public stash tab api, but if you specifically want forum posts you will have to wrote your own crawler.
unique night
here's why you should not bother crawl the forum https://web.archive.org/web/20150820192910/http://exiletools.com:80/blog/2015/08/07/dealing-with-acquisition-while-running-your-own-indexer/
lethal flume
So you would prefer that trading stays exclusive to the paid-for premium stash tabs? Locking out all players that doesn't want to pay for those tabs out of the trading game (excluding the lovely trade chat)?
ebon oasis
what if everybody got 1 premium tab by default for trading
lethal flume
Bumping threads is now a thing, If your site is serious enough you'll get the online ap
get access to the online api*
What's left of the arguments is then the cumbersome crawling basically.
Which might be a fun exercise for some.
unique night
it's only 5 bucks.. if that's a problem then there are bigger problems to worry about
lethal flume
Good reputation, is important isn't it? Isn't this the nice guy of free2play games, where content isn't locked out? Many enjoy the trading game.
velvet fog
GGG needs money, so they only provide public stash api
I don't think they will provide public forum api like stash
vestal axle
If we're using public stash api, how can we know if someone selling it or putting price? Is it the acquisition app?
mortal wave
Anyone good at AHK that could write something similar to the poe.trade macro that could do the following? Or know of something that does these things?
- Display the number of mods on a map
- Display whether mods are prefixes or suffixes
- Display how well rolled those mods are
2/3 would be on items... i don't think they matter on maps
desert needle
the only way to solve the forum thread crawling problem is to get a pull request to acquisition approved that submits notifications to a second (your) service
otherwise it's effectively an unsolvable problem
torpid mesa
Just buy every single player a premium stash tab
chrome topaz
the first step would be to send the said pull request instead of writing a blog article
torpid mesa
Would you accept such a request?
mortal bone
Spicy....
desert needle
haha checkmate
for the record that blog article was never targeted at poe.trade, it was targeted at GGG. Ultimately I don't think that's a fair thing for the community to solve.
There was some work being done awhile back on a forum api for bumping that would be accessible to everyone, dunno what happened with that or if it was ever done
torpid mesa
Also what's with that Elemenopy guy in the comments that acts like he spent 1000s of hours on a project then dumped it cause someone was rude
sly oracle
accounts getting a free premium tab just encourages botters.
with free premium tabs, you can fully automate price manipulation without spending a single dollar.
It opens up a huge can of worms
torpid mesa
I was joking that the guy making the indexer spends the money
desert needle
I can't really imagine a typical botter cares about dropping a couple bucks
since presumably they're doing it to make money anyway
pseudo ocean
@ebon oasis https://youtu.be/Vwi6U7MojDQ?t=18m27s
talks about what would happen if everyone got one free tab
@mortal wave you might be interested in http://pathofmaps.com/
@mortal wave and https://github.com/aRTy42/POE-ItemInfo/releases
chrome topaz
@torpid mesa sure, you wanna contribute some?
torpid mesa
Neat and idk I only use it for searching my 500 STD tabs but there were some things I wanted out of the search that it wasn't doing
mortal bone
I have a feeling you would have to prove yourself as a service
torpid mesa
I meant acqusition
mortal bone
mhm
mortal bone
I am thinking about writing on in Go
need to learn the language, and it seems like an easy project
torpid mesa
Depends on what constructs it has for data storage and binding imo but a clean and simple open indexer could be fun
mortal bone
I definitely can't afford to run one. It would just be a hobby project
torpid mesa
Clean and simple code wise, lots of people come in asking how to scrape Poe.tradr
A sample project would prob be fun and cull a lot of that scraping
mortal bone
definitely. I am not sure why you would want to scrape poe.trade anyway. You only get 99 results
desert needle
I remember a time when you could search for a unique item in a league and not actually hit 99 results.... for like, a lot of items
_>
unique night
there is already a indexer on github you can run yourself and get as many results you want. tradeoff is the bandwidth
mortal bone
what language? or link at least? lol
unique night
uh cant remember.. watching prison break atm
desert needle
it was python iirc
https://www.reddit.com/r/pathofexile/comments/5zap2r/here_is_my_python_trade_sniperindexer_now/ 404 tho
mortal bone
Damn
torpid mesa
rude
mortal wave
@pseudo ocean Thanks for those links! Only problem is that the trade macro and the item info keep openning at the same time, regardless of the macro I use :[
pseudo ocean
@torpid mesa https://github.com/licoffe/POE-Stash-indexer as well
torpid mesa
Js :| but it links to ng version which is c++. Pcewl stuff
velvet fog
how to know a essence mod is prefix or suffix?
pseudo ocean
velvet fog
@pseudo ocean thanks, strange place
little pine
is there a way to tell which buffs/debuffs are on a player?
little pine
funny.. I meant like a log.. trying to make a notifier for when my witchfire brew is on and off
mortal bone
oh, nope no way of telling
little pine
I wish there was.. would sure be nice to put tiny icons around my character that told me what was going to kill me next
mortal bone
they already do that for you
most debuffs show up on the model
unless you are wanting more of the wow style effects
little pine
distinguishing 1 visual effect out of 10 that could be shown at any given time is impractical
velvet fog
maybe poe 4.0 will have a combat log for that
little pine
but yea, I'm a mythic progression raider on WoW.. its nice to have everything pertinent centralized. was trying to see if I could do that with buffs/debuffs/flasks
maybe with some sort of graphics processor that see's the buff/debuff icons and parses them.. but idk
velvet fog
poe has no support on addons
torpid mesa
anything that does what you are asking is bannable
little pine
huh
how is displaying an icon in a different place bannable when it wouldn't interact with the game?
torpid mesa
" graphics processor that see's the buff/debuff icons and parses them.. but idk"
little pine
meaning looking at the icons in the top left corner.. determining what the icon was.. and displaying an additional icon elsewhere.
torpid mesa
ggg is super strict, something like that can easily get you banned. i woudlnt risk it
little pine
what I'm suggesting wouldn't even be detectable because it would be like watching a video feed, and overlaying something on top of the game in a seperate program. think mercury trade but with buff/debuff icons.
torpid mesa
mercury trade doesnt read the screen/memory of the game
torpid mesa
thats not the part that matters. look you can do whatever you want but ggg doesnt allow things that seem like they would
little pine
if looking at the game is bannable, everyone playing the game would be banned.
delicate ore
@timid hemlock can you display curse effectiveness in the calcs tab?
little pine
regardless, it doesn't matter, I'm not going to even try to make this. just an idea that got squashed once I looked at the limited logs
torpid mesa
making snarky comments to me doesnt change their policy just be aware ggg hates modding or any kind of inspection
delicate ore
and could you show aura effectiveness as well?
unique night
it's not enforceable tho due to no anticheat however ggg can track the metadata instead... behaivor and interaction and such
timid hemlock
I suppose so, if I can figure out a good place to show them
little pine
clearly my idea wasn't described properly. It had nothing to do with modding, metadata/behavior/interaction/sniffing/etc.. purely cosmetic "see icon, put new icon in middle of screen".. if you're familiar with WoW addons, it would be like WeakAuras. but I'll have a look through their policies again to avoid further confusion with what is acceptable.
torpid mesa
check out some of the ahk threads, plenty of stuff other games would allow but they dont.
unique night
there is no UI modding so best you can do is use ahk with pixel detection and dont advertise it but boss fight is basic compared to wow so tthe benefits is not really there. just dodge, slam flasks, attack, repeat.
little pine
the purpose of it would be more for maintaining flask uptime
a major problem is people just spam flasks and waste charges
unique night
yea ppl goo pathfinder and spam flask and it recharge like immediately after clearing a pack
torpid mesa
flask management is intended to be a core mechanic though
little pine
correct, I just want the notification to be easier to see
torpid mesa
could be nice
little pine
looking in the top left with buffs coming in and out detracts from the gameplay, which in HC also means you can die.
torpid mesa
maybe ggg will finally split temp buffs and perm buffs
obtuse citrus
there is no support for UI changes in PoE and it sounds like there won't be because they dont want to "dilute brand identity". They seem to be touchy about reading memory or screen contents (understandibly, because it can be used to bot), so I don't think there is a way you can do this.
You can email support though and ask if it's okay
little pine
I think what I'm going to do is just map it to the keybind, and use manual timers
like, this flask is up for 6.00 seconds, key was pressed that this flask is on, display icon
unique night
find out flask duration, press flask, sleep for duration, press again. no need to scan for buff icons. look up chickenscript, autoflask for some coding idea but that's the basic of it
obtuse citrus
If you do the pressing with a script it's bannable though
they haven't been allowing autoflasking for a few years now
unique night
but it's not enforceable unless you are a streamer as those are banned
little pine
player presses button, app detects button press and displays icon. no automation.
obtuse citrus
@unique night Unless you emulate "human" behaviour in your script you could potentially track this on the server side
unique night
they do
split pollen
desert roost
No, it's not. #1┃endgame-help would be the best place for future reference.
torpid mesa
@timid hemlock what graphics toolkit does path of building use? is the code for the runtime available online?
pseudo ocean
you also have c# bitmap pixel detection which is impossible to detect but against the ToS ? 😛
timid hemlock
@torpid mesa It's all custom built; I haven't published the runtime source anywhere yet for various reasons, but I can provide it to you if you want to look at it
torpid mesa
@timid hemlock please do, i was hoping to see how hard it actually would be to make it cross platform. the offlineskilltree requires redoing the entire ui since mono doesnt support the toolkit it uses
mortal bone
PoESkillTree works in wine. The runtimes have been updated quite a bit in the last year @torpid mesa
stone lagoon
um, how would I go about getting all of the stash tabs from the api? is it gigs and gigs of data?
pseudo ocean
@stone lagoon http://pathofexile.gamepedia.com/Public_stash_tab_API
@stone lagoon tl;dr make a request to http://www.pathofexile.com/api/public-stash-tabs?id=220-1652-744-1341-230 , get next_change_id from the resulting JSON, and make a request to the next_change_id by replacing 220-etc.
@stone lagoon http://poe.ninja/stats for stats relating to public stash tab API
stone lagoon
ok, so that ggg api download means that one text file containing all of the stash tab data for every character in the game would be 9 terabytes in size/
desert needle
that's more likely to be a historical counter of all the api pages he's downloaded
stone lagoon
That's kinda what I was hoping
desert needle
a fresh start from 0 would not be that big, but it will still be many gb (probably more like 5-10)
(fresh start from 0 doesn't replay historical stuff, it just gives you everyone's stashes starting in that moment)
also make sure whatever client you use enables gzip compression which has a significant impact on the total network transfer size
stone lagoon
I'll check for that, thanks!
torpid mesa
@mortal bone icic
mortal bone
Although, I personally haven't checked for some time now, but someone normally creates an issue if it stops running on wine
delicate ore
@timid hemlock can you make a Flask section in the Calcs tab?
for things like flask effect, duration, charges gained, etc?
oh nevermind, just hovered over a flask
velvet fog
@compact isle I got an atlas map at http://web.poecdn.com/public/atlas/Atlas.png, but some nodes are in wrong place, ex: T8 Cemetery.
compact isle
yeah that one doesn't auto update or anything
velvet fog
Is it possible to get a new one?
compact isle
http://web.poecdn.com/image/Art/2DArt/Atlas/Atlas.png is this one up to date?
oh lawd discord no
velvet fog
wow, thanks!
compact isle
np 😄
mortal bone
@compact isle putting <> around the link stops the preview
compact isle
huh did not know that, thanks Emmitt!
mortal bone
sure!
vestal axle
If we download all the json received from public stash api and next_change_id, how many mb will it takes? Is there any way to get live API, so we only process new item ?
vestal axle
So latest next_change_id is the newest update happened?
ebon oasis
i didnt do enough stuff with the api to answer that confidently
deft jolt
next_change_id from poe.ninja isn't exactly the newest, but it's close enough
grab it from there first and then work up the chain
also, the amount of data it downloads depends how many players are actively changing their public stash tabs
right now it's slow, only 380KB/s
vestal axle
Nice! So it means i just need to start from poe.ninja stats id to get close enough to live right. Thanks alot
ebon oasis
if someone wants to do stuff with api, is there a need to get all the historical data or do you just start with whats there now and go form that?
deft jolt
depends what you want to do
ebon oasis
right, so as the "next_change" name suggest, the response only contains what changed from the last one
deft jolt
i suppose you could put it like that
here's how it works: whenever someone changes something about an item in a public stash tab, the whole tab is pushed through the api
including every item that's in the tab
ebon oasis
thx
velvet fog
@obtuse citrus how GenerationWeight in Mods.dat works?
obtuse citrus
It should be multiplicative with the other weights, otherwise just as usual
it would seem the mod generation is procedual anything else makes no sense
Since no items have has_attack_mod or has_caster_mod tags by default the first mod is entirely random
if the first mod added one of these tags, if a mod with the relevent generation weight is a potentical candiate to be rolled, it's weight would be adjusted accordingly
For example
Mod 1: Weight 1000, adds "has_attack_mod"
Mod 2: Weight 1000, generation weight "has_caster_mod 100, has_attack_mod 65, default 100"
Mod 2 would be adjustd to have a weight of 650 instead.
mortal bone
has_attack_mod is giving more weight to attack mods so it needs to be multiplicative
your equation is correct
frigid badge
deft jolt
https://gyazo.com/897e0fe7f2cf05a068be93fe5c5bbb5e
generating poe.trade urls was a lot easier than i had imagined
mortal bone
just a base64 of the form input?
deft jolt
hm? don't know. just fed some json data via HEAD and got the data from there
ive got no background in http though
wind garden
@mortal bone have you run PoESkillTree in wine before?
I know you've talked about it, but wasn't sure if you've tried it yourself
mortal bone
Some people here tried and got it working
I don't have a Linux machine running anywhere, so I haven't personally tried it
wind garden
good grief, wine is a nightmare
hmm it looks like the primary mirror used to download .net framework 3.0 is down
unique night
no point making it work on linux/mac since you can't play the game there lol
wind garden
i'm actually trying to run something else on wine
i just remembered emmitt talking about wine
i can't get .net 3.0 to install at all in wine
which is a dependency for .net 4.5
mortal bone
Yeah, I am not sure how they got it to run. They said there wasn't any problems
deft jolt
Got a question about python. I'm trying to download the stash api using "r.iter_content(chunk_size=1024)" but cannot manage to join the chunks up so I could parse the stash tab data.
r = requests.get(url_api, params = {"id": next_change_id}, stream=True)
for chunk in r.iter_content(chunk_size=1024):
if chunk:
# First chunk contains next_change_id
if count:
count = False
# Find next_change_id
fixed = re.findall("\d*\-\d*\-\d*\-\d*\-\d*", str(chunk))[0]
QUEUE.append(fixed)
^Parts of the code I'm using
Using r.json() after the for() loop returns "RuntimeError: The content for this response was already consumed"
I more or less understand why this is happening but could anyone help me find a workaround?
obtuse citrus
Partial code isn't that useful, can't really see your r.json or the for loop. Is requests a custom library? Either way I'd assume it's at the end of the stream, if the interface supports it you could either go back to the start of the stream (r.seek probably) or store the contents in memory [which though makes it not that useful to use a stream in the first place I guess]
pseudo ocean
requests is a python library
deft jolt
I'm experimenting with stream to get the next_change_id faster than I would by downloading the whole API data, parsing it as json and then getting it from there. The request takes place in a daemon thread so it doesn't really impact performance.
The problem I'm facing is when I do the whole "r.iter_content(chunk_size=1024):" thing, I do not know how to get the rest of the stash data from it
Any way I can add the chunks back together or buffer it so I could convert it to json?
deft jolt
I managed to get it working by adding a byte buffer, decoding the buffer and converting it into json
deft jolt
WITH STREAM
Data:
Sending request...
Change_id from stream: 0.6200346946716309
Request finished: 2.8591628074645996
Starting decode+parse...
Decode+parse finished: 0.05800271034240723
Total runtime: 2.917165517807007
Sending request...
Change_id from stream: 0.6350359916687012
Request finished: 3.04217529296875
Starting decode+parse...
Decode+parse finished: 0.06800341606140137
Total runtime: 3.111177682876587
Conclusions:
1. next_change_id is found 0.6 seconds after sending request
2. Total download time before stashes can be parsed: 3.0 (sec)
WITHOUT STREAM
Data:
Sending request...
Request finished: 3.4101955890655518
Starting parse...
JSON parse finished: 0.0630025863647461
Total runtime: 3.4751980304718018
Sending request...
Request finished: 3.3311920166015625
Starting parse...
JSON parse finished: 0.05900311470031738
Total runtime: 3.39019513130188
Conclusions:
1. next_change_id is found 3.45 seconds after sending request
2. Total download time before stashes can be parsed: 3.45 (sec)
* All floats are (sec)
some data from improved delays
simple ravine
Aren't you going to get throttled anyways?
deft jolt
Right now it looks something like this:
1. Start downloading data
- 0.6 sec delay
2. new next_change_id is forwarded to a new thread while this one continues to dl/parse data and find items
3. new thread sleeps for <delay> then starts from the top again
Vs the old one:
1. Start downloading data
~3 sec delay until whole data package is downloaded
2. new next_change_id is forwarded to a new thread while this one continues to dl/parse data and find items
3. new thread sleeps for <delay> then starts from the top again
Difference is I don't have to wait for 3sec every single time to be able to start downloading the next package
A base delay of 1.1 sec is enough to avoid being throttled at this point in the league
I also have implemented a dynamic delay calculator that should help me avoid sending too many requests
desert needle
That's pretty smart, it will make a huge difference when the league is busy.
I was doing something similar, except dumping the stream to a message queue as it came down to be handled by another process.
I'm not 100% sure it's done the same way, but the way it used to be setup, the rate limiting was a X req over Y seconds rate limit
so you can check the time of the requests, and if the request takes more than 2-3s to download, you can immediately make another request when that finishes without getting rate limited
the only time you need to rate limit it is when the updates are super small and download very fast
also I think python requests does it by default but just because a lot of people have missed it, make sure you are accepting gzip encoding in your request, massive difference in performance
deft jolt
Hm, I recently improved my delay stabilization code.
It attempts to hold the delay at about 1.2 seconds (+-0.1 sec) between each request.
Starting from poe.ninja's next_change_id gives me an initial (total) download time of 3-5 seconds.
Working up the stream with that takes about 2-3 tries and yields an average download time of 1.4 seconds.
The overlap should be around 0.1 to 0.3 seconds
I've ran my code for the better part of around 24 hours. No throttling has occurred as of yet.
Seeing how things are going, I might decrease the delay of 1.2 sec until I find a suitable, non-throttleable limit.
mortal bone
from what I know you should be able to do 1 per request per second without throttling
depending on the time it takes to hit the server you could get close to that 1 second mark
frosty bloom
how do make indexer to scam noobs
deft jolt
Appears to run fine when sending requests every 0.8 to 1.0 seconds
mortal bone
Hmm, you may be able to do 2 requests per second. I remember some talk about that here, but I thought it was 1
deft jolt
Appears to be true
However I'd rather not stay that close to the edge, 0.9 sec delay is more than enough
elder hearth
What happened to TheSmellOfVictory's script on github?
I really liked his bash version, and after that he made a python which i'm working on...
pseudo ocean
@elder hearth there was a discussion about this, and I believe the consensus was the he took it down and that no one had forked it
elder hearth
@pseudo ocean what kind of discussion? About being unfair or something like that?
wind garden
@fickle yew do you know why Presence of Chayula is listed as 41ex value on poe.ninja?
pseudo ocean
@elder hearth discussion that it was down and no longer available even via cache searches on google
obtuse citrus
what kind of script was it?
torpid mesa
There are other people with similar scripts though if you Google Poe indexer source or something
pseudo ocean
i believe it was the one that kept multiple poe.trade live searches open without killing your browser
elder hearth
@obtuse citrus hey, do you know what "Generation Weight" does? I know Spawn Weight, but Generation? and it only appears with has_attack_mod / has_caster_mod... i'm trying to play around with a crafting simulator, also do you have any idea how to come up with how many mods Alteration Orbs (1-2) or Chaos (3-6) may generate?
woeful sphinx
@elder hearth generation weight is used to make non-attack/caster mods less likely to roll on an item if it has attack/caster mods. Item generation was changed so items are more likely to roll related mods (attack/caster) if there is already one of those on it. Not sure if that applies only to complete generation (e.g. drop or Chaos orbs) or also on crafting additional mods (e.g. Exalted orbs).
elder hearth
@woeful sphinx i'm thinking about multiplying Spawn Weight with Generation Weight, i tested here and all values return an Integer (no decimals or some weird calc, looks like it fits perfectly)
lets say i have a Staff with an already rolled attack mod, and then i decide to use a regal and eventually rolls a Prefix, so:
to roll a SpellDamageOnTwoHandWeapon5 ("Occultist's" prefix) on a Staff wiki says:
200 weight
but Generation Weights says if has_attack_mod = 50 (i think it is like %), so
200 * 50 / 100
new weight = 100
woeful sphinx
makes sense if it works that way. but I don't think anyone would be able to 100% confirm as GGG generally tries to not publish that kind of stuff
elder hearth
@woeful sphinx yeah there's no way to simulate 100% like they do (we don't have the same pseudo RNG algorithms to run)
obtuse citrus
You're correct in what it does. Mods are generated procedually, even with chaos orbs
opal falcon
Hi everyone, I am looking for tool for Stash Sort, Do you know any tool and Is that accept to use in POE?
desert roost
That tool would not be legal.
opal falcon
@desert roost Ok I see now
So how about the tool for modifield color of items?
Sorry I am a new player in this game
desert roost
That's a supported modification
I'm assuming you're talking about lootfilters?
Something like this? https://www.pathofexile.com/forum/view-thread/1246208
obtuse citrus
there is also a list on the wiki https://pathofexile.gamepedia.com/List_of_item_filters#Filters
obtuse citrus
@compact isle will there be a copy of the trade api for the beta?
pseudo ocean
forum scraping hype
gritty olive
elder hearth
@obtuse citrus GGPK Viewer (PyPoe) looks like it is not parsing folders that contains space, like Audio/Sound Effects/
they don't show at file manager
obtuse citrus
It parses things just fine, the folder you've mentioned doesn't exist anymore since they've been using FMOD. They've moved audio effects into bank files
obtuse citrus
to my knowledge there isn't a tool out there that can open them
I suppose someone who has some knowledge with how FMOD works and probably some reverse engineering knowledge could come up with somehing
desert needle
in the last beta they didn't even support forum based trading :/
obtuse citrus
@desert needle it's because the forum is integrated with their backend AFAIK
desert needle
theoretically it would make sense to have a beta version of the stash tab api at least just to make sure it works right though
yep exactly
elder hearth
forum based trading
damn,
so Mercury channel scan will rule
obtuse citrus
but I feel like they could setup something like beta-api.pathofexile.com
desert needle
the hardest part of poe without forum/stash based trading is finding rares to fit a build. Uniques are generally easy in chat, but spamming chat asking for someone with a 40% fire resist +70 life ring is just impossible.
obtuse citrus
yeah
desert needle
(and you can't scan those stats)
elder hearth
yeah, you cant :S
obtuse citrus
I hope they have trading, they've changed ES and ES gear has always been harder to get self found IMHO
desert needle
legally at least :x
elder hearth
elder hearth
@obtuse citrus it's my first time trying to play around with "real" programming (I'm used to VBA for office stuff, so...) and I'm using latest Python and PyQt, would it be annoying for people to download those package or it doesnt matter, also should I use Widget or QtQuick (QML)?
bronze elbow
??
ebon oasis
<@&174997701513969665>
bronze elbow
<@&174997701513969665>
pulsar moat
Boom
ebon oasis
<3
mortal bone
Bingo
bronze elbow
-kisses
obtuse citrus
@Uncandango: Most people don't like doing more then 1 click to install anything, though PyPoE is directed at other developers and experienced users, not average joe. If you do releases (which I dont do yet) you can bundle the stuff it needs with a python install though.
Widgets are probably better for an desktop app, but just might want to google that question there are a number of good responses out there.
Also on a note in PyPoE I use pyside even though it's outdated instead of PyQT because PyQT is GPL licensed (and not LGPL like QT itself), which essentially means that your project must be GPL licensed as well.
There is also TKInter for making interfaces, it's bundled with python, but my experience with it is "dont use it unless you hate yourself"
mortal bone
Can confirm have hard TKInter and I wanted to hurt myself after
summer stratus
Is it a bot that has gone cray-cray?
elder hearth
<@&174997701513969665>
vast quest
ty
summer stratus
Thanks.
foggy vortex
Is the guy who makes PoB on here?
I think I have a inconsistency / clac bug to report
Basically the Unique affix on Choir pf the Storm that increased Lightning damage 50% done on crits is not accounted for in dps.
timid hemlock
@foggy vortex I do lurk here from time to time; I assume you're the one who posted about this in the forum thread? I've just posted a reply.
foggy vortex
@timid hemlock Yea thats me. Do you have paypal for donations? I dont use patreon but would like to support.
timid hemlock
I don't have a paypal set up at the moment
foggy vortex
Id imagine there are other in similar camp as me on this topic. You should consider setting one up.
timid hemlock
Perhaps; although it's not nearly as straightforward to set up as Patreon
foggy vortex
Well that's too subjective to comment on. Consider that more people use PP and than Patreon, then consider the likelihood of someone making a Patreon account they dont already have just to support your project.
PP is pretty ubiquitous.
timid hemlock
It is indeed
That doesn't stop it being any less of a pain in the backside to set up a donations page there 😃
There's a reason people like Patreon so much
foggy vortex
The prospect of money usually does the trick in motivating undesirable tasks. Not unlike the exchange everyone makes 9-5..
Obviously it's your show so do as you see fit. PoB is incredibly impressive to me. Very tasteful software.
timid hemlock
Oh I'm sure I'll get around to it eventually
But last time I looked into it I ended up giving up after spending a few hours digging around
simple ravine
@timid hemlock I'm also witnessing a thread consuming 100% CPU (1/8th of a core on my CPU) when having the tree open. I'm assuming that you're constantly evaluating changes rather than using an event driven design. I'm not familiar with lua, but perhaps it's something to take a peak at.
Otherwise, fantastic tool. Love the detailed calculations and such.
I would also agree with @foggy vortex that PayPal will convert far better in terms of donations than Patreon.
timid hemlock
The CPU usage is mainly down to the fact that the program's UI is rendered in OpenGL, and is being re-rendered ~60 times per second; and since the program's renderer isn't particularly efficient, and the passive tree is composed of about 15000 primatives, it tends to use a bit of CPU when displaying it.
Rewriting the renderer is one of those things I really need to find the time to do :).
clear lake
The way you've described it as a machine learning problem, it doesn't make any sense
given gear and items, it is already known what the dps would be
the relevant problem would be inversion
violet spire
hey
i really need to speak to a GGG admin
my account my roommates accounts they all got locked
about 6 accounts
...
there all kinda pissed
It just happened now
Could I speak to Novynn
velvet fog
@obtuse citrus PyPOE new branch for 3.0 beta?
obtuse citrus
@velvet fog I'll have a new specification for the beta specifically, I dont think I'll need to do a new branch for it
I'll be able to tell you for sure when the beta hits though
elder hearth
Hi, is there any way to relate Unique items name with their mods? I couldn't find a connection between unique names + mods (using PyPoe files).
Also is the any way to know that an item is generated corrupted? Like leaguestone, breach ring, talismans
timid hemlock
The data for unique items isn't included with the game client, and for good reason as it would allow their stats to be datamined
So associating uniques with mods has to be done manually
obtuse citrus
I don't think knowing the stats is really a big problem for ggg, more that unreleased items could be found out
and probably that have spawn chances in there
and other stuff
timid hemlock
Indeed
Unfortunate though; maintaining a current database of uniques is a pain in the rear 😃
woeful sphinx
yeah, what's in the wiki is pretty good and you dont need to maintain it yourself
velvet fog
@obtuse citrus https://pathofexile.gamepedia.com/Le_Heup_of_All , how is the requires level generated?
obtuse citrus
80% of level requirement of mods on item
elder hearth
@Omega_K2#7332 base item stats like Energy Shield, Crit chance, Damage, APS, follows the same rule as unique items? (should be get from wiki API)
velvet fog
max(item_level, max(mod_level)*80%) ?
mortal bone
@velvet fog yes
elder hearth
item_level
drop level
mortal bone
^
obtuse citrus
@elder hearth you can get all base item stats from the data. Might want to look at the wiki exporter, you need several files depending on item class
WeaponTypes for example for things like crit chance, damage, aps
on weapons
ComponentArmour.dat for ES/AR/EV
etc*
stone lagoon
does anyone know if a website that makes checks in public stash tabs for organizational purposes would be against TOS or something?
pulsar moat
can you be more specific?
stone lagoon
I want to, but I also don't want someone to steal my idea (in the 10000 years it'll take me to actually start) xD.
hmm...how to explain it sensibly.
it has 0 in game automation, it simply helps you find things across stash tabs (maybe multiple tabs depending on the search) beyond the scope of what you can conveniently find with acquisition, etc.
ya, you know what, who cares
The idea is basically just to check if you have the ingredients for such and such a vendor recipe in a set of stash tabs
pulsar moat
ah like procurement does?
mortal bone
procurement*
stone lagoon
I somehow figured if one of them did it they both would, and I've only used aquisition
ok, ya way ahead of me
stone lagoon
which doesn't surprise me, what did surprise me is that it seemed no one had thought of it
why doesn't acquisition do it too? or maybe it does and I've just missed that feature all this time
stone lagoon
I guess that makes sense
mortal bone
really there is no reason to do it through the public stash api when you can do it through the normal get-stash api
stone lagoon
ya, see, I didn't realize that existed
mortal bone
Ah
mortal bone
nope, they use your login information to pull down your stashes
stone lagoon
are there any other api's I don't know about?
mortal bone
Probably. I don't know what you know
simple ravine
0
pulsar moat
1
mortal bone
2
trim sinew
Anybody made a syntax highlitning for items you paste in Discord ?
simple ravine
7 am - looks like my 3-year-old got to the computer some how
waxen ridge
You can't syntax highlight very well in Discord, best you get is some basic code blocks and that's about it. i.e no custom CSS at all and it's not coming.. I asked, I had a bot once that worked like reddits unique item finder.. Yep fun to make but the output wasn't amazing
simple ravine
public interface IModsReader
{
IEnumerable<IMod> Read();
}
well, that's good enough for me.
simple ravine
Geez this mods data is a jungle.
I'm sure they had a good thought with it, but without any documentation this gnarly
Has someone written any notes/documentation on this that one can read to understand things a little?
simple ravine
@mortal bone No, the stash tab API is quite clear. I'm talking about the reverse-engineered mods and stats data.
Specifically, how they apply to certain item types (spawn tags).
woeful sphinx
If you are talking about RePoE, there is a docs folder: https://github.com/brather1ng/RePoE/tree/master/docs
You can get the tags an item has e.g. from the wiki API relatively easily
simple ravine
Hi @woeful sphinx. Yes indeed, I was the one opening the issue the other day regarding the spawn weights 😃
I just created an poc app to explore the data, and I think the tags are not as straight-forward in how they're used as I first thought.
Or I am just misunderstanding the meaning of the tags themselves somewhat
For a pair of dexterity based boots, I would assume, correct me if I am wrong, take the modifiers that contains a spawn tag with a value > 0 on default, armour, dex_armour, and boots?
@woeful sphinx Would you quickly describe the importance of the ordering of the spawn tags perhaps? That would be very appreciated.
woeful sphinx
You would not take all mods with any matching tag with value greater 0. You would take all mods where the first matching tag has value greater 0.
simple ravine
You mean for the actual weight itself?
woeful sphinx
Value greater 0 for the matching tag' s weight, yes
simple ravine
What also confused me later on after our brief conversation on GitHub, was "Where are all the boot mods at?", Like the elemental resistances and so on. And my current assumptions are that they are inherited from the armour tag
woeful sphinx
Yes, boots have the armour tag, so mods that can spawn on tag armour can spawn on boot basetypes (assuming there is no matching tag in the list before armour)
woeful sphinx
Np! Was also not easy for me to initially get what the spawn weights do
(at least I hope my interpretation now is correct)
simple ravine
Now, the tricky part will be to have the computer come up with the optimal strategy to craft an item with a given set of desired mods
Yea, your interpretation seems logically sound, at least
woeful sphinx
Yeah, that's probably not easy. I only implemented manual crafting with it, for which the interpretation seems to work
simple ravine
Right, there are some methods that could potentially work, but I think the level of commitment to get it working is higher than anyone's ambition - such as employing a deep reinforced learning algorithm (Q-learning) or with policy gradient methods.
My first attempt will be to employ a few 'known' strategies manually, and let it simulate with crypto-rng to find the least amount of currency that it would take. E.g. chaos-spamming, essence-spamming, transmute-alt-regal-exalt-loop
But also to see how big of a difference it would make to "block" a modifier in the process with a crafted mod etc, perhaps even go further with meta-crafting... not sure.
obtuse citrus
As for the latter with a more manual simulator you can tell pretty much
if you have some given mods, and remove a set of mods from the pool you can just calcuate the chances for both and compare
woeful sphinx
You'll also have to consider that there are unknown probabilites, like whether chaos rolls 4, 5 or 6 mods. Plus no one knows if the spawn weights are actually correct, they are most likely not used client side, so they don't need to be up to date.
simple ravine
@obtuse citrus Indeed, I have recorded a sample of 200 alts and 200 chaos orbs, which isn't statistically significant, but that is what I could do at the moment. Perhaps a collaborative effort could benefit a group of interested people.
obtuse citrus
yeah server values are used but
the values should be accurate afaik they keep getting updated when they release patches
I think 4/5/6 mod chance might be easy to figure out by just spamming a bit
I think alts were 50% prefix/suffix and 50% chance for second mod
simple ravine
@woeful sphinx Yes, that is for sure a likely scenario. For an instance, I was crafting some on a Steel Ring, and got a higher Phys-attack roll than was in the Item-Info AHK macro, so there are some edge cases there as well possibly. Thoughts?
One way would be to manually record, or capture the screen to then OCR, and then feed it into a database and aggregate
However that in my mind is on the edge of unethical, or?
woeful sphinx
I'm pretty sure that stat values in the data are correct. At least it wouldn't make much sense to hide them and doing that would require manual effort as it would require pushing only parts of the data to the client (the mods themself are necessary). Maybe the data in the Item-Info script is outdated.
I'm also with @obtuse citrus on that the weights are accurate, but the possibility that they aren't is there.
simple ravine
Based on 200 chaos spams on an ilvl84 diamond ring, there are the results I observed:
3 2
4 129
5 48
6 21
I think there needs to be more data to establish a better statistical significant sample though
For alts, I observed a 50% 1-mod and 50% 2-mod result. I never observed either a two-suffix or a two-prefix, what I can remember.
woeful sphinx
Chaos orbs can't roll 3 mods. You can't just count the lines on the item. E.g. rings have two item rarity mods, one prefix and one suffix. If the chaos orb rolls both of them, they'll only appear as one line on the item but are actually two mods.
simple ravine
oh true... i was quite tired when I did that.
guess I'll have to redo it and account for composite mods 😄
good catch, thanks
elder hearth
@simple ravine i would try that on a Belt (Crystal Belt is the most useful so you don't waste your chaos), because they don't have hybrid rolls
simple ravine
good tip!
mortal bone
Also, 2 prefix/suffix isn't possible with an alt/aug.
You get a prefix and a suffix on a blue item
simple ravine
@mortal bone Do you know if there are any similar characteristics with regal+exalt? i.e. Could one potentially have a rare item with 1 prefix and 3 suffixes by crafting methods?
mortal bone
Yes
woeful sphinx
I think even chaos can actually produce 1 prefix/3 suffix
mortal bone
Yeah
elder hearth
yes
mortal bone
The only special case is blue items
Also, you can have 1 stat rares through crafting
elder hearth
@simple ravine when rolling an affix you need to evaluate preffix + suffix weights (if both are open ofc), for ex: ilvl 1 Driftwood Club have 3 prefix (2500 weight), 7 suffix (7500 weight), so when you alt and it rolls 1 mod only, they are more likely to be a suffix
simple ravine
👍
elder hearth
@simple ravine are you doing it with python or something else?
simple ravine
C#
Been doing c# since .NET Framework 1.1... had to look up how long ago that was. That's almost 15 years ago 😂
elder hearth
i need to learn again about MVC / MVVM 😭
simple ravine
a lot of options out there
obtuse citrus
@elder hearth I don't think that's true
pretty sure it rolls prefix/suffix 50:50 not based on the weight
I havent tested it for chaos but it worked like that with alts
it uses the weights within a generation group
simple ravine
@obtuse citrus With Generation Group, you mean 'Prefix' or 'Suffix' in this case?
simple ravine
Also, anyone here who have some quick insights into how to query items using the mediawiki API?
(I'm looking to acquire a list of item bases with their tags)
simple ravine
Right, I'm just not very familiar with python either, so leveraged RePoE's, which I modified slightly to add the weights
woeful sphinx
If just want to have it once, you can take this one https://github.com/EmmittJ/PoESkillTree/blob/master/WPFSKillTree/Data/Equipment/Items.xml
see the Tags element
simple ravine
Excellent, thank you
simple ravine
@obtuse citrus I recalled you gave an example on reddit a while ago, and thought I'd give it a shot to use your project and python to extract the items. It's giving a warning however when trying to translate something on Driftwood Wand, thought I'd let you know.
Driftwood Wand
c:\code\pypoe\PyPoE\poe\file\translations.py:928: UnknownIdentifierWarning: Warning uncaptured quantifier reminderstring
'Warning uncaptured quantifier %s' % name, UnknownIdentifierWarning
c:\code\pypoe\PyPoE\poe\file\translations.py:1362: TranslationWarning: Uneven number of quantifier handler/value paris. This indicates an issue with the GGG file. Skipping.
Text found: ['divide_by_one_hundred', '1reminderstring', 'ReminderTextLifeLeech']
TranslationWarning)
c:\code\pypoe\PyPoE\poe\file\translations.py:928: UnknownIdentifierWarning: Warning uncaptured quantifier mod_value_to_item_class
'Warning uncaptured quantifier %s' % name, UnknownIdentifierWarning
(8-12)% increased Spell Damage
aware of the warnings
Warning uncaptured quantifier mod_value_to_item_class for this http://omegak2.net/poe/PyPoE/_autosummary/PyPoE.poe.file.translations.html#PyPoE.poe.file.translations.TranslationQuantifier.install_data_dependant_quantifiers
simple ravine
well, after playing around with it, python isn't that hard... takes me back to the BASIC era and indentations
successfully exported a list of item classes to a json file, wohoo
but I am missing the stats
elder hearth
@obtuse citrus well, i tested with near 1.5k alts with ilvl1 Driftwood Club (high suffix weight) and ilvl 11 Silken Vest (higher prefix weight) and it worked as expected
elder hearth
ilvl 1 Driftwood Club (2500 pref weight + 7500 suffix weight)
vs
ilvl 11 Silken Vest (11000 pref weight + 7000 suffix weight)
ilvl 1 Driftwood Club:
1004 alts:
533 rolled 2 mods
110 preffix
361 suffix
ilvl 11 Silken Vest:
590 alts:
290 rolled 2 mods
183 preffix
117 suffix
simple ravine
hmm interesting
do you have a good way of recording them, or just alt+notepad+alt+notepad?
mortal bone
I don't know of any system, made public at least, that records things like that
elder hearth
@simple ravine i did with autohotkey so everytime you click an item you send Ctrl+C and AHK parse your info into a txt file, AFAIK GGG allows you click and send CTRL+C
simple ravine
ah, smart
I was going way overboard in my head how to record them with video capture and ocr etc
simple ravine
I guess you can waste currency a little faster, but your method is fully adequate
for the purpose
simple ravine
well the clic, ctrl+c, click, ctrl+c
elder hearth
i did:
RShift+Click -> sends Click and after some delay sends Ctrl+C and parse
elder hearth
i think i replaced my code 😦
let me check
yeah i got rid of it :/ now i have the one i'm going to use for chaos spam crystal belt and check how many mods it rolls on avg
elder hearth
https://www.reddit.com/r/pathofexile/comments/689yze/discussion_new_poe_tool_need_your_help_with/
Therefore I made a makro for this.```yes, their main concern is sending commands to the server afaik
i.e. pressing mous-button 4 and it'll press 1, 2, 3, 4, 5
simple ravine
Steuerung apparently 😃
obtuse citrus
@elder hearth thanks for testing that out
velvet fog
@elder hearth 'ilvl 11 Silken Vest (11000 pref weight + 7000 suffix weight)' where is this from?
mortal bone
so you have a 61% chance to hit a prefix?
velvet fog
got it
elder hearth
@mortal bone exactly
mortal bone
interesting
it would be nice to know the chance of hitting 2-4-5-6 mods
you can't really extend that math without some sample size
elder hearth
2 is like 50% with alt/transmut
chaos spamming we need to test with items that do not rolls hybrid, like belt
mortal bone
is it safe to assume they would be the same across items?
it may have to do with spawn weights
elder hearth
hmm, i don't know why it would be related.
for sure it is different with jewels (max 4 mods)
mortal bone
more mods in the pool may lead to higher numbers of mods
elder hearth
i just used 748c on ilvl82 crystal belt and:
4 mods: 488 (65,24%)
5 mods: 204 (27,27%)
6 mods: 56 (7,48%)
65 / 25 / 10 ?
elder hearth
yeah...
elder hearth
Just did with 100 Cobalt jewels (i'm aware of multiline freeze/shock/ignite)
100 chaos:
3 mods: 64
4 mods: 36
@civic sluice maybe some useful data for your tool (check chaos spam for belts too)
elder hearth
yes
simple ravine
This is great guys!
simple ravine
@obtuse citrus Do you know if there is a way to circumvent this issue?
PyPoE.poe.file.dat.SpecificationError: <ERRORS.RUNTIME_MISSING_FOREIGN_KEY: 3001>: Data/ComponentAttributeRequirements.dat:BaseItemTypesKey->BaseItemTypes.dat:Did not find proper value for foreign key "Id" with value "Metadat
a/Items/Weapons/TwoHandWeapon/Bows/Bow0"
Ok, is that a PyPoE configuration, or a python thing? Sorry, I am fumbling around in python
obtuse citrus
when you make the relationalreader instance
simple ravine
ok
First, I thought I'd make a C#/F# port of PyPoE, but then I figured it's just a waste of time, and took the opportunity to get out of my comfort zone and write some python 😃
Thanks for the help
obtuse citrus
well you could continue developing the c# stuff that is already there
libggpk
it's pretty much dead though
simple ravine
yeah, you know how it is - brownfield / taking over someone elses code is many times 2-3x the effort than starting from scratch
i got pretty far in my proof of concept, but there were some issues with the string reference reads, where it would take too much/little or wrong string - can't remember, it was a while ago
i might re-open it sometime, but i figured, yours is kind of the defacto standard, and why not embrace it?
obtuse citrus
yeah
I ended up writing it because I had a number of tools already written in python and I found myself working around or fixing issues in libggpk and no real intention to take it over
also python has very rapid development speed if you know it and has excellent errors usually so working on new stuff isn't a pain in the ass
simple ravine
right, i started it on a whim when i was first starting out to learn about the files and the structures out of sheer curiosity and when the gui was terribly slow in reading and displaying the dat files
and thought that it might be a python thing, but oh well
obtuse citrus
the GUI needs some serious work still
I'll probably add some option to display specific lines only or toggle the data off
for the large files it slows it down a lot
the reading itself is "good enough" in python, I renember profiling the code to make it run reasonables well
but I could always just make a c++ or c reader for the files
and pass the d ata to python
simple ravine
right
well, I don't have "too" many issues with python itself, other than I am not familiar with the syntax of some the things
i guess i don't like the global functions, like dict() etc, and too used to having LINQ etc
i had a weak moment where I thought, "wonder if there's an IronPython way", but dismissed it for now as it'd just be wasting time away from the more interesting parts
which is using the data that is being generated
obtuse citrus
yeah also it's important to keep in mind that my idea behind the way the extractors work is that they read all of the file and k eep it in memory
I found it the most sensible way when I wrote this originally because most of the time you'd need all of it as well as the relations and then work based off that
simple ravine
for sure, agree on that
obtuse citrus
An alternative would be only reading/parsing the data you need
simple ravine
the reference from i.e. ComponentArmour.dat BaseItemTypesKey isn't a location pointer to BaseItemTypes.dat, correct?
elder hearth
@obtuse citrus do you have any type of previewer when you try to find out what each column means?
Opening a .dat file with hex edit helps a bit but not so much
Leta say GGG add some new .dat file, what do you do first?
elder hearth
I read that, but i mean before you map the column/rows inside that .ini file
elder hearth
@obtuse citrus oh, didnt know empty worked
obtuse citrus
it gives you a column size mismatch error
I do most of this in the UI of pypoe
and just reload the spec
also there is test_data.py
which I can run after updates to see if there are errors
elder hearth
Nice, GGG usually add columns and stop using old ones or they happen to delete/replace old ones? I found that Essence file a f*** cluster
obtuse citrus
usually add at end
delete when unused
deleting is annoying because it's in the middle of a file
velvet fog
libggpk updated to 2.6.0
thin loom
Do vaal gems have a separate class type
Class section here doesn't even mention gem as a class type: https://www.pathofexile.com/forum/view-thread/1260664
thin loom
Oh, so their class type is Active Skill Gems?
obtuse citrus
yes
thin loom
Thanks, working now 😃
inner vessel
Hey guys I was working on a script to count currency/map values and was wondering if there's a way to pull data from poe.ninja to get constantly updated values?
torpid mesa
The official way is to pull from gggs api and figure the values yourself but you could try to email the ninja guy and see if he has an api
I think he mentioned wanting one but I'm not 100% on that
Otherwise you have to hope he is ok with scraping
inner vessel
Ok thanks for the help
Seems like a lot of effort when it takes like 3 min to update the values manually lol
hot prairie
Hi Guys, I made this based off all yall's amazing work, thanks 😃 https://www.reddit.com/r/pathofexile/comments/6ex7zw/path_of_exile_3d_model_viewer/
inner vessel
Wow interesting tool
How do I request a specific tab for my account in Python? Something like this URL: https://www.pathofexile.com/character-window/get-stash-items?league=Legacy&tabs=1&tabIndex=0&accountName=Anonymuse
It keeps returning false for me because I assume I don't provide some sort of authentication
torpid mesa
@hot prairie what tools do you use for that? I was hoping to import the characters I to source film maker
@inner vessel you need the phpsessid cookie
Acquisition is open souce and supports it, check out how they do it
inner vessel
Ah ok, appreciate the help
hot prairie
@torpid mesa I figured out enough python to be able to smash the exported smd files from visualGGPK into noesis, thoes are really the two big ones. The rest was just a lot of copy paste javascript
torpid mesa
Hmm I didn't know we knew the format of the 3d files and such
hot prairie
I don't think we did recently. Closest code I found was from the beta, and I think the noesis guy wrote that. They changed most of the sizes of the verticies and such.
torpid mesa
You mean you figured it out? If so pls share
hot prairie
😄 i did, im quite proud, ive hit my head against these types of files a few times in the past, and this is the first one ive been successfull in
torpid mesa
Nice man
hot prairie
The site has the obj files up for download, the code needs a bit of cleanup before I am ready to post it publically, but ill pm you it
hot prairie
yeah, i would imagine that site does very poorly on mobile 😃
velvet fog
torpid mesa
mmm
velvet fog
there are 2 tools could view poe models
inner vessel
Hey guys I figured out the poe.ninja API calls, now my tab pricing script is always up to date! Thanks for the help
hot prairie
@velvet fog I found that one, and that is what I based my work on, getting it to work with the new format. What is the other way to view the models? I tried looking around for quite a while.
velvet fog
@hot prairie http://3doc.i3dconverter.com/ I used this before
hot prairie
no way.. that thing?! Thanks, i will try it out, might help me get the bones right
velvet fog
I don't sure is it work now
velvet fog
paid service for output better .obj, you can compare it
I think you already know how to use .mat file to match the texture
hot prairie
yeah, this thing does great, im ashamed I missed it!
velvet fog
@hot prairie https://github.com/Chuanhsing/libggpk/releases for updated libggpk
hot prairie
wow..nice. thanks
velvet fog
@hot prairie what kind of tool you test a obj file?
velvet fog
@hot prairie would you like to share the modified fmt_PoE_sm.py?
hot prairie
ofcourse, i wanted to clean it up and make it more usable first, but i don't think ill have time for that for a while
near gate
Anyone know the .dat file in the GGPK where the unique items are listed? I seriously can't find it.
mortal bone
They aren't in the ggpk
near gate
That would explain why I can't find them. Any idea of where they're at?
simple ravine
Is there a way to programmatically deduce what spawn tags to apply from the .dat files? There seems to be some tags that are 'impled' to certain item types, such as a body armour also probably applies 'armour', and a bow applies 'two_handed' and 'ranged'
obtuse citrus
simple ravine
oh, I have a vague memory about these files when exploring. thanks for the link, @obtuse citrus
simple ravine
@obtuse citrus When using OTFileCache to read ot files, it hangs when trying to access the .ot file (max cpu, infinite loop). Any idea why that may happen?
obtuse citrus
which files are you trying to read?
simple ravine
I just attempted to do this:
ot_file = otcache['/Metadata/Items/Currency/AbstractCurrency.ot']
(because it was the first one that occurred in the BaseItemTypes.dat row)
This is how I instantiate the OTFileCache:
def create_otfilecache(ggpk):
return OTFileCache(path_or_ggpk=ggpk)
remove the starting / and see if it works
Not sure why it enters an infite loop but it goes up one directory with a slash at the front
simple ravine
File "C:\Code\RePoE-fork\RePoE\RePoE\__init__.py", line 44, in main
modules[module](ggpk=ggpk, data_path=data_path, relational_reader=rr, translation_file_cache=tfc, otcache=otcache)
File "C:\Code\RePoE-fork\RePoE\RePoE\items.py", line 21, in write_items
ot_file = otcache['Metadata/Items/Currency/AbstractCurrency.ot']
File "c:\code\pypoe\PyPoE\poe\file\shared\cache.py", line 148, in __getitem__
return self.get_file(item)
File "c:\code\pypoe\PyPoE\poe\file\shared\cache.py", line 236, in get_file
f = self._create_instance(file_name=file_name)
File "c:\code\pypoe\PyPoE\poe\file\shared\cache.py", line 214, in _create_instance
f.read(**self._get_read_args(file_name=file_name, *args, **kwargs))
File "c:\code\pypoe\PyPoE\poe\file\shared\__init__.py", line 192, in read
return self.get_read_buffer(file_path_or_raw, self._read, *args, **kwargs)
File "c:\code\pypoe\PyPoE\poe\file\shared\__init__.py", line 151, in get_read_buffer
return function(*args, buffer=file_path_or_raw, **kwargs)
File "c:\code\pypoe\PyPoE\poe\file\shared\keyvalues.py", line 331, in _read
self._parent_ggpk.directory[extend].extract()
File "c:\code\pypoe\PyPoE\poe\file\ggpk.py", line 584, in __getitem__
self.get_path(), item
FileNotFoundError: /Metadata/Items/Item not found
when removing the slash
obtuse citrus
hmm it should be looking for Item.ot in that case let me check
might be specific to the ggpk mode
simple ravine
Ok, I can try with the exported folder to see if I can replicate
looks like it worked when using the folder
obtuse citrus
I pushed a fix for this problem
simple ravine
that's fast 😃
obtuse citrus
it's trival to fix
simple ravine
thank you
obtuse citrus
on a side note, in regards to what you said in the beta channel, the update for the beta should be up as soon the beta files hit
unless they are large changes from alpha
simple ravine
oh, you have access to a closed test environment?
simple ravine
@obtuse citrus Hey sorry to bother you this much. Got a question regarding the tags ordering in the OTFile.
When accessing cache['Metadata/Items/Weapons/OneHandWeapons/Claws/AbstractClaw.ot']['Base']['tag']
you get the following list:
['default']:True
['weapon']:True
['onehand']:True
['claw']:True
['one_hand_weapon']:True
But according to someone, the ordering of these tag matters, and my working theory is that it should be
claw, one_handed_weapon, onehand, weapon, default
which makes me think that the dictionary itself should be reversed, but also the individual tags per .ot-file being read in traverse should be reversed.
Thoughts?
obtuse citrus
it's an ordered dictionary
simple ravine
sure, but the ordering question remains 😉
well, I guess the spawn-tag ordering is more important than this one
but to me they seemed a bit in "the wrong order" in this ordered dictionary,
obtuse citrus
I see what you mean
Base
{
tag = "claw"
tag = "one_hand_weapon"
}
it would mean I need to reverse reading that
simple ravine
it seems you're reading it in reverse right now
obtuse citrus
or append inherited at the end
no it's just reading inherit 1 default, then the second file with weapon, 3rd file with onehand, and the 4th file with claw and one_hand weapon
simple ravine
Shouldn't it be
claw
one_hand_weapon
onehand
weapon
default
ah you're right
instead of prepending, appending should do the trick
obtuse citrus
I'm looking at the code and it seems broken anyway
velvet fog
ordering of these tag matters? how it work?
simple ravine
when working in conjunction with the spawn tags on modifiers, you would want to check the "upper-most" tag first, and iterate through the list
one can work around it by going by the spawn tag ordering, but I think it'd be smart to have these tags in its proper ordering as well
it'd be somewhat more efficient execution if it's in order, I would believe
velvet fog
I use it without ordering
elder hearth
Me too, but only if you check mods tag (ordered) against your items tag
simple ravine
yeah, i guess it's fine
obtuse citrus
right it should output OrderedDict([('claw', True), ('one_hand_weapon', True), ('onehand', True), ('weapon', True), ('default', True)]) now
just need to check if I broke anything else in the process
simple ravine
ah, no unit tests around .ot files?
obtuse citrus
there are
elder hearth
@simple ravine what kind of tool are you developing?
simple ravine
A tool to determine the cost of crafting specific items based on various strategies
among a couple of other things
elder hearth
Nice
simple ravine
second thing would be to index stash tabs, and compare the price to the crafting cost 😃
and use that as a value indicator
(yes, some mods are dead mods, and some combinations are terrible)
elder hearth
I think it is useful for high end gear
simple ravine
yes, that would be the target audience
obtuse citrus
I pushed an update for that problem with the ordering
I had to rethink how the stuff works so I hope I didnt broke anything
the tests aren't very exhaustive at the moment
simple ravine
awesome, i'll pull the changes and test it
"tags": [
"claw",
"one_hand_weapon",
"onehand",
"weapon",
"default"
]
Perfect. Thanks @obtuse citrus
"name": "Stag Sceptre",
"shield_stats": null,
"tags": [
"sceptre",
"not_for_sale",
"maraketh",
"sceptre",
"one_hand_weapon",
"onehand",
"weapon",
"default"
]
interesting
I suppose game developers aren't an exception to the human errors that arise sometimes
simple ravine
that is the .dat + .ot mods joined together
simple ravine
i suppose python has a .Distinct(); equivalent
obtuse citrus
I have no idea why they didnt put those .ot stuff into .dat files tbh
simple ravine
legacy code perhaps
heh, python doesn't have a distinct method that maintains the order
that is quite mindblowing
obtuse citrus
well you can used OrderedDict for that
simple ravine
so spoiled with LINQ 😃
obtuse citrus
OrderedDict.fromkeys([1,2,3,4,3,5])
for example
need to do from collections import OrderedDict
simple ravine
and to make it into an array again, I'll have to do list(OrederedDict.fromkeys(...)) I'm guessing?
simple ravine
seems like a lot of casting/boxing/unboxing for the cause, but allright
>>> list(OrderedDict.fromkeys([4,4,4,3,2,5,1,5,4,4,3]))
[4, 3, 2, 5, 1]
looks like that worked as well
obtuse citrus
Ah, yea forgot it's just interating over the keys anyway
a number of people seem to work with some of the specific stuff I'll probably move some of the wiki export code to pypoe core (pypoe.sim) to get some of those tasks done automatically
To create abstract items that pull the info from the various files etc
simple ravine
here's my currenct script, on top of RePoE:
https://github.com/andreandersen/RePoE/blob/add-weights/RePoE/items.py
I'm obviously very inept with python, and the poe data itself - so this can most likely be improved dramatically
obtuse citrus
the wiki exporter does something similar
except it has a lot of extra code for wiki purposes mixed in
simple ravine
ah, i'll see if I find it when I wake up
time to get some sleep 😃
thanks for the help, bud
obtuse citrus
good night, and no problem
simple ravine
Has anyone found the drop rates for currency items?
i.e. 'how many alterations drops per chaos orb'
elder hearth
elder hearth
@simple ravine no
simple ravine
Not sure that's statistical significant data, but a good indicator nontheless
elder hearth
I think it is enough, very close to old datamined data (below exalted ratio, also looks like exalted got nerfed in mean time)
simple ravine
yeah that is a significant difference (exalts)
pseudo ocean
@wanton vault yes
simple ravine
@wanton vault There's a vendor recipe for exalted orbs too, but you're not going to like it. A Mirror of Kalandra vendors for 2 ex, 5 regal and 1 divine 😃
wanton vault
@simple ravine Is there any other repository of recipes besides poe.wiki? 😃 And yes I wouldn't really like that 'exalted' recipe 😄
simple ravine
Not that I know of (except the ones in the .dat files)
obtuse citrus
for something with a low dropchacne the number of trials (50k) still might a bit small
simple ravine
statistical significance in A/B testing is usually 1K sample data per test + control (N+1).
i'm in the ad-tech business normally, so that's my reference point, but not sure here
I would deem perhaps 1K runs quite solid, but to be more precise, you'd want to run it until you get ~10 mirrors, and well.. that's gonna take some runs.
better to distribute the workload
velvet fog
4 new .dat files in 2.6.1
simple ravine
.dat related? edit: was meaning to type mtx related, was too tired 😃
velvet fog
MicroMigrationData.dat MicrotransactionSlotId.dat MysteryBoxes.dat ItemThemes.dat
elder hearth
Does anyone know eventloop's (poeapp.com dev) Discord?
obtuse citrus
@timid hemlock do have a tool that converts the skill tree json to the lua version that pob uses?
timid hemlock
@obtuse citrus It's not so much a tool as a dirty little hack
https://github.com/Openarl/PathOfBuilding/blob/master/Modules/Common.lua
Function 'jsonToLua'
obtuse citrus
Ah ok thanks
mortal bone
is there an option for 'I am leeching' ?
mortal bone
where the other options for 'I am full life' or whatever are
foggy vortex
in configuration?
that isnt an option
Ok I am positive the retch does not work with pob
obtuse citrus
if ggg doesn't release a skill tree for beta I got something working for PoB 😃
velvet fog
there is a skill tree generator at 2.0 beta
mortal bone
Yeah, but it didn't generate the data in a good format