tooldev-general

[sturdy drum]

I cant imagine over 100k daily players and all the items. I do not envy the amount of data they have to process

[lone ingot]

gigantic amount, I dunno if there's any open source code that processes it but I'd be curions to see what people are doing

I am assuming it's finite an has a cut off date

[strong patio]

that would be my assumption too

[sturdy drum]

I bet they dont store much historic events if I had to guess

[lone ingot]

unless someone has thought of a genius data structure or way of compressing it

[worthy cape]

I believe only the current state of each stash tab is tracked. If you request older IDs, the corresponding stash entry is empty.

[strong patio]

Ahh okay. So does the river basically say which stash tabs have changed?

[worthy cape]

The web peeps know more about how it does it all, I'm only going off historical experiences.

[strong patio]

Algood. Its appreciated still haha

[sturdy drum]

Now I am imagining creating a poe blockchain, and the mining is just from these events and the user tied to the event is given the block reward.. and its just random who gets the rewards lmao

[lone ingot]

https://www.pathofexile.com/developer/docs/reference#type-PublicStashChange

that's the event right?

[oak estuary]

Yeah, was about to post this link too: https://www.pathofexile.com/developer/docs/reference#publicstashes

[lone ingot]

bloody verbose event 😄

[worthy cape]

Yeah, it's the psapi scope, or colloquially "the river".

Each such event contains a complete snapshot of a stash tab with the whole kitchen sink of item descriptions and everything.

[oak estuary]

Most people have a lightweight process to keep up with the river and offload the actual item processing to other processes, at least from what I've gathered from being a fly on the wall in here over the years

[worthy cape]

Trade sites were built on top of this by leveraging notes set on items and updating listings as they notice things appearing or disappearing.

[strong patio]

Yep

[sturdy drum]

certainly. Ive built a lot of enterprise data pipelines, they probably push the events into an event queue for their processors to chunk through

[worthy cape]

Starting from the dawn of time only really matters if you care about the state of tabs that haven't been touched in ages.
Most people who build new things tend to start off a well-known ID like the start of a league.

[lone ingot]

in theory you could just read one commit at a time, find the things you are interested in by sorting on the Item type (by name + note I guess?)

[worthy cape]

It's a fun data processing problem as the volume and verbosity of the data is large enough to cause trouble.

[oak estuary]

I'm actually surprised there's not a common library for that given how many questions about implementation have come up over time. There are like 3 different repos for processing data from the GGPK in contrast

[lone ingot]

I dont know how you keep a checkpoint and relative timeframe though

[strong patio]

The api gives u a new request id to poll

[sturdy drum]

You have to store the data you want at said checkpoint

[strong patio]

next_change_id

[lone ingot]

yeah I mean, you checkpoint it yourself, I just mean that it doesnt look like you can tell when the last change was on the actual stream

you can see a change, just not when

[strong patio]

So in theory your server should keep checking that until the stashes list is empty. Then im sure chill out for x amount of time before using it

[lone ingot]

so I guess you can walk back to where it expires?

[sturdy drum]

Could you use an id of 0 to get the first page and walk all the way through I wonder

[worthy cape]

Change IDs are logical, there's no relationship to wall time except knowing roughly extrinsically when an ID was generated.

[lone ingot]

I think? that's what the schema looks like at least. you basically get an event at <unknown time> that reflects the state in <that moment>, it's up to you to work out the when if you want to aggregate by 24 hours or whatever

[strong patio]

I believe you just request without a change ID and you are given a next_change_id to follow then

[sturdy drum]

and given its a 5 min delay, that - 5 min +- your polling frequency

[worthy cape]

Supposedly. It's going to be quite sparse as you hit a sea of empty entries.

[sturdy drum]

ah so then it would seem they do store all of a leagues data from the begining of time

[strong patio]

No. Just the most recent change objects for each public stash

[worthy cape]

Turns out that to store stash tabs you need to store stash tabs.

[strong patio]

each PublicStashChange object contains the list of item objects at that moment

[lone ingot]

petition to change the id ( a unique 64 digit hexadecimal string) to https://github.com/ulid/spec pls

[sturdy drum]

Guess I dont quite understand. If I GET /public-stash-tabs it will return a list of stashes and a next_change_id. Wouldnt not suppying an id mean I get the first, and then I follow the next_change_id to go through every change that has happened since the league started?

[strong patio]

So when a change to a stash tab is noted. The api creates a new object for that change. Then serves that one. Consumers of the api should note the presence of that stash tab again and throw out what ever information they had on it before.

[strong patio]

Not since the league started. Because that river will only have 1 object for each stash tab

[worthy cape]

next_change_id is the pagination indeed, you use it to step linearly through the history.

[sturdy drum]

omg nomad I love that! I HATE UUID so much Im so glad you shared that

[strong patio]

But in theory if some guy had not changed hist ab since the league started then yeah that would effectively be true

[worthy cape]

If you supply none, you probably get the current state of things.
If you supply all noughts, you get the dawn of time.

There's some endpoint that has the current change ID too.

[strong patio]

oh interesting

[strong jacinth]

Holy

Its been 30 mins

[lone ingot]

you can do ulid.decodeTime() or similar and it's just

[worthy cape]

https://poe.ninja/stats

[strong jacinth]

That Stash data, is that all time? Lol

Cause that's alot of data

[lone ingot]

the other uuid replacement I think is cool is this one https://github.com/jetify-com/typeid, basically the standard stripe uses

[worthy cape]

Note that there's no "league" boundary in the river, it's the whole game since some point near the beginning, probably spanning over a decade.

[lone ingot]

I like the time sorting aspect of ulids though 😄

[worthy cape]

Back when I tried processing the river when it was backed up, I sustained like 5 MiB/s right at the request rate limit.

(good number of years ago now, pre-oauth)

[lone ingot]

tbh I am surprised it's not more compressed, like a diff of state A > B etc

[strong patio]

yea

[worthy cape]

The neat thing about snapshots is that you need no previous state or external resources to decode the information.

[lone ingot]

surely it is quite expensive to have people paginate the whole thing

[strong jacinth]

Not bad, how backed up was it? Haha

[spring compass]

After half a year without coding, I’ve finally restarted work on Mapwatch: https://mypoe.site/mapwatch/
For now, I’m only sharing it here—if you have a minute, take a look, and if you have two, it’d be great if you could let me know whether everything works, what doesn’t, or what could be improved.
Thanks in advance! Oh, and just so you know—I haven’t been getting much sleep for the past two weeks.

[worthy cape]

At peak think like 30-40 minutes?

It was a weird league 😄

Great fun if you underpriced something as you would get a torrent of whispers long after you sorted your mistake.

[spring compass]

zao is now ggg employees, changes

[worthy cape]

I made a small project then with delay probes to measure how it varied over time.
Drop an identifiable item in the river, wait downstream to see when it actually manifests.

[lone ingot]

I gotta say, it does sound like a fun data processing problem

I am thinking of doing something similar for AWS graph DB Neptune, it has an event stream that follows similar semantics and is HTTP based

[worthy cape]

Coming up on half a year soon now, lots of fun to be had on this side of the curtain.

[lone ingot]

it has ops for going at/before commit nums, trim horizon, etc though so a bit more ergonomic

[sturdy drum]

first of all, https

Zao you work at GGG?

[worthy cape]

Indeedily.

[sturdy drum]

That’s very cool!

[spring compass]

On your side or ggg's side?

[sturdy drum]

Do you develop their APIs?

[spring compass]

Only first? 😛

[sturdy drum]

Idk if I wanna upload a file over http 😛

[worthy cape]

Internal tools feel so familiar and yet so different to work on compared to the community tooldev projects.

[worthy cape]

Nope, that's more of a web team thing.

[sturdy drum]

If I may ask, what do you work on Zao

[worthy cape]

Internal data tools, gameplay, UI. Bit of a jack of all trades.

[spring compass]

Is GGG still a company where anyone can do anything depending on where they are needed?

[worthy cape]

Can't really comment on organisation stuff like that.

[lone ingot]

I feel it's unlikely that a competent dev would get silo'd in a company the size of GGG's, if you've seen a lot of stuff (and care) it's kind of hard not to be useful everywhere

[spring compass]

I haven't seen this before. This isn't an upload, just math on your computer.

I have to lay down, I'll be back in an hour or 8. Maybe someone will click the link 🙂

[worthy cape]

[sturdy drum]

Oh this is very cool! Most of the panels didnt show me any data but I like the timeline

[modest path]

More like target of all blame

[spring compass]

Either you didn't select the file or you're not playing.

Some of my charts from poe2. End at 20.01, I didn't launch this game after that date.

[spring compass]

Do you work on poe1? 😄

[strong jacinth]

Zao works on stuff

[sturdy drum]

hmm I selected the log file and I did several maps today and for the past week plenty. Im not sure why it didnt work for me, maybe my browser or plugins let me try again

Here is a print out of what i am seeing. I tried on chrome and firefox with all plugins turned off

I can give you my log file for debugging if youd like

[spring compass]

Heh, I have to add borders to show people what they should click. I can't automatically read file in browser, ok I can but only if file is not inside system folder, like program files. Please click: Process File.

[sturdy drum]

I did click process file, I got the timeline and a 2 whispers but thats it

Let me move my log file to the desktop and try again

[spring compass]

Yes please

[sturdy drum]

No dice

[spring compass]

any console errors?

[sturdy drum]

Uncaught TypeError: Cannot read properties of null (reading 'date')
    at reader.onload (app.js:169:62)
reader.onload @ app.js:169
load
(anonymous) @ app.js:29Understand this errorAI

[somber mortar]

I don't really wanna look at this, let me see, just a second

oh no. its as i feared. maybe a bit too much poe2

[spring compass]

                    continue;
                }```
Added a small change, please wait 5 minutes, but this means that here you have a null value, which may cause errors in time calculation later.

[spring compass]

16h in one day!?

I thought I played a lot.

Maybe GGG won’t want people to see how many hours they’ve played?

[somber mortar]

i mean, i can see it in steam and /played :)

[spring compass]

Not if you don't have steam

[somber mortar]

in my defense, some of the longer days are just wfh days where I just have the game open for trades

[spring compass]

You didn’t need any defense, I have those numbers at league starts.

[somber mortar]

😬

[spring compass]

Data for a single character.

[somber mortar]

wdym?

[spring compass]

This number is different for each of your characters.

Ok, you can sum it up

[worthy cape]

Thank you for contacting Support. Our records show your account has been played for 1y 122d 6h 4m 55s.

[somber mortar]

you can change characters?

[spring compass]

Your or mine?

[worthy cape]

Mine, wanted to know back in September.

Lost track of how many hours were actually played as I had moved between Standalone and Steam a few times.

(number above isn't fully accurate, as purged characters do not count)

[frank drift]

purged like voided?

[worthy cape]

This value is based on played time across all your existing characters and any of your deleted characters that have not yet been purged from our databases, so it will likely not match 3rd party metrics from sources such as Steam or Epic.

I don't know the particulars here - I would assume that "purged" is something more fundamental DB-wise than the higher level concept of being voided.

[exotic egret]

Mostly this is before a certain point we didn't keep track of time spent on deleted characters for you

[lunar crypt]

heh, i started something like this as well, just before poe2 released

its on pause atm due to work on wiki tools

[frank drift]

ah

[spring compass]

I waited 6 months from the idea to actually start coding.

[sturdy drum]

hey this is cool!!!

Thanks for pushing that fix

[spring compass]

But the way I did it means that the next data will probably be wrong—my guess is a maximum of 24 hours.

The session tab is only for time calculations.

[sturdy drum]

I like the player events hah I can see when my friends and I died whil playing together 💀

[spring compass]

Look at the dots where you level up together!

I'm going back to sleep, at least now I know it works not just for me. 😅

[oak estuary]

Was that recent or do you know the approximate time? I haven't deleted many characters, but I've deleted more recently so you're saying those ones might still be counted for playtime data?

[exotic egret]

Not recent.

[scarlet ruin]

before i send any pull requests to dat-schema i thought i'd ask here for clarification.

Is there a way to declare a foreign key to a yet unknown table?

I toyed around with the NPCs table and this is what came out

type NPCs {
  Id: string @unique
  Name: string
  Metadata: string @files(ext: [".ot", ".otc"])
  _: rid
  NPCMasterKey: NPCMaster
  ShortName: string
  _: i32
  NPCAudios1: [NPCAudio]
  NPCAudios2: [NPCAudio]
  HASH16: i16
  _: NPCs
  Portrait: NPCPortraits
  DialogueStyle: NPCDialogueStyles

  # changes start here
  # foreign key?
  _: u32 
  _: u32
  _: u32
  _: u32

  Gender: string
  _: bool
  # path to a portrait image. 15 rows have it set.
  _: string 

  # foreign key?
  _: u32 
  _: u32
  _: u32
  _: u32
}

The 2 sections with 4 x u32 seem to be foreign keys but i don't know where they point to. So i would rather declare them as unknown foreign keys or at leas 2 x u64, yet u64 isn't a supported type.

@civic crane @zenith zodiac seen you two contribute to the project. Are you the right ones to ping? sorry if not.

[civic crane]

@scarlet ruin _: rid

[zenith zodiac]

also note that you can click the 'show schema' button to see any changes you've made (you need to close the existing tab first if there is one; it won't be overwritten)

[scarlet ruin]

thx. but isn't rid different than a foreign key?
in json schema to would be "row" instead of a "foreignrow"
which is 8 bytes vs 8+8 bytes
https://github.com/SnosMe/poe-dat-viewer/blob/e6835cba3ea44ca13ca1a3653146ac9d7d3a1c90/lib/src/dat/reader.ts#L17

[civic crane]

you always know the name of local table, so obv rid is for unknown foreign 😄

[scarlet ruin]

i'll check after work. If rid is the way to go and it generates what i need in the schema i'm happy, thank you.

[scarlet ruin]

thanks again, works perfectly . I've sent a PR. should have seen the _: rid in the same table but i mistook it for what _: NPCs is. Hence my confusion.

[modest path]

I'm excited to get back to rust schema analysis stuff now that rust cdn loading is pretty much functional

[spring compass]

I found a bug that no one could tell me about because they couldn’t know that something was supposed to be there. I had to launch the game and provide new data. I re-added time periods to the daily chart and the last active session to the histogram.

[spring compass]

Offtop but, I'm quite impressed. I made trades worth 29k div last year.

[spring compass]

I there some JSON where I can find relations between names used in client.txt and names uses for players?
Eg.
1_3_town -> The Sarn Encampment
AzmerilLeagueBoss -> King of the mists

[modest path]

worldareas.datc64, you can use poe-dat-viewer's lib to pull it down as json https://github.com/SnosMe/poe-dat-viewer/tree/master/lib

or for non-auto updating but easy you can use the export data button in https://snosme.github.io/poe-dat-viewer/

(data/worldareas.datc64)

[foggy stag]

Heh that is kinda 😎

I'm also doing things scanning the client.txt it's primary purpose is a death tracker, I have all sorts of ideas for analysing the data, like how long was your play session, did you die of fatigue, common day of death or death times and so forth xD

https://github.com/thmsndk/poe2-death-tracker

I'm also using snosme dat viewer to extract area names and I've been exploring boss names
https://github.com/thmsndk/poe2-death-tracker/tree/main/data/tables/English

The guide for the dat viewer extraction is here https://github.com/SnosMe/poe-dat-viewer/blob/master/lib/README.md

Working on a rewrite to keep track of sessions, current area, and so forth ☺️

Theese are some if my ideas, they might be useful for your tool as well https://github.com/thmsndk/poe2-death-tracker/blob/main/ideas.md

I'm still playing the campaign in hardcore, so I've not seen log statements from all of the game yet though ☺️

[spring compass]

Any chance you know where is the same data for boss instances?
AzmerilLeagueBoss -> King of the mists

MapWorldsPrimordialBoss1
MapWorldsPrimordialBoss2
MapWorldsPrimordialBoss3
MapWorldsPrimordialBoss4
xD

[modest path]

probably some combo of *boss* tables

or *arenas*

@spring compass I'd recommend using pathofexile-dat from poe-dat-viewer to export all the tables to json so you can grep them

tables/English/ClientStrings.json:    "Id": "AzmeriLeagueBossRitual",
tables/English/WorldAreas.json:    "Id": "AzmeriLeagueBoss",

AzmeriLeagueBoss's area name is Crux of Nothingness, but the MonsterVarieties table entry is linked to is index 9944, which is Metadata/Monsters/LeagueAzmeri/VoodooKingBoss/VoodooKingBoss2

@spring compass If you're a js bruh you should consider PRing the addition of row filtering to poe-dat-viewer :)

and row reference cell hyperlinking to dest table row ;)

[spring compass]

I want to stop here, by the way my faith in humanity has been lowered. One 'tester' tried to parse client.exe instead of txt, but that was my fault. Now the site checks if the file has the right extension.

[worthy cape]

Now upload Content.ggpk

[spring compass]

I also think I still have plenty of time before the next league.

[spring compass]

I didn't catch that. There you can see what is in content.ggpk: https://snosme.github.io/poe-dat-viewer/ 😄

[worthy cape]

I assume that the browser would refuse or fail to read an 80 gig file before it ever got to your site.

But yeah, client log, client exe. Same thing, isn't it? 😉

[oak estuary]

So can I rename client.exe to client.txt and upload it?

[spring compass]

Ofc you can! If you wanna see nothing

[strong jacinth]

Wouldn't your OS just have a meltdown

[worthy cape]

There's only one way to find out.

[modest path]

@spring compass Stream read body, check first dozen bytes for valid strings, http error if not?

[strong jacinth]

I think my PoE1 client.txt is only 40 mb?

Unless I didn't count the 0s correctly

[worthy cape]

It's all local, I believe.

[modest path]

ahh

[worthy cape]

Healthy choice as there's probably some privacy concerns in sending large parts of your chat history into the browser in the first place.

[spring compass]

All local, it is github, I can't upload anything here(?)

[worthy cape]

The page could assumedly make an XHR outwards to some host to exfiltrate data.

[modest path]

Shit you can exfiltrate data with an img tag

[worthy cape]

There's not much one can do as a user but trust the page author.

[ashen aspen]

imo browsers could do a better job with that. making it visible for the user somehow that no data leaves their pc

[modest path]

You have to send data to get data

[ashen aspen]

but i also don't know how right now ;)

[worthy cape]

Best one can do is outline what the file can contain and pinky promise to not do anything naughty.

[ashen aspen]

best example are those base64encode/decode sites. i think the popular ones work serverside

yikes

[modest path]

Shit, even on the proper data end, you can put a form that posts to a remote host in a page and use js to submit it and avoid same origin shit

You can use that to be able to make http requests to local servers that don't have auth or a header requirement if you can get someone to visit a public webpage

(an example of this was that some crypto wallets defaulted to having their API on localhost with no auth, and you could make a request to the api to send coins if you could get them to load a page)

[ashen aspen]

it's not that simple, cors does a decently good job preventing stuff like this

[modest path]

It hopefully has gotten better since I messed with it

Jesus that was 10 years ago

[compact isle]

browsers are pretty good at keeping things pinned to your domain nowadays

which is great when you get told to make some sort of site2.com

[modest path]

Ehh, seems like current firefox does not prevent the onload submit posting

[compact isle]

cross-domain form submissions are prevented in protection mode or whatever, we had folk struggle to sign up to the mailing list because of it

[modest path]

formtester doesn't have any CORS configured for that URI

[ashen aspen]

more or less what zao was talking about

google analytics is basically the same thing.. post meta data on load

[modest path]

God I hate the web

Lets go back to the invention of the hyperlink and redo everything since then

[spring compass]

The site doesn't upload anything. You just mark the file for calculations. At least 3 people write about sending. Why this fear, what can I do with the logs? This data is definitely irrelevant if I don't have access to the admin panel.

[modest path]

@spring compass A way I've seen to combat that is to make a download link on the site that downloads it as an html file that will work even if you disconnect the network. It's not that anyone is going to use it, it's that you care to support it

[strong jacinth]

Yeah i thought as much, I just meant the size of my client.txt

I can't remember how big it is

[modest path]

@spring compass the shorter answer to those people is "look at dev tools network tab if you're concerned"

[worthy cape]

It contains whispers and guild messages, something that a player may not know or expect to be there.

For web applications, even if they only ever have in-browser state - the problem isn't what the site actually does but what it could possibly do.

[zenith zodiac]

l recently added world areas to repoe https://repoe-fork.github.io/poe2/world_areas.json you can use that data to render layouts as at https://repoe-fork.github.io/webapp/#/areas/G1_2 . l haven't added the exporter for poe1 yet but l believe the data is similar so it shouldn't be too hard

[spring compass]

My "JSON":

    [
        "NULL",
        "NULL"
    ],
    [
        "CharacterSelect",
        "Character Select"
    ],
    [
        "PersonalHideout",
        "Your Hideout"
    ],```

Does this look bad? Maybe I'm missing something? Do you think I can post it on reddit?

Or maybe it's a bad time, beacuse of the situation?

[worthy cape]

Which one of the games does it support, if it cares?

[spring compass]

both

Oh, you gave me a hint. I don't have ingame names parser for poe2

[worthy cape]

Didn't know if it was broken or not, but a bit curious given that you seemed to do some translation 😄

[spring compass]

Tyvm, just in time

[worthy cape]

For a while in both PoE1 and PoE2 separately, some whispers were missing @From and @To markers. Not sure if that's something you care about or may stumble on.

[spring compass]

I'm not sure if I can anything about it. All lines with '@To' and '@From' are selected. If something is missing, it means my regex didn't extract data from that line. But if this happend it's black magic to me.

const regex = /^(\d{4}\/\d{2}\/\d{2}) (\d{2}:\d{2}:\d{2}) .*?\[INFO Client (\d+)\] (@To )([^:]+): (.+)$/;

@worthy cape added

[spring compass]

And boom 🚀

https://www.reddit.com/r/pathofexile/comments/1ie49dz/game_statistics_on_your_computer_map_watch/

First 15min

[strong jacinth]

Definitely gonna hit it up again when I get home

Keen to see what you've worked on today!

[spring compass]

Unfortunately I haven't connected anything to see site stats.

[spring compass]

Poe1 reddit is on 🔥

Thanks guys for the upvotes, I'm sure they're all from you. See you tomorrow

[rocky bobcat]

@spring compass FYI if you upload a log with no data for $current_day you get:

TypeError: sessions[(sessions.length - 1)] is undefined
    TodayChartGeneratingLevelsData https://mypoe.site/mapwatch/js/charts/todayChart.js:278
    onload https://mypoe.site/mapwatch/js/app.js:151
lockdown-install.js:1:97687

[spring compass]

I added messages without @ [client info number] anyword: message. Now: local, party, and channel messages are also visible on the "today chart."

[delicate stratus]

are Citadells still bugged ? I have 1 monster left and can't find it

[spring compass]

Do you see this error now? I wrote some new code today and uploaded it 5 minutes ago.

[rocky bobcat]

I've since booted the game and didn't get the error with the new logs, but I can trim some entries and re-test, 1s

nope, error is now gone -- I see this instead and the rest of the charts work, neat tool 🙂

[spring compass]

So it works.

Maybe I should hide this chart when there is no data?

I'm not very good at this

[proper glen]

What kinds of things can you do with PoE 2’s tool APIs? Am a SWE but haven’t looked into what all is available

[frank drift]

PoE2 has no developer APIs as of yet

there are one or two internal website APIs like the trade site API, where use is tolerated

[proper glen]

Does that mean you guys are reading memory or something to build tools? Is it allowed?

[frank drift]

reading memory is not allowed

generally tools just use the clipboard, but it depends what tool you mean

[proper glen]

I’m not even sure what kinds of tools are possible - never played PoE 1, didn’t even know there were tools for PoE 2 til I noticed this channel haha

[frank drift]

https://www.pathofexile.com/developer/docs is the docs for PoE1

[proper glen]

I guess I’ve seen screenshots of some overwolf overlay for trade

[frank drift]

that will be using the clipboard to copy the item, and the log file for messages

[proper glen]

I’m not a fan of overwolf so I didn’t look further

[frank drift]

that's Overlay, there's an open source version that was forked years ago when it went Overwolf, however the open source version doesn't support PoE2 yet

[proper glen]

Ah okay so you can read from the clipboard and read PoE log files and never have to actually interact with the game? Nice

Bummer about the overwolf thing. Pretty sure overwolf meets most definitions of malware

[frank drift]

I would say spyware but yeah, Overwolf makes its money by collecting and selling user data

[proper glen]

Spyware is malware

So what types of things will be in log files?

[frank drift]

you can go open your logfile and find out 🙂

[proper glen]

Where do they live?

[frank drift]

a tool that uses the log was posted a bit above #tooldev-general message

[proper glen]

Nice thanks

I saw some options in PoE 2 settings about “map mods,” does that imply you can build mods for the map or?

[frank drift]

"mods" is short for "modifier", an ARPG (and just RPG probably) term of art

as in an effect that modifies an item

[proper glen]

Ah word

[worthy cape]

The client has no mod interfaces and modifying the data files or touching memory or traffic is forbidden.

[proper glen]

Is there any word on whether they’ll be adding the PoE 1 API functionally for PoE 2?

[frank drift]

someday™️

[proper glen]

Nice.. lol

Guessing probably low priority atm eh

[worthy cape]

Visual and UI-wise, what's built-in is what you get. Behaviour-wise, it's mostly item data via clipboard and pasting chat messages into the chat box to send a thankful message to trade partners and go to hideout.

[proper glen]

Given that they’re delaying Poe 1 league etc

[frank drift]

yeah safe to say that developer APIs are on the low end of priorities

[worthy cape]

APIs are largely a web team concern.

[proper glen]

For the pasting into the chat box, would that be like triggered by a hotkey that the overlay listens to? Like no actual api to send a message?

[worthy cape]

Correct.

[proper glen]

So something like:
press hotkey
overlay presses enter and pastes text

[worthy cape]

The only interactions you can do are sending a few keys to perform actions in accordance to the dev doc constraints.

[proper glen]

I remember reading that macros can only perform one action per key press or something to that effect

[worthy cape]

Pretty much. It's hilarious when APT accidentally pastes your clipboard novel into chat.

[proper glen]

Wouldn’t sending a message be > 1 action?

[worthy cape]

One "game action".

[proper glen]

does that exclude UI actions?

[worthy cape]

It's a bit loosely defined, but something like Enter, paste, Enter to send canned messages is "fine".

Automation of item use and cursor positioning to do things like rapid crafting is not.

[proper glen]

ah word. kinda lame tbh. this game is so tedious with all the clicks to do every single thing

[worthy cape]

As are things like making an user action do a different thing based on external or parsed state, so no spam clicking until a desired outcome happens for example.

There are a lot of parts of the game that have friction to them, but they're fair and tools should not be mandatory to be competitive, if they makes sense?

Note that this isn't normative to what's absolutely allowed and not, mostly just me noting how things have been ruled and seen in the past.

[spring compass]

It's a good topic. Can we use SendKeys? I don't know how awakened poe trade do this. Do they send one key or both in one function? SendKeys.Send(^c) // if wrote it corectly https://learn.microsoft.com/pl-pl/dotnet/api/system.windows.forms.sendkeys.send?view=windowsdesktop-9.0

[worthy cape]

A good yardstick for the desired experience is the visuals and game interaction you see on a stream should be the same if you decide to download the game and get started from scratch.
No mandatory mod packs that break all the time, no fiddling to get the UI just right, no macros to get the game playable.

I don't recall how APT does it but it's open source and not too horrible JS/TS. Some sort of mix of sent input and clipboard contents.

[frank drift]

something like that, probably, for the ctrl+c ctrl+v

[proper glen]

Yeah I feel you. But when you have the ability to automate a lot of tedious tasks, it's a bummer that you can't due to TOS

I get why it's not allowed though

[worthy cape]

The trickiest part is whether you restore the clipboard and what timing to use to assume that the game has seen your input.

[spring compass]

Yes, but these are two keys at once. They always say one key, one action.

[worthy cape]

One user interaction resulting in one game action, hit F5, have a /hideout command sent.

Orchestrating the text input to achieve that action may require multiple steps, I hope that difference carries through.

[spring compass]

Oh, ok. So I can make SendKeys.Send("Enter", "you are the best", "Enter")?

[frank drift]

I think tools would generally put the middle part on the clipboard and ctrl+v? but yeah

[spring compass]

Is something like alt+w (ingame) -> poe wiki with that item?

[frank drift]

you can do that with a tool, sure... several tools have a shortcut section to add stuff like that

[spring compass]

I have no idea for use this, but it's good to know

[lone ingot]

example of sending a chat command in rust https://github.com/jchantrell/chromatic-poe/blob/main/apps/backend/src/filter.rs#L60-L141

like zao said, one user action, one logical step. you can't have a user press a key then do some sequence of things in game. in my mind the actual sending of the chat cmd is the action, the steps leading into it (like opening chat, pressing the keys for the msg, etc) are supplementary to the action

it's a hard thing to define clearly honestly, basically if it feels like you're entering automation territory, you're probably breaking the rules

[spring compass]

Oh, nono. alt+w (ingame) { send(ctrl+c) -> read clipboard -> extract data for wiki -> wiki page} We can do this?

[lone ingot]

I think there is already something in awakened poe trade that does that

(and the poe2 fork by proxy)

[frank drift]

I assumed that's what you meant, yes... it's conceptually exactly the same as price checking an item on trade

and yes, APT, Overlay, etc already have that functionality

[lone ingot]

can follow where this fn is called for awakened poe trade https://github.com/SnosMe/awakened-poe-trade/blob/aecf7c5128a0275ad3d460ffe91efc40a99d79b7/renderer/src/web/item-check/hotkeyable-actions.ts#L25-L27

[spring compass]

I used it in APT, but it stopped working for me.

[lone ingot]

https://github.com/Kvan7/Exiled-Exchange-2/blob/61dcd4bcd01327ced941d9e1c0369c5412a59515/renderer/src/web/item-check/hotkeyable-actions.ts#L42-L44

that's the fork poe2 version

[spring compass]

End of dreams, time for playing

7 people have checked page

[loud flint]

@worthy cape for some reason that emote deeply disturbs me

[worthy cape]

It's somewhat ambigious, isn't it?

[frank drift]

zao is an emoji connoisseur

[frosty glacier]

Hi I'm a noob. Where is the goto place to get datamined passive tree data/ image assets for poe2? Alternatively, how can I extract myself? I tried some tools but nothing worked for me..

[charred lake]

If you're not afraid of the commandline, you can use this to extract the assets https://github.com/adamthedash/poe_data_tools
poe_files --patch 2 extract "output/folder" "*.dds" (or another pattern if you want specific ones)

Then you can use something like imagemagick to convert the dds files into something more friendly like pngs
magick "image.dds" "image.png"

[frosty glacier]

Hey, thanks a lot! First tool that didn't shit itself as soon as I ran it.

Wait.. I got a 404.. can I let it ignore errors and continue somehow?

[worthy cape]

[charred lake]

Oh by default it grabs data from the poe CDN site, you can point it to your steam folder instead if you have it installed?

[frosty glacier]

Cool, will try that!

[charred lake]

For eg: poe_files --patch 2 --steam "E:\SteamLibrary\steamapps\common\Path of Exile 2" extract "output/folder" "*.dds"

[worthy cape]

I'm curious which tools you tried beforehand and how they went pear-shaped.

[frosty glacier]

I don't recall exactly what went wrong, probably some parse error/unexpected schema iirc. Tried pypoe and visualggpk.

Most likely user error though 😅

[frosty glacier]

Running it like that produced an error:

Caused by:
    0: Failed to read file
    1: Path not found in index: art/uiart/scrolling_blue_hotmetal.dds

I got around it by only grabbing the stuff I needed in the passive art folder, but thought I'd tell you.

[charred lake]

Cheers! Will take a look and see if I can fix it. I think you might be one of the first people actually using the tools so any & all feedback is appreciated 🙂

[frosty glacier]

Awesome! Really appreciate the effort to create the tool btw. I like making apps but suck at hacking game files. 😄

[charred lake]

https://github.com/adamthedash/poe_data_tools/releases/tag/v0.2.1 It'll print a message but continue on now

[frosty glacier]

Sweet, that was fast.

[royal pulsar]

Hello, sorry if its too basic question, but what would be the best source for current set of gems in poe1?

[frank drift]

try https://repoe-fork.github.io/

[royal pulsar]

Awesome, thank you.

[inland kestrel]

First, damn @charred lake - good shit and response time lmao. ❤️
Second, it was giving “I can’t search this discord channel” 🙄😅

[zenith zodiac]

https://poegems.com/ has a json file (click the question mark icon to show the link), and it's hand-maintained so it might be a bit more user-friendly than repoe, which just dumps all the data in the file (which can include removed or unimplemented gems)

[mint karma]

Do anyone have a list of rare waystone names? Even an incomplete list will help me out 😄

[modest path]

Anybody know why in datc64, selfrow refs are u16s but foreign row refs are u32s? seems weird given they an index into the same shit? Or maybe self rows are generated as u16 based on the length of the table because that's available in that scope of the generation code, and if a table gets bigger than 2^16 it will automatically switch to a u32?

erm, u32 and u64 I mean

not u16 and u32

brain mush

I kinda feel like writing a ui tool like ggg's for generating tables and code to access their data would be an interesting thing to do just to invert how I usually am thinking about the tables to make parts of it make more sense

[rapid pagoda]

IIRC, foreign row references get turned into pointers at runtime; the field is sized to allow that to be done in place

[modest path]

Wait are they 32 bit in poe1

Also interesting re: in place

[modest path]

Also I do not understand windows program files directories

Why did they even split them

[strong jacinth]

Cause Microsoft like to be different

[frank drift]

the answer is usually "for backwards compatibility"

[worthy cape]

The amount of effort they've spent on the transitioning from 32-bit to 64-bit and everything UAC is amazing.

(also frustrating)

PoB just recently ran into some fun times where the a very ancient user installation was spread between C:\Program Files (x86) and C:\ProgramData, in a semi-virtualised manner.

[worthy cape]

By convention due to how libraries with different bitness don't mix overly well, you need disjoint folders for both the system and for installed applications as there's shared components in there.
"Program Files" by convention is the name for the native platform of the OS. On a 32-bit OS that's 32-bit, on a 64-bit OS that's 64-bit.

In order to keep old apps installed in "Program Files", that was transitioned into a separate ditto (x86) folder working there were layers of WoW64 redirects and illusions on top of that to make legacy programs think that they still lived in the old world. There they could write wherever they wanted and were installed where they "knew" they were installed.

[modest path]

Did you just paste Wikipedia at me instead of some sort of ambiguous emoji reaction

Are you ok

[worthy cape]

All organic straight from the dome; horrible grammar and run-on sentences courtesy of that.

[modest path]

Emoji reaction twice if you need help

[worthy cape]

In their defense, they might've had some hopes that it was more of a transitionary thing and only for legacy applications.

[modest path]

Also /bin, /use/bin/, /sbin and /use/sbin were mistakes, change my mind

[worthy cape]

Took the rest of the world an eternity to finally dare to ship native x64 code, and we again see the Fun with arm64.

[modest path]

Yeah but how is IPv6 going

[worthy cape]

I like the distinction between bin and sbin, arguably keeping the system tools away from regular users.

[modest path]

That's what permissions are for

[worthy cape]

Debian still refuses to add the latter to regular user paths, I believe.

[modest path]

Iirc /usr was because one of the OGs ran out of disk space though

[worthy cape]

Now if you want proper separation, how about FreeBSD's three levels?

/, /usr, /usr/local.

[modest path]

I can't even deal with bsd cli util flags, let alone fs structure

[worthy cape]

Not fully up to speed if they have a clearly documented plan for the first two, but putting anything user-installed in /usr/local is such a nice thing.

This includes /usr/local/etc for rc files.

[modest path]

I mean linux has /usr/local too - it's for shit not managed by the system package Manager

[worthy cape]

Anything outside of local is in the core OS and is predictable as such.

[modest path]

RIP CoreOS. Red hat killed you too soon

[worthy cape]

Even things installed with FBSD's pkg go into local.

[modest path]

Wait, different than Core OS

[worthy cape]

I had to set up a box with FBSD recently to test something, and it's neat how much you remember all the old trivia but also how it's all so foreign after a decade of systemd.

[modest path]

The closest I ever got to bsd was being a gentoo package maintainer, and the MacBook I'm getting forced to move to by the company that just bought my employer

[worthy cape]

Any decade now. I finally got my home one sorted by buying Mikrotik again; knock on wood.

[modest path]

Oh man I used to run mikrotik

Turns out that won't help you if your fiber provider doesn't give a shit through

Though

[worthy cape]

Reminds me of when I generated loaders in Rust from the PyPoE schema dumped as dicts into JSON.
Quite fun to think about and implement, not great for build time 😄

I didn't go all the way then tho, I just used them to load each row or cells into a carrier type that was detached from the underlying data.

Strings being transcoded from what was in the disk file, arrays being copied into Vec:s, etc.

For the Hideout Editor and Model Viewer in C++, I went the less hardcoded way and just interpreted the schema at runtime, using nlohmann::json as my fundamental unit of row data 😄

Not fast nor great, but did the trick for exploratory code; particularly when the underlying schema was unreliable.

[modest path]

Oh man our rust implementation for poe1 lazy-loading from json files was rad af - Reference fields are pointers with deref impls so you can method-deref them into the row of the pointed to table... I'm currently working on reproducing that for poe2 but instead of from the json, reading directly from the datc64 files

Attempting to get as close to zero-copy as possible

https://gist.github.com/vitaminmoo/961f13703fe1a825c06e2f79e98d6626 example of generated code for mods table

So like TABLE_Mods[283].mod_type_key.whatever_column_in_that_table worked

[worthy cape]

[modest path]

Lol abuse is OVERWRITING THE FUCKING BYTES OF THE FOREIGN REF ROW WITH SOME RANDOM POINTER YOU UNSAFE FUCKS

[oak estuary]

I like the 5th(?) schema the community has now

[modest path]

Hey I don't have a schema

yet

It's on my list

lol

[worthy cape]

I made a new schema internally instead of using one of the existing ones.

[modest path]

Additional, @worthy cape

You made an additional schema

GGG needs some new employees

Not additional employees

New employees

ಠ_ಠ

[worthy cape]

Me fail English? That's unpossible.

[modest path]

I am excited about my schema plan though - It's not going to be a new schema it's going to be a meta schema

[worthy cape]

<xkcd>

[modest path]

srs

Instead of ordered types it's going to be an arbitrary list of column claims, where a claim is an offset and a length and a type, with metadata like provenance and certainty

I'm gonna import the other community schemas as those

And combine with my heuristic-based column claim generation

and hopefully a web tool that lets you edit schema live but also submit it

Or I'll run out of steam and go back to playing noita

It's a mystery

Even if I fail it's been a nice project to almost almost learn almost rust on

[worthy cape]

#[derive(Clone, Debug, PartialEq, PartialOrd)]
enum DatValue {
    Boolean(bool),

Looks like I went for the super-naive route the last time I tried using Rust.

[modest path]

If only bools were bools

I get it, alignment blah blah

[worthy cape]

impl<'a> DatFile<'a> {
  fn read_cell(&self, row: usize, col: usize) -> anyhow::Result<DatValue> {

[modest path]

https://github.com/vitaminmoo/poe-data/blob/horrible-type-spike/src/dat_parser.rs

P sure there's ways to get the type conversions to be implicit From impls or whatever vs explicit methods but I'm playing with the bat I've got atm

Once I get that code vaguely non shit I'll fix up the jsonnet code generation for tabl eaccess and then I literally don't know what I'll do; poe2 crafting is so simplistic that our original project motivation from poe1 (ml-based crafting path finding/optimization) doesn't even apply

If you want proper misguided hilarity, check out my formal parser for the clearly yolo-ass kv preformat function "language" (not adjusted for poe2): https://github.com/vitaminmoo/poe-data/blob/main/src/statdescriptions_parser.rs

I can parse preformat functions that don't even exist yet, unless some fucking muppet decides to use a new word for 10

ಠ_ಠ

[worthy cape]

That would be put a decadent in your code.

[modest path]

Heyooooooooooooooooooooooooo

Also someone already used deciseconds

Try again

"deciseconds_to_seconds".value(StatFunction::DivideByRound(10.0, 2)),

multiply_by_point_two, divide_by_ten, deciseconds_to_seconds, locations_to_metres

I only made up multiply_by_point_two there, everything else exists

oh shit I forgot times_point_two

oh shit you could do 1000%_of_value and that would be similar to things that exist, too

[spring compass]

Acceptable? There's something missing.

[strong jacinth]

[strong jacinth]

this is for both games right?

[spring compass]

Something is missing in 2, or 1?

I have faith in new events. Once I add calendar, people can send this summary from time range.

Poe2

https://www.reddit.com/r/pathofexile/comments/1igijsm/mapwatch_added_daily_summary_panel/

[spring compass]

I noticed something interesting. The first reactions to my posts are mostly downvotes. And quite a lot—1 upvote, 3 downvotes. Do I have a dedicated downvote fan, or is this normal for everyone?

Just a little longer, and I’ll make 10 accounts just to cancel out this effect.

[modest path]

@spring compass reddit def has some weird vote obfuscation stuff that it does iirc

[spring compass]

I have a conspiracy theory in my head. People see that the site asks for a file from their computer, panic, leave, and hit the downvote.

[frank drift]

fuzzing

it shows fake upvotes/downvotes so that nefarious users can't track exact vote totals

[oak estuary]

I think you'd be lucky if any downvoters clicked the link at all. But like the others said, vote fuzzing is the likely culprit.

[spring compass]

I don’t have any ads—it’s all about social points, or maybe I’m just too lazy. My last PoE project had 8 returning users. Those numbers are way too low to monetize.

[grave wren]

Id assume posting progress when people actually play should get you more users that are willing to try your site

[spring compass]

I don’t know… PoE2 has a lot of players, but whatever I post on Reddit just dies in new.

I just want to point out that 8 months ago I did the same thing and got way more interest. Maybe this Mapwatch was only needed by me and people like me.

Something happened that took away trust. Just speculating.

[frank drift]

helps to post at the right time of day, and the right day, etc... there's a whole rabbit hole you can go down there

[strong patio]

poe2 subreddit seems more "normie" than poe subreddit. This might explain the difference in interest in posts such as yours

[frank drift]

NeverSink gets annoyed at how good I am at reddit posting

[loud flint]

does PoE 2 subreddit have a lot of users?

i'm still guilty of using the "old" one as my source

[wind garden]

it's abotu 1/3rd the size

300k vs 900k

given how many more poe2 players there are vs poe1, seems the % of them on that subreddit is smaller

[frank drift]

poe2 subreddit seems significantly more active though

[golden knot]

Reddit is normally a garbage heap anyway but right now it's even worse because all the subreddit wants is memes. Thousands of upvotes for the "duplicate Izaro" thing every single day.

Probably just not the time to post anything attempting to be serious

[frank drift]

to some degree yeah but not something insurmountable

for sure reddit would be more receptive after the announcement of a new league or similar

[somber mortar]

Not sure if you still want or need a list, it potentially may just be any word in the words table that has 57 in it's SpawnWeight_Tags

["Dire", "Cinder", "Iron", "Steel", "Agony", "Dark", "Cruel", "Death", "Chaos", "Blood", "Terror", "Pain", "Savage", "Grim", "Abandoned", "Ancestral", "Ancient", "Anguish", "Arcane", "Arid", "Barren", "Blasted", "Bleak", "Breeding", "Buried", "Cabal", "Cursed", "Cyclopean", "Damned", "Desecrated", "Desolate", "Destiny", "Dread", "Dream", "Evil", "Fetid", "Forbidden", "Forgotten", "Forsaken", "Fungus", "Ghastly", "Ghost", "Godless", "Graven", "Hate", "Haunted", "Hidden", "Horror", "Insane", "Lost", "Morbid", "Mystic", "Nightmare", "Profane", "Putrid", "Remote", "Rotting", "Rugged", "Secluded", "Spectre", "Stained", "Torture", "Tranquil", "Twisted", "Unholy", "Vortex", "Warped", "Wraith"]
[" Abyss", " Apex", " Bind", " Cage", " Chambers", " Coffers", " Core", " Court", " Cradle", " Crag", " Cramp", " Crate", " Crest", " Depths", " Dregs", " Edge", " Frontier", " Haven", " Keep", " Nexus", " Oubliette", " Panorama", " Pit", " Point", " Reaches", " Refuge", " Refuse", " Remains", " Remnant", " Rupture", " Sanctum", " Scum", " Secrets", " Shadows", " Shard", " Slag", " Solitude", " Spires", " Trap", " Vault", " View", " Vine", " Waste", " Ziggurat", " Abode", " Quarters", " Roost", " Sector", " Zone", " Precinct", " Dwelling", " Shelter", " Course", " Path", " Compass", " Crosscut", " Direction", " Navigation", " Route", " Trajectory", " Bearings", " Expedition", " Intent", " Vector", " Control", " Charge", " Progression", " Order", " Choice", " Resolve", " Determination", " Gambit", " Stone", " Carving"]

[zenith zodiac]

thats not the conspiracy, the real conspiracy is that Reddit is a p2w game

https://www.reddit.com/r/marketing/comments/1496xp5/comment/jo3ojqf/

[frank drift]

22 points 1 year ago...

reddit does love a good conspiracy theory though

[hushed relic]

I have no idea what you're talking about (not being sarcastic here, but maybe I forgot something)

[frank drift]

I think you complained about my google ranking

or was that Zoey

[hushed relic]

Oh, that, I was wondering why your profile was listed as part of the filterblade results, until I realized it is a totally valid URL, since, you're linking it and people use it, google recommends it!

[frank drift]

which is mainly reddit juice I think yeah

[hushed relic]

probably, yeah!

[mint karma]

Thats amazing, thanks!

[swift thorn]

Oh boy.
Imported affixes list for bows from poe2db (made a script to transform it into json), in order to read and process on chrome console so I can identify affixes and ranges for bows, and then calculate min and max DMG range for divines rerolls and how much DMG it gains through iron rune slotting

https://media.discordapp.net/attachments/1324862098567725066/1336371088258498600/image.png?ex=67a39010&is=67a23e90&hm=cc6fc9c317f43b18b0eb98b81a2ece9e15dc96fcfea02e1edeb57f0ed0eb0106&

A lot of work >.<

And I'm no web developer xD

[oak estuary]

Someone made something very similar, but for in-game: https://github.com/shonya3/poe2-weapon

@crisp kestrel

[swift thorn]

That's nice 😄
I was able to finish the iron rune simu and write on the actual trading site screen, but the div-reroll is what I really want. Not far from it though, I already have everything that I need sorted out, just need to calculate base weapon damage (taking out quality if needed, affixes and runes) and then calculate dps with min and max possible rolls for each affix

the saddest part was parsing and getting used to js (I'm a C# developer)

memory management is quite different

[crisp kestrel]

btw, not only ingame, it would popup anywhere if you press CTRL+C and have item text in clipboard in this format.

for example with this button which does not exist in poe2 yet

but someone made this button for poe2

[mint fable]

Hey everyone!
I’m working on a personal project and noticed some links to other PoE2 projects, which got me curious. I checked the docs and saw there isn’t an official API for the game yet, but I was wondering if there’s another way to access the data. Does anyone know how others are getting it?

I’d really appreciate any pointers! Thanks in advance!

[strong patio]

no

as of yet poe2 dev apis are coming but in terms of time frame its anyones guess

My assumption would be no point working on apis until the game has fully released.

[oak estuary]

Depends on what data you're talking about @mint fable . Reading the GGPK or the trade site is what most people are doing right now

[mint fable]

Got it, thanks. I saw in someone’s project that they were using a database.json file with extracted game data. Do people usually generate these manually from the GGPK, or are there tools/scripts to automate the process?

[spring compass]

https://github.com/SnosMe/poe-dat-viewer

[oak estuary]

This is the other project people usually use: https://lvlvllvlvllvlvl.github.io/RePoE/

[mint fable]

Thank u guys, that’s what I was looking for! Appreciate it

[frank drift]

I believe the URL is now https://repoe-fork.github.io/

or https://repoe-fork.github.io/poe2/ for poe2

[swift thorn]

got the numbers \o/

[swift thorn]

Quite useful for flippers xD

[somber mortar]

Issue opened: important core feature.

can you get around to putting in the "estimated price" thing [...] ?
the single most annoying thing about this [...] is that i have to constantly click on the checkboxes and hover over search
(image)
that. that is what we want. that is the only thing we want.
wow super useful issue lol :)

[worthy cape]

The ML-driven thingie that used to be up-top?

poeprices info, was it?

[frank drift]

probably

[somber mortar]

yep!

as far as I can tell, not yet updated for poe2 so I had removed it since it was returning chaos and in reference to settlers. Since it is machine learning I would guess it may take some time to train up back on exalts.

[frank drift]

I'd be surprised if it's been updated in years

last I looked at it the model was very mid 2010s too, you could probably do much more interesting things these days

[somber mortar]

oh no, i'm not sure i want to go down a rabbit hole lol. Maybe I'll take a look though. I did do ML stuff recently though for college so that sounds really fun :)

[frank drift]

I've seen projects here and there but none that really got traction

[crisp kestrel]

now can copy like that

from the trade site. There was a minor limitation

https://github.com/shonya3/poe2-weapon/releases/tag/app-v0.5.0

[swift thorn]

I've added the ability to account for runes for the DPS/DivReroll calculations, after some testings seems pretty accurate

after buying and rerolling

[swift thorn]

https://media.discordapp.net/attachments/1324862098567725066/1336808432287613051/image.png?ex=67a5275e&is=67a3d5de&hm=8ca8c04b13c5a2329bdd46d9aac4b63e8da4f64f5fd10bbd39dcdad38f1c418e&=&format=webp&quality=lossless&width=2880&height=1204

Now it appears on the item itself.
I need to add support for other weapons, sorting, culling, UI and then I think I'll publish

[strong jacinth]

Looks neat

[spring compass]

Looks good, but I’m having dejavi. I’m almost sure that something similar exists for PoE1 as a browser extension.

[swift thorn]

thanks

[swift thorn]

I would like to know the name of it if it does exist, I can borrow some ideas

[spring compass]

I can't help you—Lily used it on stream, but that might have been years ago

[spring compass]

Should I wait with updates until event announcements or until the events start? Waiting for the new league might be too long, motivation could disappear.

In other words, are you waiting?

This is sad:

[swift thorn]

Ok, thanks

[spring compass]

What else could I add or change? I could include: the biggest WTB whisper, number of players who joined the HO (but does that even matter to anyone?)

[worthy cape]

I think the safe zones are typically called "towns". Also a bunch of spelling for "average", "level-ups", "NPCs", "received", "sent".

I was gonna suggest tracking a ratio between whispers identified as outgoing trade whispers vs. responses, but many people trade without saying anything, or reply with some variation of "sold"...

Maybe: "longest streak of trade whispers without accepted trade?" 😄

[strong jacinth]

I haven't really dove too far into client.txt, does it record items received etc (picked up or traded), and it doesn't show monster kills yeah?

[spring compass]

Sadly no. There is only line: "Trade accepted" without any more info

[worthy cape]

Way less granular and informative than that. Trades are "accepted" or "cancelled".
General stats like monsters and bulk info like items are generally not emitted.

It's a mix between your chat box and diagnostic information.

[strong jacinth]

That's what I thought, so you'd only be able to get the item from the whisper. And for monster kills, using the / command

[spring compass]

What bothers me the most is that there’s no info about map content (Breach, Harvest...) - if there is no master connected to that content. I can search for masters voice lines.

@worthy cape are you in New Zealand?

[worthy cape]

Yep.

[worthy cape]

Haven't visited since the initial reddit link.

Or if it was a direct link in here, been a few days.

[zenith zodiac]

iirc only the item name is logged, not the content of the tooltip popup, so you can't see much about the item

[frank drift]

number of players who joined HO would be cool

maybe number of different zones entered

[worthy cape]

Or for some period of time, a literal _ underscore.

[strong jacinth]

Yeah, you'd be able to pull the item name (not the stats)

"I wanna buy your Call of the Brotherhood for x price"

[strong jacinth]

Yeah that's it

You could do like a "most traded item" column

[spring compass]

For most people, it would be one item in each column

[strong jacinth]

I haven't used the site for a number of days now, are those tiles clickable?

[spring compass]

No, but I'm thinking about it. For more tiles, moving them can be useful.

[strong jacinth]

Not sure if you'd have a separate page for after you click the tiles or not, if you wanted to provide more in-depth detail about any of the current tiles

[spring compass]

I can't, I don't want to or I don't know. I'm not doing it today. Have a great time working on your projects!

[strong jacinth]

That's completely fair!

Your "Bigest trade" tile is missing a 2nd g btw. But other then that it's looking solid

[foggy stag]

Trade accepted and the cancelled one also triggers in the logs when you talk with NPCs, Una for example. It's not exclusive to players 🙂

[strong jacinth]

yeah I just noticed that:

haven't traded a single player

[foggy stag]

Could probably do something clever with detection of npc talk / whispers before the event. it's not gonna be bullet proof though xD

[clever thistle]

could help if you customise your "ty" reply

@To Flickersofkalguur: ty4t:: Flickersofkalguur (Einhar's Memory of Harvest Beasts for 3 Divine)

[oak estuary]

To be fair, it does say and NPCs 🙂

[spring compass]

It literally says that NPCs count too. I could search for transactions happening right after a player joins the instance. But there’s no info about when they leave (PoE1 has it, PoE2 doesn’t).

What about cases where a player joins, you trade with an NPC, then with the player (2 trades) or trade 10 times with the same player?

NPC interactions don’t generate any logs, except for the same trade entries as with players. I don’t see a good solution.

I added a note that NPCs count too and left it like that, but I probably need to bold it now.

I’d really love to remove NPC transactions—just tell me how! 😆

[spring compass]

This channel is actually very useful! You just gave me an idea for a new chart—a word usage map.

[spring compass]

or this is useless?
https://user-images.githubusercontent.com/4129778/88903326-cbb55a80-d253-11ea-9fb3-ecca1e1ef67f.png

[swift thorn]

Sorting and culling is already working, will add UI and see what is the better way to release it

Current Features:

• Calculates dps with iron runes (even if it has something equipped);
• Calculates what is the minimum and maximum possible dps after using a divine orb;
• Says how good is the current item rolls within their affixes brackets in percentage;
• Allow for hide and unhide items from the trading results that have more than x/y prefixes/suffixes;
• Can sort the items based on maximum possible dps after divines;
• Can sort the items based on current rolls within their brackets.

[floral marten]

I woud like to share this passive tree view in web (WIP), using the same assets from POB2, meaning DDS and compress assets.
Only Windows pc and MacBook are able to use this.
this implementation use PIXIJS and 2darray texture support (PR was already submitted into PIXIJS) in batch mode, thx Zao for teaching about it 😄 with POB2

Here is the live demo https://justjuangui.github.io/, and here the code https://github.com/justjuangui/justjuangui.github.io/tree/main in case you wanna learn.

The passive tree still have white arc, because IM learning how to map a textura correctly with UV and shaders, Im new in the Graphic world, so please forgive me.

If you tried let me know how you feel the performace while zoom in/out, pan around the tree

[fickle yew]

Nice work! Very smooth.

[swift thorn]

zooming in and out was very smooth, loading took a while though

tooltips is also working pretty well btw

[fickle yew]

Mine is svg based, so it can't compete with pixijs, but I'm happy with it for now.

[glacial lake]

Does anyone have resources on how to parse bundle.bin files?

I found the PoE dat schema https://github.com/poe-tool-dev/dat-schema

But the schema doesn't really make sense to me, and I haven't found any reference code for parsing these bundles

[oak estuary]

https://poedb.tw/us/GGPK

[strong jacinth]

Yeah, I just wasn't sure which NPCs counted.

[glacial lake]

edit: nvm, there's a bundle schema on that page. thanks

[charred lake]

https://tenor.com/view/oogway-my-time-has-come-gif-8019684

https://github.com/adamthedash/poe_data_tools

Here's an implementation in rust. There's also a diagram of the format itself in the readme

[glacial lake]

This is super helpful! Thanks so much!

[spring compass]

    ["mirror", "<img title=\"Mirror of Kalandra\" src=\"https://web.poecdn.com/gen/image/WzI1LDE0LHsiZiI6IjJESXRlbXMvQ3VycmVuY3kvQ3VycmVuY3lEdXBsaWNhdGUiLCJ3IjoxLCJoIjoxLCJzY2FsZSI6MX1d/7111e35254/CurrencyDuplicate.png\" style=\"width: 36px\">"],
    ["divine", "<img title=\"Divine Orb\" src=\"https://web.poecdn.com/gen/image/WzI1LDE0LHsiZiI6IjJESXRlbXMvQ3VycmVuY3kvQ3VycmVuY3lNb2RWYWx1ZXMiLCJ3IjoxLCJoIjoxLCJzY2FsZSI6MX1d/e1a54ff97d/CurrencyModValues.png\" style=\"width: 36px\">"],
    ["exalted", "<img title=\"Exalted Orb\" src=\"https://web.poecdn.com/gen/image/WzI1LDE0LHsiZiI6IjJESXRlbXMvQ3VycmVuY3kvQ3VycmVuY3lBZGRNb2RUb1JhcmUiLCJ3IjoxLCJoIjoxLCJzY2FsZSI6MX1d/b102771c23/CurrencyAddModToRare.png\" style=\"width: 36px\">"],
    ["chaos", "<img title=\"Chaos Orb\" src=\"https://web.poecdn.com/image/Art/2DItems/Currency/CurrencyRerollRare.png\" style=\"width: 36px\">"]
]);``` Are these links correct, or should I not be doing it this way?

Can they be mad that I’m not using my own hosting?

[swift thorn]

added buttons 😄

[spring compass]

I think I’ll be posting updates slowly, about once a week or once every two weeks. Though I still have ideas for two more updates. I also don’t want it to grow too much.
https://www.reddit.com/r/pathofexile/comments/1ijdoph/mapwatch_you_can_check_your_ingame_stats_here/

and:

2025/01/20 20:14:34 123123 123123 [DEBUG Client 1234] [InGameAudioManager] TalkingPetAudioEvent 'PlayerToPlayerTradeCompleted' triggered

**TalkingPetAudioEvent **

[worthy cape]

This reminded me of that there's a rule on the sub about excessive self-promotion. Probably a good thing to look that one up if you're not already familiar with it, I believe it's fairly generous.

[golden knot]

Yeah, Rule 9. This channel is full of people rooting for your tool success but be careful on Reddit, you're way under the 1:10 promo ratio.

[sturdy drum]

I liked your post I support all the tools! Well unless it literally played the game for you, that I wouldn’t support

[spring compass]

 A recommendation is to follow the 10/1... If you don’t have that much to say, that’s fine...```

[frank drift]

yeah it's fine

[molten kernel]

does anyone have access to the standalone image file for this glyph:

in the style of this?

not seeing it in any ggpk viewers

[brittle oasis]

dont think it has one, probably easier to just take the outlined version. remove the black outline and recolor the orange to that brown and same thing

[frank drift]

not really same thing if you're looking for transparent/lossless, short of redrawing it

[swift beacon]

I have all of them. Let me know if you need any others.

[hazy wigeon]

Can any filter whizz's here confirm for me if its impossible to actually hide relics in poe2?

#2┃general message

[strong jacinth]

Yes you can

[hazy wigeon]

Okay, I cant figure out why on earth my relics arent hiding while somehow the Hide block is applying the styling to them

These are my only blocks that even mention relics and they are pretty much at the top too

[frank drift]

that's on the list to get fixed

[hazy wigeon]

oh, can you not hide them?

[strong jacinth]

Ahh I didn't know that, my bad

I guess i don't run Sekhema enough

[hazy wigeon]

Im confused now which is it lol

Cuz atm I am seeing this which shouldnt be possible, if an item is shown with that "very smol text" styling, that only ever shows up in Hide blocks anyways with my style sheets so it implies somehow a Hide block... didnt Hide the item

[frank drift]

anyone can submit a FilterBlade pull request to fix customizer issues like that, I'll probably do it in the morning now that I've been reminded

[hazy wigeon]

This isnt in filterblade

this is a .filter file itself, this is an in game issue I am having

I dont use filterblade, not related

You can see my filter file I posted up above
#tooldev-general message

So yeah if anyone knows whats going on here lemme know, Im just gonna assume for now its an in game bug or something GGG did to intentionally make it so relics just cant be hidden, which sucks

cuz Id very much like to not pick these dumb things up, they clog up my inventory and are worthless

[worthy cape]

Hide
    BaseType = "Amphora Relic"
    SetFontSize 48
Hide
    BaseType = "Sapphire"
    SetFontSize 48

Filtered vs. holding the show button:

[hazy wigeon]

yeah looks like they just straight up cant be hidden eh

[worthy cape]

I'm going to be really confused the next time I play my live toon and I find a relic and a jewel in my inventory, as I pulled them out of some stash dump tab to test this 😄

[hazy wigeon]

XD

[eternal sand]

Hey, anyone know if there is a library for simulating crafting?
Basically something which consumes the RePoE data and lets you generate items.

item = Bow(ilvl=20, rarity='normal');
print(f"{item.base_type=}")
item.apply_currency('transmutation_orb')
print(f"{item.mods=}")

[eternal sand]

This looks like the best option - https://github.com/NickRyder/PoECraft/
Though not updated in a while, and now uses rust instead of python - I'm sure there's some method of interop

[remote kite]

hey guys, im trying to gather all uniques in poe2.
https://repoe-fork.github.io/poe2/ basing here, just parsing the json
question - how do i link a certain unique to its basetype (from baseitems?)
e.x. astramentis -> stellar amulet?

[carmine merlin]

UniqueStashLayout.dat is how I assume most people do it if they're not grabbing it from trade

[wet forum]

Prob doesn't matter, but maybe for documentation the header section in Bundle is the OodleLZ_SeekTable https://github.com/themixednuts/poe2kit/blob/master/bundle/src/lib.rs#L241

https://github.com/EpicGames/UnrealEngine/blob/0c544b150542b59fc87bdcf64caae09f4e3bc11c/Engine/Source/Runtime/OodleDataCompression/Sdks/2.9.12/include/oodle2.h#L473

[spring compass]

The first post what started with 4 ups before he got any downs. I always have to start by linking post here, ty guys.

[brisk kernel]

There's a bug with the stats on passive tree. Showing flipped stats btw after the most recent update.

[swift thorn]

I've uploaded my tool (only works with bows for now) in case someone want's to try it:
https://github.com/JohnnyDalvi/PathOfMartialWeapons

[spring compass]

Has there ever been a version of Mapwatch that stored any data? I only know of one, which was shut down by its creator, unnecessarily.

[charred lake]

Oh great, thanks!

That 2nd link gives a 404 btw

[wet forum]

Have to sign up for UnrealEngine team

[grave wren]

dont think so, maybe he just stored the html output

[swift thorn]

O.o
What are you using unreal engine for?

[wet forum]

At the moment, nothing crazy. You can sign up to view their source. I use it to understand some internals and fix issues when I can.

Since they bought RAD a while ago, Oodle source is included

[swift thorn]

oh, got you

[spring compass]

https://github.com/briansd9/exile-diary?tab=readme-ov-file
He thought it was another tracker like WealthyExile or PoeStack… That’s not what I want to do. The work isn’t that much, but the costs are too high.

[grave wren]

yeah og mapwatch also didnt do it i think its more than fair enough to keep it simpler

[spring compass]

Does OAuth work outside of browsers? Do .exe programs only use cookies and a possessed ID?

[grave wren]

mostly cookies and poessid i think, i think dealing with the secret management side is iffy so you'd need to provide a backend as you don't want to share the secret ggg gives you

[oak estuary]

GGG provides support for public OAuth: https://www.pathofexile.com/developer/docs/authorization#clients-public

[frank drift]

yes

[grave wren]

ah misremembered it as not being available due to the missing service scopes

[frank drift]

I think it's generally the best way to handle logins, Chaos Recipe Enhancer, Exilence Next, etc all use oauth

I think APT too, though I rarely use APT

[lone ingot]

if anyone didn't tell you yet, my advice is set the alpha channel of the text, border and background to 0

it's "hidden" then

[spring compass]

https://youtu.be/bb1n7Hdr0pE?si=HlaT8aCePhHlJInK&t=231
How did he detect changes in the file without reselecting it?

[frank drift]

what do you mean reselecting?

it's probably just polling the logfile?

[spring compass]

The old Mapwatch checked for new changes in the file. The user selected the file once, and when the game added new data, the page refreshed it. I didn't know that was possible with files saved in system folders. Data protection etc.

[ashen aspen]

probably polling using https://developer.mozilla.org/en-US/docs/Web/API/File_System_API

might also soonish be possible without polling: https://developer.chrome.com/blog/file-system-observer

[spring compass]

I tried it, work only for files outside system folders (Program Files)

[ashen aspen]

hmm, was just a guess ;) idk

[frank drift]

if the source exists somewhere you could just look I suppose

[spring compass]

WTF, AI:
In the case of web applications, the only way to monitor files is to upload them to the server. This allows the server to analyze the file and track changes, but it requires copying data outside the local system.
I guess I’m stupid because I don’t understand any of this. If I were to upload a file to the server, I could later compare its changes. If I want to do this locally, the browser doesn’t have such permissions.
edit: It will only work if the user launches the browser in administrator mode.

[spring compass]

He uploads the file to the server.

[frank drift]

aha

[spring compass]

What nonsense, I leave it.

[spring compass]

Are ppl in China forced to play they own version of poe?

@worthy cape you have a friend!

[worthy cape]

[frank drift]

for poe1 I think the vast majority find it much easier to just play on the China client

the few that choose to play on a non-Chinese client will most commonly use Kakao afaik, the South Korean client

the one where messages from it do not get logged

[loud flint]

The Chinese one is similar to League right? Just 100% ran by Tencent and no linking to other realms?

[frank drift]

it has Tencent specific adaptations for the Chinese market, like loot pets semi famously

a bunch of streamers participated in a China race a few years ago, you could find their reactions on youtube

[crisp kestrel]

yes. It can be normal oauth via server, or just public client

[grave wren]

https://github.com/mapwatch/mapwatch

especially since it used to be connected to a downloadable program

Maybe that functionality depended on another app with live view

[crisp kestrel]

🐔

https://github.com/mapwatch/mapwatch/blob/master/packages/electron2/src/electronBackend.js

[spring compass]

I played PoE for so many years and never knew there was a desktop MapWatch connected to a website. So that person from YouTube had downloaded the MapWatch program earlier… sad.

[austere jacinth]

Does anyone know what the various level values in the PoB skill luas translate to?
For example, SummonPhantasmFadingProjectile has these values:
[1] = { 0.80000001192093, 1.2000000476837, damageEffectiveness = 1.5, critChance = 5, levelRequirement = 1, statInterpolation = { 3, 3, }, }

What are the first two values, and what is statInterpolation?

[oak estuary]

https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/dev/docs/addingSkills.md#combined-data

[austere jacinth]

Thanks! ❤️

[lyric umbra]

anyone know what the search query limit is on trade?

making a tool and would like to avoid getting rate limited lol

its basically a bulk pricechecker, would still have to copy the item content to your clipboard, but it doesn't get any better without APIs

[frank drift]

if using the api, there should be rate limit headers

[rain wren]

yeah, headers identify it, but basically it maxes out at 30 requests over 302 seconds, with less requests its a bit more lax

what do you guys think about trade2 api? I find that filters endpoint doesnt map specifically to item properties

[unique lantern]

Hey guys, I'm new to developing poe tools but I was wondering if there's a way to watch "events" in PoE2 such as entering low life state, being debuffed etc
I just thought it would be useful to catch those and use custom sound and screen alerts because I'm pretty distracted and I miss the game original cues xD

[frank drift]

no

anything like that would go against the whole design philosophy of PoE

[unique lantern]

Ohh okay, sorry I wasn't aware
But thanks!

[frank drift]

they specifically do not want 3rd party tools to become mandatory because they give a gameplay advantage

[unique lantern]

Makes sense honestly

[loud flint]

I wish Riot Games did the same

[worthy cape]

I'm always baffled when my mates run LoL tools like porofessor with artificial objective timer overlays and matchup info/builds.

[lone ingot]

that's what the LoL auto chess has become, mandatory 3rd party app or lose

[spring compass]

GGG doesn't like me. They added lines without a date or time to the logs (or maybe I just didn't notice them before), and it crashed my page.

    Skill Type Id = 23757
    Skill Instance Id =34947
2025/02/08 20:56:25 ***** LOG FILE OPENING *****```

[worthy cape]

As long as you're aware when scanning the file, it should be possible to re-join such lines into a multi-line body based on the lack of "header" on the continuation lines?

Might hamper any concurrent processing of log entries as they're apparently not 1:1 to lines.

I wonder if things like green text from reliquary keys can have embedded newlines and if they are emitted to log.

[spring compass]

I'm just looking for lines with regex. There was an error here because there's no entry for close, so I took the index-1 and used the data from it.

    contentPrevious = parserGamingSessions(lines[index - i--].trim(), false);
}``` Work enough good for me.

[worthy cape]

Ah, something like using the timestamp of the entry above to tell when that session ended?

[spring compass]

y

I got scared when I visited the site today and encountered an error. I wonder how many people saw it.

[loud flint]

At least they removed the statistics of augments from the API

Enough off-topic from me tho, carry on with business as usual

[charred lake]

Anyone work with rust / c++ ffi? I'm trying to create a small sys crate for https://github.com/matyalatte/Texconv-Custom-DLL/tree/main so I can get at the dds file contents, but the linking / build script piece is breaking my brain

[worthy cape]

It's deceptively straightforward until you need to make it portable and adapt to some existing native codebase.

I forget if I ever ported pyooz and pydds to Rust or if I saved that for the next time I think Rust is a good idea.

[spring compass]

Even the biggest off-topic enthusiasts write more on their own than they spend time here. Except for zao

[loud flint]

Zao is the emote connoisseur

[zenith zodiac]

I've been using the image.rs fork from this pr https://github.com/image-rs/image/pull/2258 and haven't had any issues

[charred lake]

Ooooohhh awesome! Lost some days in the rabbit hole but hey at least I learned about block compression & FFI 😄

[spring compass]

NPCs like to talk.

Have you ever come across a list of all NPCs, bosses who say something?

[velvet fog]

https://poedb.tw/us/NPCs

[spring compass]

More than nice, ty

@velvet fog Lycius, Midnight's Howl is missing

] Lycius, Midnight's Howl: The smell of fear betrays you!```
It's impossible not to smile

[spring compass]

There are (at least) 666 talking NPCs in both poe.

[spring compass]

I'm starting to do stupid things, aren't I?

[worthy cape]

Still sane, exile?

[strong jacinth]

Definitely not at this point

[worthy cape]

[strong jacinth]

Someone out there is looking for a tool with this exact feature

And you've just done it, run with it

[spring compass]

Jun have 121 voicelines?

"I love all children the same." ggg

[oak estuary]

Well one is a boss fight, the other is an NPC with dozens of personal relationships

[spring compass]

True, but Einhar 24, Niko 18

[oak estuary]

18 immortal syndicate members to talk to. Beasts don't talk much, and Niko just talks to himself

[strong jacinth]

Niko is high af 24/7

all that delving

[spring compass]

This looks really bad, I'll continue spamming in a few hours.

[swift thorn]

made a quick script to sum all value we have (up to 100 items) selling in our account, search for your name on the trading website, order by most expensive and run it:

async function loadAllItems()
{
        while (true)
        {          
            const loadMoreButton = document.querySelector('.btn.load-more-btn');
            if (!loadMoreButton) break;
            loadMoreButton.click();
            await new Promise((resolve) => setTimeout(resolve, 500));
        }
}

await loadAllItems();
totalDivines = 0
totalExalts = 0;
document.querySelectorAll(".row").forEach(mod => {mod.querySelectorAll("div.right > div > div.price > span").forEach( price =>  
    {
        priceText = price.textContent.replace("Exact Price: ",'').replace('Asking Price: ','').split('×');
        //console.log(priceText);
        if (priceText[1].includes("Divine"))
        {
            totalDivines += Number(priceText[0]);
        }
        else{
            totalExalts += Number(priceText[0]);
        }
    }
    )
});
console.log("Total Divines: " + totalDivines);
console.log("Total Others:  " + totalExalts);

only accounts for divines vs others for now, because I made it originally just for me >.<
but should be easy to edit

[somber mortar]

You probably already thought of this but, this seems like it would be really cool to have at the end of leagues as like a “Spotify wrapped”. All the weird stats would be cool to have in it, “Niko said Hehehehe 408 times, that’s more than 4 times per day” or other stats like “you spent 78 seconds in hideout on average between maps, that’s a lot of wasted xp”. Overall would take a lot of UI and UX work, but would be really cool

[crisp kestrel]

https://www.bleepingcomputer.com/news/software/brave-now-lets-you-inject-custom-javascript-to-tweak-websites/

[frank drift]

... that's just tampermonkey?

anyway you couldn't pay me to touch Brave cause it's run by Brendan Eich

[grave wren]

firefox all the way

[oak estuary]

They just made tampermonkey native essentially. Now if they ever follow Chromium updates to remove support for manifest v2 this will still work (not that I think they will)

This comment in the related issue made me laugh though: https://github.com/brave/brave-browser/issues/25586#issuecomment-2645946935

[strong patio]

Edge >

[zenith zodiac]

is it still called edge? I would have expected it to be Copilot 365 Series E by now

[spring compass]

Just a moment, and I'll split the page.

https://mypoe.site/mapwatch/
Shadow update.
Tooltips + next chart on the end of the page.

[charred lake]

Added direct png export for assets to my cli tool https://github.com/adamthedash/poe_data_tools/releases/tag/v0.3.0
@frosty glacier might be of interest to save you the pain of piping stuff to imagemagick

poe_files dump-art --patch 2 /output/folder "*.dds"

[frosty glacier]

Yo that's huge! Thank you.

[charred lake]

No prob, haven't tested it against a lot of stuff yet, so if you run into anything let me know!

[ebon oasis]

wheres poe2.ninja grabbing build info from, do we have some profile access now ?

[frank drift]

Rasmus has access to a beta version, there's no public access

[ebon oasis]

thx

[spring compass]

Poe1 has better, more interesting NPC dialogues.

Not too dark?

[frank drift]

*better NPC hymns

[abstract tusk]

Hey, who can I talk to when it comes to the API early access? I'm the creator of https://poe2.app. I saw the news that poe2.ninja has early access. Is there a way to also apply for that?

[carmine merlin]

I would assume not. You'll probs get access to it when it goes fully public

[clever thistle]

aren't they all just builds pulled from Mobalytics?

[spring compass]

I'm currently looking at this site. Is that allowed? To make a page with links and data from another site?

@worthy cape what do I have to do to be green?

[clever thistle]

@spring compass #tooldev-general message

[hexed mesa]

a little bit of paint works too

[spring compass]

"you think you deserve" think and deserve, not yet

[clever thistle]

dw if i got in you should be fine 😉

[analog pagoda]

eh i can't get it even after relog wtf

[analog pagoda]

alright.. so where to post it so someone help me

[oak estuary]

https://www.pathofexile.com/forum/view-forum/2214

[lone ingot]

god I am so dumb. I only just realised the intersection is the Tags table after picking this back up

[lone ingot]

are these ordered, like can I just use the whole thing as an ID? or do I need to build more complex data structure to map them?

maybe @zenith zodiac knows also

[velvet fog]

It's ordered

[molten kernel]

@carmine merlin Are the skills in POB with level differences from their existing counterparts displaying official stats or just copied from the originals?

I assume just the old versions for now? not sure if they need disclaimers or something

[oak estuary]

Which ones are you looking at? Old versions though, since we don't have the data yet for the new ones. E.g.

[molten kernel]

Just in general. Just worried players might get confused and think the stats were datamined/known already

[tranquil token]

@velvet fog https://poedb.tw/us/Doryanis_Touch#Doryani#039;sTouchdoryanis_touch "Doryani's Touch" skill information is not displayed.

[velvet fog]

fixed, thanks

[eternal sand]

PoE2 filters removed Import statement?

Import "Base.filter"

[daring moss]

Yes, import isn't a thing in PoE2

My guess would be that Import was added to poe1 after they split poe2 off into its own game and never received those additions

similar to how you can't CTRL+RMB to move arbitrary amounts of currency items into and out of your stash, or see the total amount of a currency in trade windows

[spring compass]

8 days have passed, I’ve done some things. I highly encourage you to waste some time! 🙂
https://www.reddit.com/r/pathofexile/comments/1ip5nd5/mapwatch_check_more_of_your_statistics_here/

https://www.reddit.com/r/pathofexile/comments/1iozgt3/mapwatch_check_more_of_your_statistics_here/

[spring compass]

6 days until the new league... hmm, I need to spend the weekend working on the date range for the second summary.

[spring compass]

I deleted the previous post, 5 points in 6 hours. Tragic. I uploaded a new one.

Are you working on something? It’s gotten quiet. Maybe you have some inside info and are preparing tools for the event?

[nocturne finch]

@spring compass Cool website! Works surprisingly well.

[spring compass]

It really doesn’t make sense. If the post doesn’t get a boost at the start, it just disappears.

At least it’s already in 5 place on Google. It’ll be a site for the old-timers those who remember that MapWatch existed and will search for it out of nostalgia.

[spring compass]

Thanks! Check the RAM usage when you upload a 200+ mb file. Chrome takes up 1.1gb

[loud flint]

I wonder if WASM could make that more effective

[daring moss]

is the big amount of used RAM from storing the parsed info in a wasteful way or are you loading the whole file into memory?

[oak estuary]

Honestly if I didn't already know what MapWatch was I wouldn't click on that title. But overall, I wouldn't worry about internet points, it'll bring you down.

[true mantle]

Very cool stuff. Just forked it and looking forward to taking a look under the hood.

I'm new to PoE and am a software engineer, so I'm looking forward to contributing to some of the open source stuff out there.

[lone ingot]

wasm can almost always help if you use it properly

with a file upload I think you're just limited to the browser though, that said post processing of the file could be what it's bottlenecking on

Im assuming it's just because the entire file is stored on the input https://github.com/ugimser/mapwatch/blob/4ec36252033d088b30ebe7e2b804eb7ced717fea/js/app.js#L38, you can just drop the element and the RAM should clean up (post you processing the data obviously)

I guess the FileReader reference might also not be being gc'd possibly, so after finishing you could also just deref that by setting it to undefined or something

1.1gb in use post upload almost 100% means there's some kind of dangling reference that the gc doesnt know about/doesnt want to touch

[worthy cape]

I like the note on https://developer.mozilla.org/en-US/docs/Web/API/FileReader/readAsText:

Note: This method loads the entire file's content into memory and is not suitable for large files. Prefer readAsArrayBuffer() for large files.

readAsArrayBuffer still seems to load the whole thing 😄

[lone ingot]

yeah pretty much my thoughts, just standard web dev too many API issues

my default is just go for arrayBuffers/uintArray, stream and do what you need then discard

has anyone messed around with the web standard audio stuff

so many things that just make you go "why", pure design by committee stuff

[worthy cape]

Best would be some incremental API that lets you read it in chunks, but the API surface of FileReader seems fairly uninteresting.

[spring compass]

//input.value = "";
?
In that case the file will have to be re-selected by the user.

[lone ingot]

as in just drop the element and replace (free memory by dereferencing)

I havent looked but Im assuming the user doesnt actually look at their client.txt raw, so really it's just a process and throwaway problem where you make the assumption one upload is what's necessary and no more

issue with input elements in html is the upload based ones assume pin it to the element itself, meaning you can continue on execution of the page and at some point later, a random user event could also use the data that's pinned to it

it's built around the assumption that something can come back later and use it without needing to be in one code execution block / one contiguous logic loop

in your case Im assuming the reference is useless after you're done making the metrics, same for the FileReader (though I am less sure if this gets automatically cleaned up or not)

btw, when you upload something using html input you'll get back a type that has the text() method on it, which is probably pretty good for what you are doing

I do something similar in chromatic for uploading filter files

[spring compass]

For me, it’s just an interesting observation—I’m not worried about it. From what I’ve seen, I only use the file once, process the data, and store what I need in other objects. The file stays in the browser’s memory if the user has available RAM. If not, it gets automatically detached.

Reloading it multiple times (if the file changes) causes the browser to detach it from the input, requiring the user to select it again.

Do you think this is a problem? I’d prefer that users could refresh the data with a middle button rather than having to reselect the file every time.

I’m only reading the file, not uploading it.

[lone ingot]

I guess it depends if they are mutating the original or not

my assumption is that you read a file > parse to metrics > display metrics and that you no longer need the original once you have parsed the metrics out of it

because each following upload (if there is one) is an entirely new reference also right?

[spring compass]

no

[lone ingot]

what do you mean?

the references do get cleaned up btw, just had a tinker

[lone ingot]

reading this made me assume that the usage stabilised at 1.1gb afterward

each following upload is a new reference as far as memory goes btw, even if there's minor changes they aren't going to point to the same addresses (even if that would be more efficient). what you do with those references in your logic looks like it's incremental though

[worthy cape]

I wonder what benefits an arraybuffer has over text. I guess it's the actual bytes rather than some richer text encoding (UTF-16, 32-bit runes?)

I'm quite happy that I don't ever touch the web 😄

[lone ingot]

im assuming encoding, probably

[spring compass]

I'm not uploading the file. The reference is maintained by the browser as long as there is available RAM. Unless this is just playing with wording and you mean uploading to the input. I can manually force the file to be discarded, but why? That would be removing part of the site's functionality. And I know no one else uses it besides me.

I tested whether detaching the file matters, and it seems like it doesn’t. Simply setting value = "" doesn’t seem to have any effect.

[lone ingot]

read this @spring compass , it's a non issue

browser is already doing the hard work, i just assumed it wasnt bc of how I interpreted "1.1gb chrome memory" originally

[spring compass]

That's an interesting fact. Handling MBs of data consumes GBs of RAM—at least for me.

[lone ingot]

mine jumps 600mb~ uploading a 100mb file

browsers are amazing but also kind of just insanity in how some things are implemented

tbh I dont envy the engineers that make them, having to maintain seemingly infinite backward compatibility is probably really hard

[worthy cape]

I'm curious how sites like MEGA handles larger uploads, might have a reasonable cap without the app.

Back when I was working on Inya, my file manifests for each release were stored as large JSONL files.
Some were well above 100 megs.

Naive loading in Python or C++ could easily eat a gigabyte.

[lone ingot]

probably slicing it up i guess

im assuming you can upload whatever to the local browser, then local implementation just operates on ranges of bytes like s3 multi uploads

[worthy cape]

This, but half a million lines 😄

[lone ingot]

it's kinda amazing that mega and similar sites were working when they did really

ive been trying to think up a clever data structure for storing baseType > mod pool collection relations and it's actually been pretty painful

have something naive like this at the moment, which is just awful as there's so much duplication, but it works

anyone got any sage advice?

[worthy cape]

I designed Inya in a way so that it could be used by pure web clients, obtaining manifests from the API and then obtaining individual files by hash, relying in part on CloudFlare to cache the data along the way.

Never shipped it before I joined the company and now there's successors that are more immediately useful like lvlvllvlvllvlvl's site.

[spring compass]

True. But all these restrictions… So many times AI has told me that something can’t be done…

[worthy cape]

Great fun to actually build a system from end to end, so many things you learn along the way.

[lone ingot]

heh, I went the polar opposite, I just want everything done on initial load and <ideally> zero networking past that

that said it's like different spaces, chromatic is essentially akin to productivity apps where I am just trying to abstract away syntax and make things searchable

[spring compass]

I enjoy doing the first 40%, but then I have to force myself to continue.

[lone ingot]

haha if you keep at it you'll enjoy it more and more with time, programming and sys design is much like PoE in terms of the learning/fun/addiction curve

[somber mortar]

I think i enjoy it too much sometimes. 12+ hours a day coding is probably too much :)

[lone ingot]

I actually feel like I get more addicted to programming than games nowadays, it's funny

[worthy cape]

In the past, playing PoE1 made me want to work on my tools and working on the tools made me want to play PoE1.

[somber mortar]

yes. that is exactly me right now lol

well poe2 but same idea

[worthy cape]

12 challenges in Settlers 😄

[lone ingot]

wondering if I can treat a mod name like a pk or am I gonna get collisisons?

8x smaller if I do it that way, I dunno if it's a cooked way to do it tho

[worthy cape]

What about mods that have no name or more classful ones like veiled items?
Vorici's Veiled and of the Veil?

[lone ingot]

hmm, I actually need to check what they look like for poe1

that said its for the purposes of filtering using HasExplicitMod rule, I basically have a list of bases and I want the list of selectable things to be filtered down to only those that are valid if that makes sense

[worthy cape]

There seems to be quite a bit of overlap in names if you sort Mods by Name.

[lone ingot]

pretty much just need the name, the extra attributes on those JSON entries are just to enhance the searchability of them

[lone ingot]

ahhh pain

[worthy cape]

Ones like that:

[worthy cape]

(no idea what these apply to)

[lone ingot]

it makes sense, 1h and 2h variants

I guess I just use id then do something after to clean it up

[charred lake]

Anyone know what the .mfp / .tdt / .tgt / .gt files are in the bundles? Looks like they link back to the sprite sheets so I'm assuming some sort of position information

[lone ingot]

tdt/tgt are probably build file remnants?

[loud flint]

I think there's some smart chunking and streaming going on there. I wrote a program to ingest some larger files quite frequently

I don't remember exactly how I did it but it didn't consume a lot of mem

Woops was gonna respond to Zaos message a bit above here

These old man hands cant deal with technology

[swift beacon]

.tdt, .tgt, and .gt are terrain files

You can use them to generate 3D terrain tiles.

[worthy cape]

Or sink months and months into trying.

Ask me how I know .

[swift beacon]

I've gotten decent at it, but not up to reading the graphs to generate full areas.

[worthy cape]

For the Hideout Editor I stopped when it became apparent that I needed terrain generation outcomes to get the correct tiles, and ceased publishing any information as any interested people were predominantly cheaters.

[charred lake]

Gotcha thanks, know any tools out there to interact with them already? Otherwise down the rabbit hole I go!

[swift beacon]

None that I know of that are public.

[swift beacon]

Tile keys are hard.

[lone ingot]

any idea where the description text for stats is hidden away? e.g. (range)% increased Presence Area of Effect

[somber mortar]

in the description files :)

Metadata/StatDescriptions/stat_descriptions.csd

do let me know if you figure out how do decode the number stuff(#|-1) at the start of each line though. I've kinda figured it out but it still confuses me.

[lone ingot]

oh, it's this file

I remember this from affliction lol, I should have remembered

[spring compass]

How should I record the average arena level to keep it in line with poe terminology? I’d rather avoid "tier" since there's a difference between poe2 and 1.

[grave wren]

why not just adapt depending on the log you got, are there markers to differentiate?

[spring compass]

Generating level 20 area "HideoutOssuary" with seed 1

[grave wren]

poe2 seems to have a line identifying it in the log: 2024/12/06 07:51:58 1124265 a1e2d1f5 [INFO Client 15336] [HTTP2] User agent: PoE poe2_production/tags/4.1.0 Windows x64

ah poe1 too: [INFO Client 10124] [HTTP2] User agent: PoE release/tags/3.23.2b Windows x64

[worthy cape]

Might be a bit unclear how far back such nuggets can be found, but definitely distinguishable currently.

[spring compass]

wait, what?

[worthy cape]

I was gonna suggest that you could also look at port numbers for login and instance servers, but those aren't necessarily stable.

[grave wren]

yeah user agent seems to be the most simple, just check for any string then store if the log is poe1/2 and adjust display imo

its interesting that the first instance of the user agent seems to be 3.23.2b in my logs, theyre a bit™️ older

[spring compass]

You're suggesting that I may determine whether it's poe1 or 2 based on the version number. I was looking for where in the game files the pair of tier - instance level is located, and I was wondering why I even need it since the tiers are different in both games anyway.

[grave wren]

oh, then i misunderstood that

[spring compass]

Thanks, I wouldn't have figured it out without you. At least not right now 🙂

Last 3 panels

[mint sequoia]

u here ?

[daring moss]

Are you counting tracked voicelines from NPCs in general or once per map?

because you might hear a lot of them if you run past a few times before actually clearing their mechanic

[spring compass]

What you see on the panel represents the number of messages in a given time period. If you want to see content related to NPCs, you will find this information in the last graph.

All messages 1 092

Maps with her 87

[grave wren]

Just got home

[charred lake]

Had you looked at the mtp file format during this? Not looking to go down the terrain generation route, just trying to crop out the sprites since they're in a kinda non-standard grid pattern

[worthy cape]

I've got no idea what the format is for, I had enough other formats of more relevance to look at back then.

[lone ingot]

these csd files are giving me the shits

who came up with this standard for storing data 😄

[hazy fiber]

whats wrong with .csd?

[lone ingot]

all the irregularities of line breaks, spacing, etc

[hazy fiber]

arnt they basicaly .txt files?

[somber mortar]

I think i have attempted to write 3-4 different parsers for it. the current one kinda works but next time i write it i'm just tossing everything into pandas dataframes instead of trying to put it in dictionarys

[hazy fiber]

yeah well, I assume they compress fine, and ggg clearly have a parser for them that doesnt care about the irregularities

[lone ingot]

it's meant to be a lookup table Im assuming, using the stat ID

just some entries have multiple IDs, sometimes there's random line breaks

it looks organised at a glance but it definitiely isnt

it looks like multiple standards merged together lol

victory for now tho

[worthy cape]

Back when I did my own parsers for all the text-based formats, it was quite fun to see all the human mistakes that would still parse properly in the game but break my world.

[hazy fiber]

maybe you should implement somthing to fix all the human mistakes

[worthy cape]

Some of them seem really easy to parse if you have the definitions of what fields something can have, but not enough structure to parse freestanding.

[lone ingot]

it's the linebreaks and stuff that's driving me mad 😄

[worthy cape]

Components seemed to be one of those things that were context-sensitive with mini-DSLs inside for some things like animations.

[lone ingot]

also can't figure out what the numbers are for.. like sometimes the ID of the lookup is behind 1 but other times 2

[worthy cape]

Stat-descriptions seemed fairly structured to me last I looked at them, but I haven't ever written a parser for them.

[lone ingot]

is that just saying there's 2 IDs there for that entry?

[worthy cape]

PoE data files are the embodiment of Hyrum's Law but for data.

[somber mortar]

pretty sure that is just how many ids that description hold?

[lone ingot]

makes sense from game pov, it's handling semantics for +/- and none

from a parsing pov it's just confusing 😄

[worthy cape]

This one from PoE1 is a beauty:

[somber mortar]

description
    4 map_fishy_effect_0 map_fishy_effect_1 map_fishy_effect_2 map_fishy_effect_3

bruh how do you actually have an emote for everything

i need to go join servers for fun emotes

[lone ingot]

I stumbled on stuff for the winding pier the other day, pretty interesting

woo

[worn light]

Hi just making maybe useless alarm report but i think it should be reported...

Total virus finding fakes or either valid (hope that not) posititives in current new version of Path of Building PoE 1

[frank drift]

4/72...

[frank drift]

check github issues and make one there if there's not one