But do keep in mind that indexing requires a buttload of resources and a powerful pc
#tooldev-general
deft jolt
simple ravine
@deft jolt if you use Python, sure 😛
opaque hatch
by parsing you mean just scaning the json file and then deleting it after?
deft jolt
aye
simple ravine
I mean, for sniping you don't really have to index that much
opaque hatch
well I really wouldnt index anything would I?
deft jolt
I suppose not, just displaying some items from the data is easy enough
opaque hatch
the main problem I see is that lets say a new changeId comes every 1 second. and I run the wscript every 5 seconds that means that after a few minutes the item data would be way behind and not really usable
for sniping anywway
simple ravine
nah
not necessarily
under heavy load perhaps
if you consistently get 5MB per request, yes
opaque hatch
how often does the changeId normallly update?
simple ravine
I think that is based on data ingress on their end - meaning people doing updates
deft jolt
depends how many players are constantly updating their stashes
opaque hatch
yeh, I mean from your personal exp is it generally more than once every 2 second?
deft jolt
right now its about 1sec
simple ravine
I haven't done much indexing/sniping yet, but I will. I did some initial testing a while ago, and my statements are based on the conclusions I drew from that
I don't know much about python, but if you want a performant application, I believe that you would want to use dataframes and perhaps numpy
as Python natively doesn't support arrays for some reason
and it's not that efficient, unless you use c-binding libraries, such as the ones aformentioned.
deft jolt
Doesn't it?
simple ravine
One of the most fundamental data structures in any language is the array. Python doesn't have a native array data structure, but it has the list which is much more general and can be used as a multidimensional array quite easily.
But then again, I have limited experience with Python. But I've watched quite a bit of learning materials on python in machine learning contexts
obtuse citrus
array module if you actually need arrays
simple ravine
And it's quite often mentioned that performance is gained through libraries written in c
obtuse citrus
but it's only for primitive datatypes
simple ravine
Oh..
I'll stick with c# for my purposes then 😃
(not trying to start some x is better than y discussion)
deft jolt
I suppose you might be right. It was rough going from c to python
simple ravine
heh...
deft jolt
Hm, didn't know that
simple ravine
just wrote that 2 minutes ago
obtuse citrus
well I think you meant libraries that are writte in c or c++ and implement a python interface
but you can also call into c libraries with ctypes
ancient nymph
hello friends, can someone give me a sniping script for python
simple ravine
ah, like p/invoke, I suppose?
obtuse citrus
ancient nymph
is there a list of open source poe scripts apart from teh ones on the reddit side panel
opaque hatch
well I got some of it working now at least 😃
obtuse citrus
unfortunatly the same doesn't work for c++ because of how the compilers mess the code up AFAIK, but in any case you can still write c++ code and expose a python interface (Boost Python is probably better then the native python api though, it's less of a pain to use)
ancient nymph
@opaque hatch lemme see the code man
obtuse citrus
I might probably end up moving the ggpk and dat parsing to c++ eventually using above method :p
simple ravine
@obtuse citrus move it to c# and I'll collaborate 😛
opaque hatch
@ancient nymph keep in mind that this currently uses the most uptodate changeID which is incorrect still need to change that
ancient nymph
ok
obtuse citrus
thezensei you can use libggpk then
opaque hatch
ancient nymph
thx man
opaque hatch
its probably shit code
ancient nymph
i'll run it after i go wash dishes
opaque hatch
ok
simple ravine
@obtuse citrus I think we discussed this a while ago. Last time I looked, it seemed to be it wasn't actively worked on
obtuse citrus
yeah it isn't but the raw ggpk parsing/dat parsing still works just fine
simple ravine
I'm a big believer in unifying efforts 😉
Working on this at the moment:
[WIP]
It'll be a multi-purpose tool, but the current view is basically to simulate crafting cost
To help people get a deeper understanding on how much it'd cost to craft the item you're after
opaque hatch
thats really cool
deft jolt
If I were to buffer the stash api data through var buffer, would len(buffer) give me more or less an accurate representation of how many bytes were downloaded
(asking for a friend)
deft jolt
yes
opaque hatch
import _pickle as cPickle
dict = cPickle.dumps(JSONDATA)
print (str(round(((sys.getsizeof(dict))/1048576),2)) + "mb")
Just tryna get some statistics
opaque hatch
dict = cPickle.dumps(massJson)
print (sys.getsizeof(dict))
or for just the bytes
massJson = th json file you got
gotta import sys aswell sry
deft jolt
For me, buffer is the raw response in byteformat
opaque hatch
ah sorry I misunderstood 😦
deft jolt
I mean, if i were to get the amount of bytes there, would that be accurate?
After it finishes buffering it converts it to json and does the parsing
opaque hatch
well, the file you are getting is just a json anyway so it should be the same really?
id say it would be pretty fking close, however I base this on absolutely nothing
obtuse citrus
it might be better to check with wireshark or so
you'd download the headers for the protocol and it might be compressed so unpacked bytes are probably not what you're downloading
simple ravine
rule of thumb with gzip normal text is roughly 50% compression
json-esque type of text
ancient nymph
man @opaque hatch do you know how to use panda dataframes
ancient nymph
o ok, im going to run the poe json through it i think
deft jolt
@ancient nymph if you want a trade sniper I can let you test out mine
ancient nymph
gimme @deft jolt i'll try it man if its in python
deft jolt
the only thing is..
you'll only be able to see the front-end
since it outputs to discord
ancient nymph
why does it output to discord
deft jolt
for easy sharing
ancient nymph
i'll run it, gibe man
deft jolt
😃
ancient nymph
also does anyone know how pobdb.tw rips its data?
whats going on that in channel @deft jolt
deft jolt
which one
ancient nymph
merger
deft jolt
well that's the front-end of a trade sniper
outputs anything that could make a profit there
ancient nymph
man have you run a sniper through a league before
deft jolt
mid-legacy til present
ancient nymph
have you been tracking your gains, lemme see man
ancient nymph
o ok doesnt happen often then i gues
i will just parse the json for some insight then
ancient nymph
what price gap are you using
deft jolt
10c min, 10% min
ancient nymph
thats great returns!
ancient nymph
can i see the code @deft jolt
@deft jolt respond man, are you using a script to message people
opaque hatch
@ancient nymph chill bro hes probably busy, I am sure he will answer when he gets a chance
deft jolt
It's mostly a private project right now
deft jolt
Nice!
opaque hatch
I am thinking I need to get the integer from the note "buyout 5 chaos" so I think maybe using a split or something is best
maybe
deft jolt
But whenever someone adds an item to their jewellery tab, you get messages for every single item there
opaque hatch
ohyeh this is just a test
I wont actually be searhcing for RING
tbh this will probably be a pretty baseline one to snipe uniquely named things
at this stage anyway
ancient nymph
@opaque hatch lemme see yoru code man, how are you parsing it
opaque hatch
I already gave it too u
I am just picking out the pieces that I want and displaying it
opaque hatch
I am down for helping as I get a lot of help but you shoulda t least try some stuff out on your own, here is the code that prints the items.
for x in massJson['stashes']:
for y in x['items']:
if ('note' in y and 'typeLine' in y and y['typeLine'] != "" and y['note'] != ""):
for itemName in snipeItems:
if(itemName in y['typeLine']):
print (x['accountName'] + " - " +y['typeLine'] + " - " + y['note'])
thats all I have added
and I added an input thing at the top to write items into an array to search for the array is called snipeItems
when you whisper does it use the account name?
like @accountname
@sssiq2 Hi I would like to buy your Two-Stone Ring listed for ~b/o 10 chaos
@sssiq2 Hi I would like to buy your Ruby Ring listed for ~b/o 20 chaos
@sssiq2 Hi I would like to buy your Amethyst Ring listed for ~b/o 70 chaos
@lamarkhamar Hi I would like to buy your Velvet Gloves listed for ~b/o 1 chaos
@jidh Hi I would like to buy your Sorcerer Gloves listed for ~price 3 exa
got this working now ,obviously the mesage can use some work ie splitting the string "buyout 1 chaos" but feeling good 😃
deft jolt
I use regex for that
ancient nymph
man im watching your bot @deft jolt are you using hte avg currency price for the item
deft jolt
yea
obtuse citrus
FYI: Empty strings are False in python, so you don't need to do != ""
deft jolt
^
obtuse citrus
unless you specifically want to do that
ancient nymph
help me @opaque hatch your code doesnt work man, are you running it in python 3
opaque hatch
yes
ancient nymph
please man how do i search the json for maps
which header is it in
@opaque hatch help me man
opaque hatch
I just found this, might be better suited for you since you dont wanna do any work. https://github.com/licoffe/POE-sniper Seems about 600000% better than mine aswell
ancient nymph
o ok i got it working now @opaque hatch
i'll show you my script man
look @opaque hatch http://pastebin.ws/48ahmk
@deft jolt can you tell me how you setting up your serach??? getting prices from poe ninja? then search ing for differences???
ancient nymph
yea it work sman python 2.6
i cant keep scrapping constantly because of my net though
opaque hatch
is that your entire code?
opaque hatch
you dont need to know how to use that script
it is an entire program
anyone can use
just downloed the .exe
watch the youtube video
its is very user friendly
opaque hatch
I dont think your script works mate
ancient nymph
it does man
opaque hatch
how?
ancient nymph
you dont have hte poedb file
opaque hatch
where do you even call the api
ancient nymph
i have it saved
opaque hatch
oh ok I asked if that was ur entire script rofl
wouldnt your script just print this
{
"name": "vaal regalia",
"rarity": "rare",
"league": "Legacy",
"online": "x",
"mod_name": [
"(pseudo) (total) +#% to Cold Resistance",
"(pseudo) (total) # Life Regenerated per second",
"#% increased Energy Shield"],
"mod_min": [10, 1, 100],
"mod_max": ["", 10, ""],
"group_type": "And",
"group_min": "",
"group_max": "",
"group_count": 3}
yea
ancient nymph
how do i paste like that shaka
opaque hatch
``` type ```
ancient nymph
""" {'group_count': 3,
'group_max': '',
'group_min': '',
'group_type': 'And',
'league': 'Legacy',
'mod_max': ['', 10, ''],
'mod_min': [10, 1, 100],
'mod_name': ['(pseudo) (total) +#% to Cold Resistance',
'(pseudo) (total) # Life Regenerated per second',
'#% increased Energy Shield'],
'name': 'vaal regalia',
'online': 'x',
'rarity': 'rare'}
None """
ebon oasis
` not "
ancient nymph
its fine shaka it searches for the stuff man
ancient nymph
it displays this http://pastebin.ws/emhuy
opaque hatch
to me you are setting the json file = to Data
ancient nymph
yea shaka thats the inputs man
ancient nymph
what
opaque hatch
on lines 9 and 10 you are setting the variable data to the json file
then on line 12 you reassing the value of data to the search you want to make on poe.trade
ancient nymph
man you're right shaka, how do i fix this
opaque hatch
you need to probably learn the basics of python at a minimum or download someone elses released sniper
I mean you could try to fix it but I dont feel like you actually know what you are doing at all and are just copy and pasting
ancient nymph
help me shaka
opaque hatch
so you are going to need help at literally every step of making this thing at which point what is the point lol
how much coding ahve you done previous?
ancient nymph
just with scrappers and dataframes awhile ago
opaque hatch
ok well man I suggest you do some python tutorials
or any language really
you dont seem to have even the basics down.
ebon oasis
python shouldn't be hard to pick up for basic stuff
opaque hatch
If all you want is a sniper just download that other dudes
that guys is far more advanced than mine ever will be and has a nice UI
mine doesnt even function as a item sniper yet rofl
deft jolt
I wrote a trade sniper in the first place to learn python
About a month in it was fully functioning and had well over 1k lines of code
So yea, python is easy for beginners
opaque hatch
@ancient nymph there you go make it a project of your own, set out to make this script and learn python as you go. Copy and pasting a mix match of code wont get you no where.
ancient nymph
no man
opaque hatch
right now this is what you program does
sets data to the json file (not sure if that even works)
sets data (the same variable) to a different json file (search parameters) that was linked earlier in here to help use to build a poe.trade link, not to use with a sniper using poe stash api
Prints the current version of data which is the search parameter one
man i found a python sniper @opaque hatch its all over for you now man
simple ravine
lol
opaque hatch
nice mate, I am glad I met you. Now I can say I met the living embodiement of a script kiddy.
simple ravine
tell me how fast you process each stash
and i'll guarantee you i'll process it faster
opaque hatch
@simple ravine at what point does speed become a non issue? If I process mine in 1 second and you process yours in 0.1 seconds is that a huge deal?
ancient nymph
are you going to host it on a website @simple ravine
simple ravine
why you asking?
ancient nymph
asking because you show it man
simple ravine
what I show?
ancient nymph
screenshot of the gui
simple ravine
that's not a website
ancient nymph
no i mean will you host for people like poe trade
simple ravine
i will make the application available when I feel it's ready for the spotlight, yes
https://github.com/licoffe/POE-sniper <-- here's another one, made by poe-rates.com develoepr
deft jolt
@simple ravine how fast I parse each stash when sniping items?
simple ravine
@deft jolt nvm bud, was just trolling that dude a little
simple ravine
let's do this
parse time / total bytes uncompressed json payload
because each stash payload varies in size
deft jolt
the numbers are too long
simple ravine
what do you mean too long?
deft jolt
0.01642805154746071
0.021961489159992107
0.015124925887018015
0.023417014388624823
0.01654106075113917
0.010628965286715772
0.012272517809101099
0.011048627834643108
0.01570172734954203
0.0168871440219122
0.028826136231565363
0.01593885071849947
0.014627391083795396
0.026981915902594085
0.017758829818082425
0.022964476792553708
0.015917164812159188
0.08406559838444552
0.01942725609184321
0.037816674439945436
0.017825337198918243
0.010474836835333737
0.025713722564059942
divided by 1 000 000
simple ravine
oh wait the other way around of course
or wait
bytes / millisecond parsed
is what you want
sorry, my thoughts are a bit in disarray
opaque hatch
my script takes like 5-6 seconds to fully search the json file for 1 word rofl
actaully no i was wrong
its under a second I had a puase function in there from testing earlier rofl
obtuse citrus
lol
if you want to improve speed considering profiling with something like line_profiler
opaque hatch
I think I am good for now at under a second 😃
Dont wanna overload myself, it doenst even function yet really rofl
deft jolt
C:\Users\Admin\Desktop\buntu>"merger.py"
Bytes: 2149821 ; time: 0.03509 ; bytes/time: 61260.5582
Bytes: 407410 ; time: 0.00804 ; bytes/time: 50647.6598
Bytes: 304986 ; time: 0.00702 ; bytes/time: 43451.2228
Bytes: 608827 ; time: 0.00905 ; bytes/time: 67286.91
Bytes: 534014 ; time: 0.00802 ; bytes/time: 66550.3047
Bytes: 481211 ; time: 0.00903 ; bytes/time: 53319.206
Bytes: 26440 ; time: 0.001 ; bytes/time: 26360.2086
Bytes: 350568 ; time: 0.00501 ; bytes/time: 69931.9302
Bytes: 23268 ; time: 0.001 ; bytes/time: 23181.2507
Bytes: 14894 ; time: 0.001 ; bytes/time: 14831.4254
Bytes: 208938 ; time: 0.00398 ; bytes/time: 52482.3026
Bytes: 324950 ; time: 0.00805 ; bytes/time: 40355.8785
Bytes: 146305 ; time: 0.00303 ; bytes/time: 48212.4172
Bytes: 220214 ; time: 0.00403 ; bytes/time: 54592.1426
Bytes: 423315 ; time: 0.00802 ; bytes/time: 52781.349
Bytes: 108776 ; time: 0.00103 ; bytes/time: 105684.4132
Bytes: 42427 ; time: 0.002 ; bytes/time: 21167.0912
Bytes: 145352 ; time: 0.00203 ; bytes/time: 71530.0334
Bytes: 73854 ; time: 0.00098 ; bytes/time: 75295.6071
Bytes: 109555 ; time: 0.00301 ; bytes/time: 36431.2198
Bytes: 26864 ; time: 0.00103 ; bytes/time: 26106.5298
Bytes: 150399 ; time: 0.00202 ; bytes/time: 74512.0632
Bytes: 138883 ; time: 0.00201 ; bytes/time: 69256.6309
Bytes: 530495 ; time: 0.01005 ; bytes/time: 52791.5275
Bytes: 286304 ; time: 0.44949 ; bytes/time: 636.9529
Mean values:
Bytes: 313522.8; time: 0.0234008; bytes/time: 50346.273411999995
there you go
simple ravine
cool 😃
I'll write a similar one right now
time unit in your case is seconds, im assuming
simple ravine
0.4ms to parse 286kb of data
deft jolt
round(len(buffer) / (asd2 * 1000), 4)
asd2 being the time in sec
Mean values from 5 consecutive tests:
Bytes: 306079.92; time: 0.005603 ; bytes/time: 56135.769408
Bytes: 349645.473684; time: 0.006386 ; bytes/time: 57819.534053
Bytes: 269351.923077; time: 0.020457 ; bytes/time: 54858.549681
Bytes: 318088.185185; time: 0.005726 ; bytes/time: 57042.076456
Bytes: 216603.44; time: 0.004257 ; bytes/time: 54362.039796
elder hearth
Pleeeeeeease when using json with Python, use ujson package
Also use multithreading (one dor downloading and other thread for parse/printing)
For*
deft jolt
If you're referring to me I've already got it covered
elder hearth
Some people forget to use ujson and gzip when requesting
chrome topaz
or don't: https://mod16.org/hurfdurf/?p=266
elder hearth
Meh, too specific, for our purpose it works great
opaque hatch
What's the benefit of using ujson?
gilded herald
What issue are you having?
ancient nymph
print(j["foo"][2]) how do iget into the next layer in the array
gilded herald
i would use a nested for loop.
ancient nymph
o nvm its fine the guy at freenode responded
gilded herald
glad to hear it.
simple ravine
IRC
elder hearth
i know :p
2017
IRC
ancient nymph
irc hasnt died man
simple ravine
@deft jolt
71046089-74791343-69984020-81345149-75636183 1.554 ms 355kB 234,095 B/ms 9 stashes 367 items
71046094-74791345-69984021-81345151-75636183 2.897 ms 636kB 225,005 B/ms 10 stashes 600 items
71046094-74791345-69984022-81345158-75636184 1.418 ms 294kB 212,696 B/ms 9 stashes 285 items
71046098-74791347-69984026-81345159-75636184 3.191 ms 388kB 124,578 B/ms 11 stashes 352 items
71046099-74791348-69984026-81345163-75636185 1.304 ms 265kB 208,168 B/ms 7 stashes 267 items
71046099-74791348-69984026-81345166-75636185 0.369 ms 76kB 212,009 B/ms 3 stashes 93 items
71046099-74791349-69984028-81345168-75636185 1.144 ms 266kB 238,642 B/ms 5 stashes 327 items
71046101-74791353-69984029-81345168-75636189 1.519 ms 312kB 211,025 B/ms 11 stashes 344 items
lost wigeon
@simple ravine Average per item or total?
simple ravine
total, entire payload
lost wigeon
Pretty fast 😃
lost wigeon
Js?
simple ravine
c# .net core 2.0
so this is deserialized into statically/strongly typed objects, with correct typing etc
Js would be way slower, probably in the range of Python
lost wigeon
I should check how fast it is in my Js code
simple ravine
would be interesting to see indeed
it becomes somewhat slower around ~1 MB, because of heap allocations
going to see if I can optimize that
elder hearth
what are you measuring?
simple ravine
deserialization speed of json
deft jolt
Oh my, your script is a lot faster
elder hearth
in python
deserialization
is it when JSON becomes an Object?
simple ravine
suppose you have a class called Stash, and StashItem
which holds your stash data. once you've materialized those
where you can count the number of stashes and sum up the total items
otherwise, the deserialization you've done is quite meh in my opinion
deft jolt
What I did was measure how long it took me to convert a raw bytearray to json and parse stash data, finding favorable items
So idk
No classes involved
lost wigeon
So, if I didn't mistake it's about 2 times slower on my side
simple ravine
faster than python, indeed
@lost wigeon is that with the built-in JSON.parse(...);?
lost wigeon
Yes exactly
simple ravine
right - i don't think you'll find any faster, as it's native
lost wigeon
Yeah, my though exactly. Anyway, this is still pretty fast.
simple ravine
yeah, I got a nack for optimizations and speed
lost wigeon
The parsing + processing takes less time than the download on average.
simple ravine
since I worked at Pingdom
lost wigeon
I can see that 😉
simple ravine
for an instance, did you know that
array.forEach is slower than for in, and for in is slower than for, and for is faster in reverse sometimes.
(for javascript)
lost wigeon
Yeah, I did some benchmarks on this when I was coding my first indexer. Anyway, I'm not using for loops in Js. This is a recipe for disaster. I prefer to use aync.
lost wigeon
Yes async. Of course, but it's pretty straightforward to use.
JSON parsing benchmark with different libraries (from https://github.com/miloyip/nativejson-benchmark)
chrome topaz
if you are parsing the api, you will get bottlenecked by ggg, not by your json library
lost wigeon
@chrome topaz Yes, definitely
elder hearth
@simple ravine are you measuring the time that takes for your program to convert JSON to object, right?
like
time1 = time.time()
data = ujson.loads(request.text)
time2 = time.time()
?
simple ravine
yea
elder hearth
@simple ravine well, so my notebook is dogshit or this python thing is complete dogshit
simple ravine
to be fair, i'm on a core i7-7700K desktop cpu
elder hearth
Intel(R) Core(TM) i5-3337U CPU @ 1.80GHz
here
simple ravine
ok
elder hearth
simple ravine
can u add a bytes / ms?
elder hearth
kB / ms?
simple ravine
no, B/ms
elder hearth
ok
simple ravine
and then just send the script and I can compare on my cpu as well
i think it's easier than you installing .net sdk etc
obtuse citrus
if the json deserialization is really a problem in your books you can try out some 3rd party libs to see if they work better
simple ravine
it's just a fun excercise, @obtuse citrus
elder hearth
@simple ravine tested here with built-in json and it is about the same performance (for python 3.6 at least)
if you want to test, replace all ujson with json
simple ravine
C:\Users\Andre\Source>pip install ujson
Collecting ujson
Using cached ujson-1.35.tar.gz
Installing collected packages: ujson
Running setup.py install for ujson ... done
Successfully installed ujson-1.35
finally
@elder hearth
E:\Code>python pyt.py
Id: 71075527-74823593-70014466-81376728-75667240 1898 ms Stashes: 42 | Items: 2500 2405 kB (1297.60 B/ms)
Id: 71075538-74823607-70014472-81376736-75667243 1143 ms Stashes: 25 | Items: 1114 1071 kB (959.57 B/ms)
Id: 71075543-74823612-70014474-81376742-75667250 710 ms Stashes: 12 | Items: 574 512 kB (737.94 B/ms)
Id: 71075547-74823615-70014476-81376744-75667251 288 ms Stashes: 5 | Items: 227 211 kB (751.07 B/ms)
Id: 71075547-74823615-70014479-81376745-75667252 286 ms Stashes: 5 | Items: 269 250 kB (896.92 B/ms)
not sure what is wrong with your thing
ah I know what it is
request = requests.get(url, stream=False)
change it to False instead of True, otherwise you're measuring download time as well
E:\Code>python pyt.py
Id: 71075639-74823695-70014576-81376847-75667328 17 ms Stashes: 30 | Items: 1123 1025 kB (63459.23 B/ms)
Id: 71075644-74823701-70014584-81376854-75667332 11 ms Stashes: 22 | Items: 844 719 kB (66741.14 B/ms)
Id: 71075649-74823703-70014588-81376862-75667335 4 ms Stashes: 7 | Items: 288 258 kB (65756.17 B/ms)
Id: 71075653-74823704-70014589-81376863-75667335 6 ms Stashes: 8 | Items: 422 400 kB (68113.01 B/ms)
Id: 71075654-74823704-70014590-81376865-75667339 5 ms Stashes: 8 | Items: 305 328 kB (67099.14 B/ms)
Id: 71075655-74823705-70014590-81376867-75667343 2 ms Stashes: 4 | Items: 128 130 kB (66480.80 B/ms)
Id: 71075655-74823707-70014590-81376868-75667344 4 ms Stashes: 5 | Items: 223 245 kB (62531.79 B/ms)
try that and see what u get
elder hearth
now i'm getting 10-50ms
simple ravine
and your B/ms?
elder hearth
25-30k
simple ravine
ok
elder hearth
i still don't understand why that stream parameter is related to it 😦
that thing activates gzip
simple ravine
no
you'll deserialize a stream instead of fetching the entire thing before deserializing
elder hearth
request = requests.get(url, stream=True)
req_text = request.text
time1 = datetime.datetime.now()
data = ujson.loads(req_text)
time2 = datetime.datetime.now()
now it is back to normal, even with =True now
simple ravine
probably because under the hood, req_text = request.text will await and consume the stream
elder hearth
im having 50k now
simple ravine
good
let me paste that in and see what I get on my CPU
E:\Code\TalaMoana\Talamoana.Indexer>python E:\code\pyt.py
Id: 71077020-74825370-70016148-81378442-75668760 20 ms Stashes: 45 | Items: 2724 2750 kB (140401.00 B/ms)
Id: 71077029-74825382-70016154-81378453-75668767 13 ms Stashes: 29 | Items: 1354 1309 kB (102864.21 B/ms)
Id: 71077033-74825390-70016164-81378455-75668772 2 ms Stashes: 10 | Items: 330 341 kB (173966.08 B/ms)
Id: 71077035-74825396-70016165-81378456-75668773 4 ms Stashes: 11 | Items: 572 578 kB (147641.15 B/ms)
pretty good speed
elder hearth
nice
simple ravine
im taking ujson is a c-library
elder hearth
it is
this is why it need that vs shit
where do you write you C# code? VS Studio?
elder hearth
i'm trying to setup VS Code to Run without typing "dotnet run" on console
if i click run it tries to execute scriptcs
do you know what config do i need to set inside launch.json?
simple ravine
there should be a "automagic" thing
did you do dotnet new... from CLI?
you should get the VS 2017 Preview (15.3) Community Edition if you wanna try c# out
it's a bit easier if you're not very familiar
elder hearth
VS 2017 Preview (15.3)
what is it? VS Studio?
elder hearth
😩
not enough disk space
simple ravine
did you pick everything there was to pick in the installer?
elder hearth
i only clicked on core thing and it requested 5 gig, i have 2.6gb free (this shit notebook got 96 gb of total space)
elder hearth
i did it, what bothers me is having to type "dotnet run" everytime
debugger is working fine too
oh wait, i added it to Task.json
i just need to map a hotkey to it
simple ravine
F5
elder hearth
F5 starts the debugger, did ctrl+alt+n to run the code
elder hearth
for now it creates a DLL
simple ravine
you can add xml <RuntimeIdentifiers>win10-x64</RuntimeIdentifiers>
to the .csproj underneath xml <TargetFramework>netcoreapp2.0</TargetFramework>
then run dotnet restore
dotnet publish -c Release -f netcoreapp2.0 -r win10-x64
that is for "self contained apps"
which you can package up and send, and people won't have to install .net
otherwise skip the -f flag
elder hearth
err, when i did the dotnet new, it created a 1.1
i think i need to download 2.0
simple ravine
yeah, it's still preview, so it's not installed by default
once you've downloaded it, you can just change the .csproj file, and run dotnet restore again, and you're good to go.
elder hearth
now it is working fine
simple ravine
cool, let me know if you bump into anything else
elder hearth
now i need documentation/tutorials/videos etc 😐
simple ravine
what languages are you familiar with except python?
simple ravine
ok, that's good
java is quite similar to c#
let me think - i get the question now and then, but i'm having a hard time answering it, as i've been developing for the past 20 years, and not kept up with the how to get started things that much
@elder hearth I assume you'd want a primer on c# and .net
elder hearth
what i usually do is:
type that language on youtube and sort for most recent
this ASP.net thing is for web service, right?
simple ravine
yeah
elder hearth
people still use it or they got better alternative? (due to cost or whatever this licensing is)
simple ravine
mkdir test-web
cd test-web
dotnet new mvc --framework netcoreapp2.0
dotnet restore
code .
there are no licensing costs per se
you can run this on linux on nginx
dotnet run
voila, web app
elder hearth
ehhee
simple ravine
add xml <RuntimeIdentifiers>win10-x64;ubuntu.16.04-x64</RuntimeIdentifiers>
boom, cross platform
elder hearth
for web app or desktop app?
simple ravine
not for gui apps, but for both console apps and web apps
elder hearth
i see
simple ravine
well, you can actually create GUI apps
elder hearth
about gui, are u using that WPF thing?
simple ravine
with electron
yeah, i am using WPF
but for that, you dont use the cross platform .net core, but .net framework 4.7 which is windows only
but - you can create gui with .net core using electron
much like slack and discord is built on
atom etc
which is essentially a web-app embedded into a shell-app
simple ravine
note that he is using .NET Framework and not .NET Core ... but they are very similar
and you can most certainly start with that one
lost wigeon
@chrome topaz Are these valid entries?
simple ravine
heh
obtuse citrus
I think he just copied the stuff from prophecies and not just the enabled ones
chrome topaz
ask ggg
opaque hatch
rofl IRC
ancient nymph
help i dont see where currency/price is listed on the api http://pathofexile.gamepedia.com/Public_stash_tab_API
torpid mesa
same as any other item
torpid mesa
have you pulled down any of the stash tab api yet?
ancient nymph
yea
opaque hatch
@torpid mesa he has not. This was "his" code last night which is bacailly a direct copy and paste of a few others peoples code they shared in this chat. He then asked why it didnt work. http://pastebin.ws/48ahmk
torpid mesa
ah
ancient nymph
no man tell me the key
torpid mesa
read the reddit post and start looking at the results from pulling the api
opaque hatch
We told him last night, he needs to learn the basics of programming, do some tutorials or something. He said "no" and just kept asking us to fix his code which isnt even his
opaque hatch
he just wants a sniper, not to actually develop one. I wouldnt not waste your time, anyway thats your choice, im off
torpid mesa
i scrolled up and saw what he said, thanks for the warning
elder hearth
😂 rekt
rotund ember
how is Legacy league called in the stash tab api?
when i query it i only see Standard and Hardcore
rotund ember
nvm it's Legacy
simple ravine
lazy people deserve no help
elder hearth
@simple ravine which lib are you using to parse JSON? i was reading about JSON.net and some others and now i'm using rpgmaker's NetJSON
simple ravine
Jil
JSON.NET is the de-facto standard library, but Jil is faster
<PackageReference Include="Jil" Version="2.15.1" />
elder hearth
i will try that one 😃
also, how do you measure time? i'm currently doing it
DateTime startTime = DateTime.Now;
Stashes stashes = NetJSON.NetJSON.Deserialize<Stashes>(jsonString);
DateTime endTime = DateTime.Now;
int elapsedTime = (endTime - startTime).Milliseconds;
var sw = new Stopwatch.StartNew();
// ...
sw.Stop();
Console.WriteLine($"It took {Math.Round(sw.Elapsed.TotalMilliseconds, 3),-5}");```got it 😃
simple ravine
you need to add csharp using System.Diagnostics;
iirc
I'll share some code in a bit, on how I do it
I'm simplifying and making it a little more.. better
elder hearth
i can't startNew()
simple ravine
oops
remove new
you can either do csharp var sw = new Stopwatch(); sw.Start(); or csharp var sw = Stopwatch.StartNew(); for shorthand.
elder hearth
does it works? my vscode lint is mad about it
The type name 'StartNew' does not exist in the type 'Stopwatch'
should i ignore?
simple ravine
screenshot?
simple ravine
ah yes. Stopwatch.StartNew() is what is called a static factory method
you have instance members (fields, properties, methods, delegates, evets etc) - and then there are static ones
elder hearth
do you use webclient or httpclient to get the content?
simple ravine
HttpClient
simple ravine
nope, it handles it all
elder hearth
ok
simple ravine
you can use ReadAsStringAsync() instead of .ReadAsStreamAsync() if you want to download it fully before processing
and return a Task<string> instead
dire river
@simple ravine nice code, what project is the code code from?
simple ravine
my own, it's not public yet
dire river
alright, I wrote some .net core code myself to retrive the publish stash data some time ago, not released either 😉
elder hearth
@simple ravine is it possible to do a lambda with it?
int items = 0;
foreach (var item in stashes)
{
items += item.items.Count;
}
worked nicely 😃
dull arrow
@timid hemlock Hi, is burning damage meant to be scaling Poison in the 3.0 poB?
timid hemlock
Is it the result of fire->chaos conversion?
dull arrow
Yeah
timid hemlock
Well there you go 😉
dull arrow
Just making sure it wasnt a bug
timid hemlock
Actually, to tell you the truth I'm not 100% sure it's meant to happen; it's just something that happens as a result of my calculation system's design
But it does make logical sense
If Damage over Time and Fire Damage can both apply, then Fire Damage over Time (Burning Damage) should also work
hushed relic
I doubt that applies
Burning damage, is not just fire over time. It's very specifically the damage of "burn/ignite" dots
you can convert elements, but not Ailments
yet, I migh tbe wrong, 3.0 is all different
timid hemlock
It is exactly "Fire Damage over Time"
GGG have said as much before
Burning is just a shorthand
chilly wharf
I agree with Neversink, Benji and I are working on the same build and it doesn't make sense for the burn to scale so.. aggresively
hushed relic
But poison is specific too. You can't scale decay with "poison DOT"
timid hemlock
Of course not, because Poison Damage is not the same as Chaos Damage over Time
hazy gazelle
decay is an entirely different can of worms
hushed relic
Well, I'm pretty sure fire damage scales chaos -> poison damage, because of the conversion
timid hemlock
It definitely does
hazy gazelle
Definitely
hushed relic
but Burning, is a completely different topic
chilly wharf
exactly
hazy gazelle
agreed
chilly wharf
I stand by it being a glitch
hazy gazelle
poison after conversion has never been burning damage
timid hemlock
Let me find the GGG quote I mentioned earlier
chilly wharf
and it's getting treated as such
dull arrow
Unless its boosting all damage over time ailments
chilly wharf
But it shouldn't be
elder hearth
@simple ravine how do you measure the bytes?
chilly wharf
25% Fire -> 75% chaos the burn should only affect the damage of the 25% at most
Not the full 100%
As Neversink mentioned, poison =/= burn
dull arrow
Agreed
timid hemlock
But it doesn't do that in Path of Building
chilly wharf
Correct
timid hemlock
Only the portion that's converted from Fire is affected
chilly wharf
Unfortunately I'm at work and in meetings all day, but it doesn't seem proportinal. The converted part being affected. I mean about 90% of my damage is poison, and sitting at like 800k, I jump to about 950k or something from the burn
That's from memory, which I know means a lot less without the actual figures/pastebin. I can work on getting that later tonight
timid hemlock
Trust me; only the fire->chaos part is affected. It's impossible for anything else to benefit from it
chilly wharf
Alright, well if that's the case, and you're confident, then you're making me very happy for the scaling of this
It definitely introduces new paths
I appreciate the help
hazy gazelle
If burning damage is supposed to affect poison inflicted by converted fire damage
chilly wharf
I think Openarl mentioned there was a blue post from GGG about it
timid hemlock
There was a post that quite clearly stated that Burning Damage is semantically equivelant to Fire Damage over Time
But I'll be damned if I can find it
hushed relic
hmm
In that case, it might actually be true
So technically: it should be easy to test
chilly wharf
Yeah
hazy gazelle
Fire damage over time =/= Damage over time inflicted by (converted) fire damage, though
timid hemlock
I'd be completely screwed if it wasn't the case, BTW
hushed relic
"Increased Burning damage" GEM with 2x consuming dark should scale poison damage, right?
chilly wharf
Yes, that's what's happening Never
timid hemlock
The design of my damage calculation code doesn't allow me to include DoT and Fire Damage without also including Burning :/
chilly wharf
That's what raised this flag in the first place, because it didn't make sense to me when I was putting it through on PoB
hazy gazelle
Hope you're right then, Openarl, because that sounds like a pain to rework 😛
timid hemlock
Yeah 😉
hushed relic
I'll see if I can test it out next beta wipe, I wanted to try a burn character anyway
just need to get my hands on 2x consuming dark...
hazy gazelle
Just need one, really
chilly wharf
Yep
hazy gazelle
if all you want to do is test
chilly wharf
Or infenal mant
dull arrow
Well i've got a poison incinerator to test after wipe
chilly wharf
Yep, we're running similar builds
simple ravine
this is tool-dev chan btw 😛
dull arrow
Yeah, we're discussion whats happening with PoB displaying burning damage scaling Poison damage
if it was a bug or not
chilly wharf
official answer is 'hope not'
elder hearth
@simple ravine what happens when the JSON owner adds a new object? does our program just crashes or it just doesnt parse that member? just like poe.ninja added nextBetaChangeId
simple ravine
It will be ignored
simple ravine
using stream or string?
elder hearth
string
simple ravine
content.Length
elder hearth
content = string?
string contains escaped characters, don't need to go that deep and remove them, right? after all they need to deal with it anyway
simple ravine
well, assume this
var content = await cli.GetStringAsync(url); // or cli.GetStringAsync(url).Result for synchronous
nah
timid hemlock
Ok, so I've been doing a little tesing on Beta, and I'm 90% sure I'm screwed
Burning Damage doesn't seem to affect fire->poison
chilly wharf
I'm sorry man 😦
I figured it was a little odd. I got Holy Fire and my damage just scaled insanely. Way disproportional to the 25% fire it should have been
timid hemlock
Yeah, now I have to figure out how the heck I'm going to handle this
Bleaaaargghghgh grumble grumble
This really sucks
dull arrow
Sorry man
timid hemlock
I still think it aught to work, logically
hushed relic
Ouch 😦
I wanted to offer help, but just noticed you used LUA
Hoped for C++/C#/Java
timid hemlock
Pfft, Lua is the easiest language to learn ever
timid hemlock
😛
hushed relic
How does it even generate a UI?
timid hemlock
Hehe, the million dollar question
Well, the Lua code is running inside a 2D graphics engine written in C++
hushed relic
I've never used LUA outside of small scripts
timid hemlock
And the entire UI framework is contained in the Lua code
hushed relic
That sounds fancy
hushed relic
Why the decision? I mean it works out rather well for you, but just wondering
timid hemlock
Well, it's good that you mentioned Minecraft, since Path of Building is written in Lua for the same reason that Minecraft is written in Java
Basically, because it started out as a quick little hack written in the most convenient language in reach
Of course it grew to be much larger and more complex than I'd ever dreamed of 😛
If I was making it again from scratch I doubt I'd use Lua again
hushed relic
Well, if it wouldn't be so convinient to write it in, you might've never had the motivation to write it at all
so it worked out quite well
timid hemlock
Indeed
hushed relic
(brb in 15 min)
timid hemlock
Well, technically there is; it takes about 200ms for the Lua runtime to load and compile the ~120k lines of Lua code used by the program
simple ravine
if you don't have compile time, you have interpretation time instead, so yea
timid hemlock
Well, it does compile into the Lua VM bytecode
But that maps very closely to the language, so the compilation is super fast
simple ravine
i highly doubt that
timid hemlock
Doubt which?
simple ravine
that is maps very closely to the language
timid hemlock
I'm reasonably familiar with the bytecode, and it rather does
simple ravine
that would mean that the compiler does very few optimizations
timid hemlock
Well, I'm using the third-party JIT which does its own optimisations at run-time
But yeah, normally there isn't much optimisation
Even without the JIT the language is pretty fast
simple ravine
@elder hearth https://github.com/andreandersen/Talamoana
elder hearth
:'(
i need help, my code is running slower than that python
maybe related to download stuff too
simple ravine
which json library do you use?
elder hearth
static string indexing(string nextChangeID)
{
HttpClient client = new HttpClient();
String uri = "http://api.pathofexile.com/public-stash-tabs?id=" + nextChangeID;
HttpResponseMessage response = client.GetAsync(uri).Result;
response.EnsureSuccessStatusCode();
string jsonString = response.Content.ReadAsStringAsync().Result;
var sw2 = Stopwatch.StartNew();
Stashes stashes = JSON.Deserialize<Stashes>(jsonString);
sw2.Stop();
var items = stashes.stashes.Sum(x => x.items.Count);
System.Console.WriteLine(stashes.next_change_id + "\t" + sw2.ElapsedMilliseconds + " ms \t" + jsonString.Length + " chars");
System.Console.WriteLine(jsonString.Length/sw2.ElapsedMilliseconds + " chars/ms" + "\t" + stashes.stashes.Count + " stashes\t "+ items +" items");
return stashes.next_change_id;
}
don't mind the structure, because i'm almost translating line by line my python code
simple ravine
that seems correct
compile in release mode
dotnet build -c Release
and run the release binary
elder hearth
maybe they are measuring:
client.GetAsync(uri).Result.Content.ReadAsStringAsync().Result
if so, it includes download time
done
simple ravine
sw2 will measure the deserialization only
elder hearth
🤔
elder hearth
nice
simple ravine
you can look at the code in the repo i posted above, @elder hearth
it's in Talamoana.Indexer and Talamoana.PublishStashFetcher
bear in mind, i moved away from measuring deserialization timing and into the fastest way for the whole pipeline
elder hearth
@simple ravine i split my code so i could return the string from a function, now it is faster....
simple ravine
you want to dispose your HttpClient when you're done with it
you can either do that by wrap the code using the client in: csharp using (var cli = new HttpClient()) { ... }
or you can do cli.Dispose();
by using the using... statement, if an exception is thrown, it'll dispose it automatically for you.
elder hearth
:/ my c# code is around 25% slower then python on this machine
simple ravine
then you're doing something wrong heh
elder hearth
ohh i think i got it... @simple ravine
my python code gets too fast on the top of the river and keeps consuming easy stashes (low stash count = faster)
my c# code looks like it takes a while to count stashes and items
i will add a 10s delay for each one, so they may have about the same stash count available from the river
simple ravine
you can take my stash code in Talamoana.Domain.Core > Stash
elder hearth
i'm just using .Count
@simple ravine this thing helps a lot? [JilDirective("accountName")] for ex
simple ravine
nah
elder hearth
i think everything is fine now, added httpclient to that using and now i'm using a handler too for gzip/deflate
c# gets fast to the top of the river too
simple ravine
yeah
using a stream will normalize it a little
a bit slower on small payloads, but faster on larger ones
simple ravine
starting to get the hang of it, @elder hearth ?
elder hearth
@simple ravine i need to learn a lot, i get lost with classes, and i need to learn about Interface too
simple ravine
interfaces are powerful indeed
elder hearth
I'm getting used to lambda functions for now
simple ravine
I removed a bunch of interfaces in the simulation code though, because they slow down slightly (compiler optimization between virtcalls and stuff)
elder hearth
Also i wont give up on Python, but c# is useful if i need to do something with GUI + great performance in the future
simple ravine
for sure
i'm learning some python, because of its stronghold in machine learning things
and will probably have to learn some R too
but also F#
but I will always prefer C# for more complex things
elder hearth
I know nothing about machine learning, not even why i should learn something related to it
simple ravine
It's a method for a machine to determine correlation between data that, detect anomalies, predict future values and classify it
Based on different algorithms
hasty creek
Is there any tool that can read either the api or poe.trade and dynamically update the unqiue item tiers for lootfilters?
elder hearth
I thought @hushed relic was about to implement that on his site with help of poe.ninja or something
hushed relic
I sorta already do it, but there are a lot of small problems to think about
- Multibased uniques, Pricemanipulation, Meta development etc. I use the poe.ninja JSON data for "recommendation", but review and adjust all changes
I do intend to implement it to the site earlier or later too, indeed
Also I do recall that there was a script on reddit for it, but I recommend building your own small tool, since it's primitive JSON parsing
hasty creek
Alright, thanks for the reply!
exotic barn
@timid hemlock Hiya! With 105% reduced effect of curses on me (2x kikazaru, ascendant guardian node), are curses supposed to buff me?
the PoB link in case there's something else causing it that I've missed: https://pastebin.com/DwN4kwcS
timid hemlock
Ahh yeah that's probably not supposed to happen
exotic barn
ok, just wanted to make sure 🙂 thanks for all your work!
simple ravine
@hushed relic What I think you need is a price range for each unique, and given uniques that fluxuate a lot, use the higher range into the filtration process. Take Ventor's Gameble as an example.
elder hearth
@simple ravine i think he could use confidence level (poe.ninja provides it, not sure about his JSON file) for price manipulation too
simple ravine
the problem is the multi-currency system
actually, it's a multi-commodity-trading environment, and the currencies are commodities, and not sure what the best way is - poe.ninja seems to treat chaos as the 'main currency' which everything is exchanged into
elder hearth
still its a nice way to price check (poe.ninja gives equivalent chaos), also the tiers are not complex, i think neversink uses only T1 and T2 to filter unique items
T1 = 10c+
T2 > 1c and Multibased uniques
and uncategorized uniques (most shit)
hushed relic
@simple ravine this works fine from a developer perspective, but from a user perspective adding all multi base uniques and uniques, where a price drop is imminent, into the same collection will result in frustration.
simple ravine
Not sure if I follow, do you have a concrete example, @hushed relic ?
(I am in problem solving mode today) 😃
hushed relic
Sure, let's say you JSON parse ninja or personal stash data, you get that granite flasks are wit worth just few c
The problem is, that this behavior differs over the course of the league and over different leagues
simple ravine
Sure
hushed relic
The natural response would be increasing the range
simple ravine
loot filters in my opinion should be situational
and temporal
take your normal->uber-strict versions of your filter
and take it to a whole new level
now, this sounds really cumbersome and quite frankly idiotic from the first sound of it, i know
hushed relic
That works for my personal version, not for one, thousands of people don't update too often.
simple ravine
that's why I think what could be a good solution for people who care a bit more about the nuances of the drops, to have a utility application that updates this for you based on a set of rules and live data
and based on the situation you're in at the moment (i.e. standard, temp league, race, beta)
the live data would solve the progress of a league pretty much automatically, as price adjust themselves
the market is liquid enough for that
hushed relic
The problem is the live data
I've learned not to trust it too much, there's manipulation, hype waves, high variance items and things the filter simply can't identify
simple ravine
Well, I have a theory on how to deal with price fixers
but it requires me to complete my indexer first
hushed relic
I might work on my own, but I see little practical use, being beyond a few gimmicks, flipping never interested me too much
Grr I hate typing on mobile
simple ravine
I don't care too much about flipping either
It'd be a reasonable method to gain some initial currency in a league, but other than that, it's probably boring as hell.
What I am more interested in, is solving a problem I see people complain about a lot, and that is the price fixers who set prices low, and when you go to poe.trade and search for "Unique item X", you get pages upon pages of the same people "selling" an item without responding.
That is a problem that a large portion of the community would appreciate being fixed.
hushed relic
How do you intend to do so, create a chrome plugin, that preprocs data and clusters it by price groups and evaluate their lifespan/size over time?
simple ravine
No, first off I will start index the publish stash APU myself
But it will most likely be required to store the data in 2 or 3 different versions.
To detect price fixers, it will be required to use a graph database, but also an OLAP datastore to hold some statistics that are pre-aggregated in a temporal fashion.
And a document store for easy retrieval of the stashes
My current assumption is this:
** Please let me know if you disagree or want to add something to this **
There are different price fixers out there, I would probably categorize them into two different categories. Currency fixers, and item fixers.
hushed relic
Fancy, temporal DBs are exotic today.
simple ravine
well, sure - you can use datomic or druid, or whatever - but you can use SQL as well, if you pre-aggregate
the old fashioned BI way, ya know
OLTP vs OLAP
Now, these price fixers - I would believe they have 1+ account for items that they will not sell, and 1+ account for items they will sell (for a higher price)
hushed relic
Mmm, had a lot of fun with evaluating those when i was working ina it-linguistics lab
simple ravine
But I think it varies here too
In one of my companies we use Druid in production right now, ad-tech
works well when you've configured it well.. it choked in the beginning when we hadn't figured the optimal settings - but that is the case for most distributed databases
hushed relic
Indeed, but let's return to the fixers
simple ravine
yes
elder hearth
i think for lootfilters poe.ninja has enough data, we don't need our filters to do the instant market price check
elder hearth
price fixers only bothers poe.ninja when that item is not so much available on poe.trade
simple ravine
I've seen the impact of price fixers being quite... annoying
elder hearth
poe.ninja dev said they need too many price fixers (not chars or multiple same items, different accounts) to change something
hushed relic
I believe, semi automated, some are able to sell and adjust prices across several accounts, with hundreds of items, while sniping nd doing that for several different items
simple ravine
I think there are some people that are more dedicated to PoE that are price fixers
and dedicate their whole awake time with it - there are a bunch
simple ravine
I'm not gonne dive into what I think about them or how they live their lives, that's besides the point.
hushed relic
Guilds
simple ravine
Right, and many of these super dedicated people are in guilds
and operate in a team effort
simple ravine
item pricing in general, shop indexing (poe.trade / ninja) and so on as well
@hushed relic If you want, we can discuss this when you have access to a PC - I'll be online the entire night
elder hearth
@simple ravine i know this is not /r/learncsharp but, i'm doing some exercises on c# for learning and i need to return even numbers only from a fibonacci function with max value of 4,000,000
i already did it, but now i want to play around and i would like to try something like:
fibonacci(4000000).EvenOnly()
what is that "EvenOnly"? a class? how do i attach that to an IList for example?
simple ravine
list.Where(c => c % 2 == 0)
elder hearth
yeah i did, i want to attach that to an IList class and have a method that executes list.Where(c => c % 2 == 0)
simple ravine
ah, extension methods
elder hearth
@simple ravine feel free to send a link or name of something to google, i'm not a lazy mofo 😛
don't need to take your time to fully answer something we have on internet
simple ravine
public static class ListExtensions
{
public static IEnumerable<int> IsEven(this IEnumerable<int> enumerable)
{
return enumerable.Where(c => c % 2 == 0);
}
}
this is the short answer, giving you the fish rather then how to catch it
I'll find a link in a moment
notice the this keyword in the argument
if you now have a IEnumerable<int>, List<int> etc you can do list.IsEven()
probably terrible naming, but you get the idea
elder hearth
nice 😄
simple ravine
Please note: If you want RAW performance, LINQ will not optimzie your code in some cases.
Like in my simulation example, I refactored some LINQ code to for-loops and gained a lot of performance
elder hearth
i read about for / linq / linq+lambda performance
simple ravine
but in vast majority of cases, this is not a problem
elder hearth
they are more readable (linq/lambda), and for most case you trade readability for performance "loss"
simple ravine
in some cases you trade the performance
where iterators are fast
If you get to see the true power of c#, I think you'll have more fun using it - and LINQ is one of the true powers
There is a library called MoreLINQ that implements a bunch of auxiliary methods for IEnumerable<T> (all the lists, arrays etc):
https://github.com/morelinq/MoreLINQ
which you can add as a nuget (in your project reference)
brb
simple ravine
back
elder hearth
@simple ravine got that working, but general guidelines encoraged me not to use it :(
well, at least is something i learned
simple ravine
not to use what?
elder hearth
extension methods
simple ravine
where did you read that?
elder hearth
General Guidelines
In general, we recommend that you implement extension methods sparingly and only when you have to. Whenever possible, client code that must extend an existing type should do so by creating a new type derived from the existing type. For more information, see Inheritance.
simple ravine
Right. Generic collections are one of the examples where you want to use extension methods
Like IEnumerable<T>
elder hearth
Linq abuses it
🤔
simple ravine
But yes, creating a collection type that inherits IEnumerable<T> can be beneficial, because you can enforce invariants in the type (Domain Driven Design)
It's not abuse, it's by design
You should not compare design choices made in a framework with your application code
elder hearth
nice
simple ravine
familiar with domain driven design?
elder hearth
@simple ravine my debug screen is filled with some shit i don't care, how can i suppres it?
simple ravine
good question - i don't know actually
elder hearth
"logging": {
"moduleLoad": false
}
add it to launch.json, at least those Loaded are gone
simple ravine
ah
in visual studio, i have this:
handy if you want to toggle something depending on situation
elder hearth
@simple ravine i just read about DDD, and it looks nice.
tried to search for some design patterns that people use in c# (Gang of Four and etc) and none mentioned it, is it unrelated or it has some other name/scope?
simple ravine
'Domain Driven Design' is a huge thing, and not something that is confined to c# or .net
java, php, python etc all have it
elder hearth
i see
simple ravine
on top of that, you should also learn the SOLID principles.
https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)
simple ravine
guys
i am trying to write a "smart" reverse translation procedure
anyone written one that is quite confident?
elder hearth
?????
simple ravine
input:
--------
+24 to Intelligence
157% increased Energy Shield
+46% to Lightning Resistance
31% increased Stun and Block Recovery```
output: list of stats/mods (stat-ids, mod-ids) + numeric valuesOh i see, ItemInfo does that or tries when it is possible
elder hearth
:joy:
simple ravine
there are a couple of "problems":
- stats are merged
- stats can have multiple lines, and each stat can also have multiple lines
so it's meh
nested
elder hearth
Multiple lines = hybrid?
simple ravine
what do you mean with hybrid, you mean composite?
elder hearth
Yes
simple ravine
31% increased Stun and Block Recovery
this is a composite
= stats merged
135% increased Recovery Speed
50% of Recovery applied Instantly
Immunity to Bleeding during Flask effect
Removes Bleeding on use```this is stats with multiple lines
that is 2 mods
bubbling eternal life flask of staunching
so i want to be able to understand that this is a "of Staunching" and "bubbling"
elder hearth
How that stats_translation.txt file deals with multiple lines? I dont remember
simple ravine
the repoe json one that i use looks like this
elder hearth
Or your mod has stats1, 2, 3
simple ravine
{
"English": [
{
"condition": [
{
"min": 1
}
],
"format": [
"ignore"
],
"index_handlers": [
[]
],
"string": "Immunity to Bleeding during Flask effect\nRemoves Bleeding on use"
}
],
"ids": [
"local_flask_bleeding_immunity_while_healing"
]
}```Can you check that energy shield hybrid? (Seraphim's)
simple ravine
well, if we go from mod -> stat(s) -> translation
(sorry for the large paste)
"LocalIncreasedArmourAndEnergyShieldAndStunRecovery6": {
"adds_tags": [],
"domain": "item",
"generation_type": "prefix",
"generation_weights": [],
"grants_buff": {},
"grants_effect": {},
"group": "DefencesPercentAndStunRecovery",
"is_essence_only": false,
"name": "Seraphim's",
"required_level": 78,
"spawn_tags": [
{
"id": "str_int_armour",
"value": 1000
},
{
"id": "default",
"value": 0
}
],
"stats": [
{
"id": "local_armour_and_energy_shield_+%",
"max": 56,
"min": 51
},
{
"id": "base_stun_recovery_+%",
"max": 17,
"min": 16
}
]
}
we can see that this mod has two stats
so I think it needs to happen in multiple steps
string -> stat(s) -> mod(s)
elder hearth
Yeah, i dont think there's other way
Even ItemInfo doesn't show tier when they have multiple hybrid combinations
Like your regalia
ES + ES/Stun + Stun
simple ravine
yeah, that is a prticularily difficult combination
I think for indexing purposes, there is no reason go try to detemine the actual mod itself
the mods are just groups of stats anyway
and what matters are the stats (what people call mods)
elder hearth
Yes
What people may like is if it has an open prefix/suffix (even that you can't be 100% sure most of the time)
If you get some hybrid thing
Or prefix/suffix mod like rarity on rings
simple ravine
{
"English": [
{
"condition": [
{
"min": 1
}
],
"format": [
"#"
],
"index_handlers": [
[]
],
"string": "{0} increased Fish School size"
}
],
"hidden": true,
"ids": [
"map_tempest_fish_school_size_+%"
]
}
favorite stat so far.
elder hearth
I still don't know anything about those "fishing secrets"
lethal flume
It's a well kept secret.
simple ravine
Let's take chris on an exotic fishing expedition trip, and get him drunk enough to tell us
elder hearth
I actually don't mind, even if each encounter is equal to a potential Mirror, it's so rare that it doesnt do anything to economy
simple ravine
😃
elder hearth
@simple ravine just used .Aggregate (Linq) here, looks nice
keen pewter
Does anyone here know how to write AHK scripts? I have something non-poe related that I want to know is feasible in AHK or not
I'm trying to find a way to make a tool that when you press a button on a xbox360 controller a specific amount of time later it plays some kind of noise trigger
hasty creek
Sounds possible considering the AHK scripts I've used in the past
simple ravine
AHK can take c# scripts as well, so yeah I don't see why not.
torpid mesa
how does it handle hybrid %+something and regular% mods since those are combined on the item display?
simple ravine
In this step, it will only take lines and translate them into "stats.dat"-values
torpid mesa
ah
simple ravine
In next step, I will take the actual item itself, and run the deduction methods
I felt I had to decompose this into manageable pieces of logic
torpid mesa
seems reasonable
simple ravine
There were some "gotchas" in the logic around the translations themselves though
torpid mesa
oh yeah, like what?'
simple ravine
well a stat can be composed of multiple lines, and then have something like "Something {1} has some {0}"
instead of "Something {0} has some {1}"
so I ended up building dynamic regex strings to match with with named groups etc
seems to be working
torpid mesa
huhh interesting
simple ravine
so for eacn translation string, you take the format, and add them into it like so:
^(?<val0>\-{0,1}\d+)%\ reduced\ Amount\ Recovered$
Because you also have to take into consideration that there are multiple instances of each stat value like "Something {0} something {0} something {1}"
torpid mesa
would that be something like adds blah to blah phys damage? on poedb the line is worded strange Adds # to Chaos Damage, Adds # to Chaos Damage
i havnt looked much at how they actually list the stats in the gamef iles
i copied chaos damage but phys is the same
simple ravine
in the actual file it's like this
# # "Adds %1% to %2% Lightning Damage to Attacks with Wands"
however, it's mangled into a json file that is easier to interpret
{
"English": [
{
"condition": [
{},
{}
],
"format": [
"#",
"#"
],
"index_handlers": [
[],
[]
],
"string": "Adds {0} to {1} Physical Damage"
}
],
"ids": [
"global_minimum_added_physical_damage",
"global_maximum_added_physical_damage"
]
}
oic interesting
elder hearth
@simple ravine interesting :) how is it going so far?
simple ravine
right now, I am parsing the text stuff
just about to start writing the deduce stuff
elder hearth
Nice, about multiple possibilities, how are you going to show that for the user?
elder hearth
Are you focused on tier or values? Or both?
simple ravine
I think both, because I don't want to be too opinionated towards the user
elder hearth
Ahk shows value range and ignores tier IIRC (it shows ** on tier)
simple ravine
take these modifiers on a Vaal Regalia
+70 to maximum Mana
67% increased Energy Shield
+40% to Cold Resistance
38% increased Stun and Block Recovery
ilvl84
This could either be 5 or 6 mods.
elder hearth
Wait a sec, not on computer but i think its 6 mods only
simple ravine
no, this is a 5-mod regalia (checked with bench)
oh wait
this might be another one
yeah, goes to 56%
anyway, there are a bunch of scenarios where it can either be N or N+1 mods
no wait, it has to be 5
because I then rolled ```
+44 to Intelligence
+70 to maximum Mana
67% increased Energy Shield
+40% to Cold Resistance
38% increased Stun and Block Recovery
I see 6
simple ravine
67% increased Energy Shield
+40% to Cold Resistance
38% increased Stun and Block Recovery``` = 5Oh i was counting mod lines before not individual mods
1 + 2 + 1 + 2 (this one already had one counted earlier)
elder hearth
I think the only scenario that you may have 5 or 6 mods is when:
Stun and ES matching hybrid roll and <= 2 real suffix