#tooldev-general
1 messages ยท Page 53 of 1
instead nullobjects are just implementing the interface of w/e you're working with
but provide no functionality or treat nulls inside
This allows you to centralize nullhandling and reduce boilerplate
that sounds like default of struct
u see the new things they're cooking up in next version of c# btw?
last time I checked it out it was ~4 month ago
async streams, heuristic null predictions
is there a new interview/preview?
Oh yeah, default interface implementions
this makes me drool
(checking out right now)
Personally I'm mostly excited for AsyncStreams
oh yes
IAsyncEnumerable
I'm pretty excited about Span<T> in .net core 2.0 too, I just haven't had an excuse to play with it yet
What does it do?
I guess, paste a little more of your thing
do this if (stash && stash.items) { ... }
@hushed relic Span<T> is like ArraySegment, but faster
same three errors
if you want help, im gonna need to see your code
sure sec
@simple ravine that's pretty exciting. Seems to be great for pipeline, sensor and all kind of continuous calculations.
prolly good for private indexers ๐
(I've had the urge to write my own indexer several times, but I'd prefer to avoid economy manipulation claims o.O)
hehe
I've had the urge for quite some time too
I started writing a bunch of underlying library code for mods, stats etc
and went down a rabbit hole pasing plain text (ctrl+c ingame) for stats, and got stuck with estimating composite mods
@hushed relic clearly youre manipulating the economy by hiding all the mirrors in your filter ๐ค
False news. I have a filter that hides everything BUT mirrors (and sorcerer boots)
@simple ravine I'd actually love to get my hands on that
I didn't update the trollfilter
i guess it just makes it better
kek
I've been thinking about making a very lightweight solution, that basically works based on ctrl+c and tries to learn from my own selling/equipping/stash only
at the very least, I can use it to make a very small browser plugin to show resulting resistance deficits, when browsing poe.trade for rings
however, it's mostly for giggles, I get bored of trading so fast, I'm considering going SSF next league
i mean if you have access to your equipped items you could have a pluggin that just shows what your resists will be when you equip a certain item
yeah, I'd love to put it in your hands, but I want to clean it up
ok this is stupid, i can do a stash.items.forEach , in the catch, after the getting the error that i cannot do stash.items.forEach
ill feel really stupid when i catch what i did wrong
I'm just gonna go rent a car, I'll be back in a few @idle imp .. I'll check ur code in a few
k, ill keep trying stuff
do people actually list items in SSF leagues and then search for them?
i have made ssf tabs public before but i had no reason to search for them
id rather use acquisition to find items
otoh poe.trades search is better
maybe people are doing that
i know its annoying to try to find items with mods that are not listed in acq
why isn't the mod list in aquisition generated on the fly?
as items are scanned just pop in mods that you haven't seen
@chrome topaz can you allow limited comments on items? it would be nice to be able to say "obo"/"or best offer"
i know why you removed them but if you limit the usable phrases it should be fine.
@mortal bone pull requests are welcome ๐
@delicate ore isn't that what ~b/o means?
@chrome topaz I would have to get the QT enviroment installed lol
was gonna ask if there are any unique identified items that dont have explicit mods
and yes there is
tabula
cool so that does seem to solve my mysterious undefined issue
fell asleep with napping the kid lol
did you check out licoffe's?
did u figure the issue out?
yes zen, i was checking exlicit mods for all items
including the ocasional tabula
which threw an undefined since tabula doesnt have explicits
and no i didnt nijiko
didnt see you mention it tbh
will look at it
why is there an __awaiter
cant really answer questions about the .js files, i write in typescript
you checking the transpiled typescript->javascript
Oh, I was reading the transpiled derp
I was like this looks horrifying
that's a shim, probably with a generator fallback?
@hushed relic I'm refactoring my code, I'll show you the things when I'm at a stage where it's easier to digest
so many solid violations
@simple ravine sure, but let's move it to ~next week
races+currently trying to get filterblade ready
stupid leveling section >.<
yeah, i i'll fiddle with this during the weekend, and we can find a time next week
oh shit.. got a flight to LA on thursday next week
so monday-tuesday works, otherwise the week after
@hushed relic what do you do for a living
@gritty olive I'm an multimedia engineer, though currently I'm employed as a .NET dev and slowly slipping into more architecture
Oh cool, what kind of media
Image Processing, Datamining, AI, Robotics are my fields of focus
.NET framework AI? ๐ค
@chrome topaz b/o means 'buyout', obo means 'or best offer'.
traditionally speaking for mmos (always seen it used this way in poe too)
i'd like to be able to set a price like "~b/o 1 exa obo"
even better if you could display obo as "or best offer" on poe.trade
i'd recommend allowing other kinds of limited notes too but i'm not sure which phrases to recommend
well, ~b/o means buybout, as you said, but that indicates that the price isn't fixed, which ~price indicates.
which in my perspective, renders "obo" unnecessary?
"buyout" => "any offers considered, but at this specific price you are guaranteed to buy it out"
yeah. obo unnecessary, as you said
It's a matter of educating the players to pick the right prefix for their item. I know many people get pissed when you offer them a price under what they've set, even when they used "~b/o"
well, to be fair, ggg sets the default pricing note to be buyout
likely not all players recognize the distinction
for sure, you are right. not blaming the players... ok maybe a little
actually, there is price vs exact price
i really wish they'd let us merge 4x premium tabs into quad tabs
ze time has come for improved stash tab processing parallelization
Maybe GGG should loosen up the rate limiting.
shrug. i don't mind rate limiting as long as the rate is sufficient to get all the data
It becomes a bit tricky to monitor stashtabs when you have 110 tabs ๐
indeed. i took a different approach
i know i want to drink the firehose eventually, so i just take events that i know are my own stashes off the firehose
so i only have to process my own stashes when i actually have stash events
silly, i admit. but i also don't need to f with session ids and other such
it's a valid point
and usually i have at most 3 concurrent stash events
what is unfortunate is that you're going to consume unnecessary large amounts of bandwidth, from your perspective, perhaps not a big deal, but for GGG - it all adds up
that is true
if ggg offered a user-scoped firehose, that would be ideal. but that will likely never be a priority as long as the other api exists
perhaps they could add E-Tag + If-Newer-Than, and have a different rate-limiting if you're using those headers
seems fair
If I were to guess, taking into consideration the language they're using - APIs are most likely not the absolute bottom of their priority.
with language, I mean the way they talk/communicate in the context
worth pointing out, actually, i turn off my engine when i'm not playing & fast forward it during a gaming session
there's a killswitch if it doesn't detect new stash updates from me in something like 20 minutes
catching up is like 5-10 requests (depends on how long since last turned on) using a binary search on the change_id
probably unnecessary since poe.ninja is publishing that now
lol
?
doing binary search
@chrome topaz can you display the current league of the character?
it's annoying messaging people who are racing
can we get them to expose it? it's kind of a big deal
they are taking suggestions for the new trade site and api, right?
Did anyone figure out why the river id format is the way it is? 5 fragments of rather close numbers... Hm.
the stashes are sharded so these are change IDs from shards
worth noting that the change ids represent stashes that changed during the time window, but the actual stash content you get is what is in the stash right now
so if you're looking at historical change ids, it's not actually historical
ie, if a stash was public at the time of the change, and it has since been made private, you'll get an empty array of items
Right. That's why you have to go through a huge bunch of useless stuff and an odd section of completely empty stashes ๐
for example
which is why i implemented the aforementioned binary search << >>
can skip you ahead right to the present
Or you could just use my value, as you mentioned ๐
ah, you're poe.ninja?
Yeah.
I actually just added the latest change id back in v1 because I was thinking of stats I could add... It turned out to be a lot more popular than expected.
@tropic shuttle but binary search is a clever idea :-D
you have to apply the binary search to each of the 5 shards in parallel
so it's a little more complex than your usual BS
& you only have to get to within ~200-500 of the "actual value" - since it's a moving target, kill the search and the poll will catch up quickly
better than polling everything in between
Favorite person of the day is you!
๐
continues to describe how LB / NAT work to enterprise company
I wrote a 5 page PDF on how to do LB / NAT configuration and DNS Record setup and I still get asked every question the PDF answers
๐
"That's described on Page 3" "Ohhhhh" "Did you read it?" "Yes"
chuckles
the docs never get read
BGP โค
but they're useful to point to
I'd take the hint that the documentation was not understandable for them then.
If you're selling to non-technical people, yet your presentation is a bunch of tech talk, then you're going to have a difficult time.
i've encountered technical people who refuse to read the docs
on the flip, same people regularly make poor technical decisions
@simple ravine they read it with me and understood it
They just never read the documentation
Ohhh, right. They're from USA?
Yep
There you have it.
Basically it is "we want you to do it for me"
We are like "lol we don't have time for that and you don't pay for that"
Yeah. Love your country. And dislike it almost equally at the same time.
it's not just the USA
No, but in prticular.
We have an AU company that is the same way so I don't know
giggles
i think in general, some kinds of people will just try to avoid work they consider "dull"
The way I think about it is... if they could do it themselves would they be buying an external product
Is it hard to find out
I guess not.
You said you were on the spectrum so I figured you'd have done it already xD
๐ just like people who don't read docs
Haha
Or he's just not intriguing enough
honestly, i tend not to read docs until i have a problem that i know the docs solve
i don't read a doc just because
sometimes. sometimes that is true
If they aren't then it indicates some red flags
yep
@gritty olive You said you had two companies; I see you have stated, but still you have an employment.
After 6+ years.
Has it been 6 years?
According to you
Is your LinkedIn profile correct?
I'm not gonna post your personal details in here, that wouldn't be nice. But yeah,
Founder & CEO...
<company>
Yes, I have multiple employments
Oh, I see two entries, but they lead to the same company
Again, my profile hasn't been updated in years. And I don't tell everything in there either
Changes in Job Titles do that on Linked In which I thought sucked ๐
I've been at that one since it was 3 people
There is another company missing from my Linked in ๐ค but I don't think I'll at that one yet until the other business partner and I discuss release stuff
How come you're not C/D-Level at this point? Looks like the company has quite a few employees now
I have delegated out all my tasks and I don't want to do that stuff
Technically my title is "Architect" but we are rolling out stuff slowly so we don't jar new employees / existing employees
I didn't want to be a "Solution Architect" though cause I like being at home ๐
Plus I have other responsibilities at my own companies ๐
Hopefully that answers it for you
How many are you employing yourself?
I have 4 on the new company so far, and on Nexua it is me and a friend that we get paid pretty well doing consulting
Parsify is me and has like 30 customers that is a nice little side income
Me and my friends invest in tech startups, in case you have an interesting and unique product
Curiously enough, partner and I don't take investments
We like bootstrapping and retaining equity
Well, I respect that. Even if it can be both a weakness and a strength.
Like how they say.. it's a fine line between madness and genius, right?
There are some cases where I can see us, later down the road tapping into a VC not for monetary reasons
For networking reasons
Yeah, we help with customer acquisition and marketing, mainly. In some aspects, technical and product development insights
We do connect people together that we see could benefit mutually, but as you hinted, that's down the road.
@gritty olive Did you start with OOP when you were 8 years old?
wow why are you stalking his linkedin ๐ฎ
do me now
nah
No not oop
Yes but I didnt start with it
I guess people bend the truth all the time.
?
If you're ~26-28 years old, and you say you've done OOP for 18 years.
Yes, you asked if I started with it though
with OOP when you were 8 years old
Ah, then yes
on another note, why are people making up their names to sound japanese?
i haven't really understood that thing
You tell me
Isnt your pseudonym inspired from japanese?
It is japanese
Ok, I've seen others do the same thing. I'm not really into that 'cultural thing'
Is it connected with the anime culture?
Nope
๐ค
๐
It's a play with two words, indeed japanese as well. However, it's form a long time ago, since my days in japanese martial arts (zen + sensei)
yeah, mine was a little damaged ๐ฆ
Rip
what's that? a comic book?
It also has interviews with the Graphic Designers at GGG
oh nice
ha, i just realized you can't search by rare-name in ggg trade
yeah no partials
y u no n-gram?
@gritty olive Every 10 minutes or so. But changes are small ๐
Got any reactions to the whole storing session ids yet? I know the intention is good, but I personally would be more wary. Even though the source is open, I have no guarantee that's the version running.
Also I wouldn't want to store them as a developer ๐
I''m going to be storing my own session_id in relation to their session that will be ephemeral but will be updated every login
So i'll never store GGG credentials or Session IDs
So I can store their session_id in their browser ephemerally (cookies, local storage, etc)
But the backend will never store it
Ah okay. So only their browser will make the requests.
Well the server will make the requests but it's an ephemeral one-time request
or ? You probably need to proxy it cause of CORS?
So the server won't store the session id, but will use it
Right, well that's good at least.
I still don't know that you don't store it though ๐
It will be open source
Sure. Assuming that's what's running.
Just playing devil's advocate. I know there's not really a better solution.
Sure, I can have a staff member vet it if they want, I'm not that kind of person anyway
So when I do the server it should reduce your API requests a lot
Ideally GGG would give you an API token, JWT or something, that was restricted in what it could do.
since it will be just the server doing the request every 10 - 30 minutes
How often do you refresh poe.ninja in the client?
Yeah but JWT / OAuth is a pain for both parties ;D -- Full disclosure, I wrote a book on OAuth haha
Dude, they have free range currently ๐
I don't really have control over it
What's the lowest interval?
Manual doesn't bother me.
But every 5 seconds might ๐
That's quite often
That was just for my testing, and nobody does it ๐
And the default is?
15 minutes should be the lowest that people would use since at 1 minute it would use a lot of resources on their server
Default is 30 iirc
Okay, can still be quite a few requests daily since it's distributed.
Yeah, that's (partially) why I want to move to server
Well clearly from my point of view the server is better.
It'll reduce load on everyone else
The API runs through CloudFlare, so I'm not that worried. But ๐
What DB are you using btw?
Initially SQL Server, now Postgres.
I have an odd question, do you have an aggregated API of all types?
Nope.
The API is only there to serve the site, which is also why I tell people it's not to be considered stable.
Looks like I've served ~15GB compressed json the last 24 hours...
I'm omitting the "pay, recieve, sparkline" objects completely
Hm 15GB in an hour isn't too bad
In the last 24 hours, not per hour.
oh h
It's still quite a bit of bandwidth for a hobby site ๐
http://picsum.photos/ this one does a few terabytes in an hour
Makes sense. How expesive is the bandwidth?
Also has to use quite a few servers for the image processing?
We got it down to two servers tbh
Ah DigitalOcean and BelugaCDN.
Novynn said that "OAuth2 is in the works"
Either way I'd be storing the token in the browser ๐
I don't need it, and getting a refresh token is easy
Well, it'd be the ideal situation, where you don't have to trust any 3rd party with information you haven't explicitly given them premission to, and you can revoke it whenever you want.
I'd just prefer it if it was seperate from the session id and restricted to only accessing the private stash api.
I'm more worried about being able to access the store etc.
all of them but that's okay
Weren't you saying the same thing about JWT a few weeks ago?
Which attack do you want to run against JWT?
Use it to pretend I am the user?
lol
Now if you're doing real-time JWT generation with ... actually nope thats still exploitable
Get the page with the users session get the token that was injected, or secret, and then bam generate the jwt, and you're in
that's like saying assymetric encryption is exploitable.
If you expose anything to the client it's exploitable
yes it is, but it depends on the implementation
There is no implementation that can prevent someone pretending to be you
If you give them access to everything you have
That's what I am saying, you don't have to exploit and obtain the values inside of it
You just exploit the fact you can obtain it
The only thing that could prevent it is a proper firewall that would prevent any traffic except whitelisted servos
Then you combine that with a CSRF
So you can't remotely exploit that page
well, if you have compromised someone's computer, JWT isn't the attack surface
Having a seperate token still allows you to create one that has a restricted set of rights.
Hm. Other ways to apply curses? Cast when damage taken, cast when stunned, curse on hit, blasphemy and self cast?
That's assuming the OAuth server respects scopes :^)
and claims
Trawling through Occultists to find out how they apply curses is taking a while ๐
Some pretty crazy cases though..
Why looking through curses?
I built one of those by scraping the forum
I'm using real characters ๐
Like this one? https://www.poebuilds.io/builds/hc
I like to think that mine encourages exploration more. But yes, kind of like that.
Basically it's based around drilling down into the data set that's order by use.
I teased a screenshot in my reddit post yesterday.. Let me find it.
So instantly we can see that 36% of Cyclone users are using Starforge
and we could drill further down into that category if we wanted to.
and restrict it across multiple filters.
Can you filter things out
Yes
Well I guess you could just click the things you wanted to see though
What does the default view look like?
Alright ๐
The UI for excluding stuff is removed at the moment, but I can do it manually.
and right now for dev I'm only doing the top 20k chars. But I have tried it with 100k+
Initial view. What do you want to explore?
Yeah things get interesting when you add more datapoints to see where it levels out
Let's do Shield Charge
Yeah that'll be there.
also that part is there, but couldn't be seen on my first screenshot
Actually how do you determine Hybrid vs LowLife
oops that was the same screenshot
Low life I have a few heuristics involving blood magic etc.
Want do dig further down in your result?
Hm actually lets do Atziri's Disfavour Vaal Axe
We can zoom in on a skill maybe... Sunder?
Yeah
It'd be nice to segment skills somehow
Raider 1% 
And they made it to level 93
What about a Skellie build
Disfavour + Sunder - RT: https://i.imgur.com/wpBrAFw.png
Summon Skeleton: https://i.imgur.com/YG9Kbuh.png
@fickle yew looks cool... here's some inspiration: https://www.youtube.com/watch?v=5IuXFSC67tM
by looking at some common combinations
@simple ravine Thanks. It's surprisingly hard to find good examples of people doing these kinds of filters.
Anyway I've been using a very raw version of this tool for a couple of years myself and got inspired to see if I could tag it onto ninja at some point soon.
yeah, expensive BI tools have them
(and some open source)
Well all my drilling down is actually (wears mad scientist hat) done in the browser.
that just looks confusing, no offense
the clustering after mid 2nd video is probably something I would find useful
I enjoy exploring builds this way at least. It probably won't be as popular as the currency part, but leading up to league start I think it'd be interesting.
Yeah it's more of a retrospective
heh
Since it's based on actual players rather than forum build
However you could extrapolate commonalities into builds ๐
too many nodes and vectors/edges will just make it unusable
I like the other pages like poebuilds.io and poe-profile.info. But when I go there I just don't know what to explore.
extrapolation of ladder data is something I've thought about too, I think that's a good idea
It is also loosly inspired by: http://square.github.io/crossfilter/
and the follow-up: https://dc-js.github.io/dc.js/
If i store the new CC stuff right I could probably use that ^
yea same with http://champion.gg/
Yeah looks similar.
Yeah, you could start generating things like this -- http://champion.gg/champion/Ezreal/ADC
For builds
Wow the champion.gg has some crazy graphs.
Especially things like the "Overall Champion Matrix" which is what I was mentioning earlier with strength graphs
it's pretty allright indeed
Alright, heading to bed ๐
I wonder how much people actually use that graph though.
It's pretty, but meh..
Night.
A lot ๐
For counters
different mindset
Also yes, gnite ;D I'll elaborate more tomorrow if you want just ping me
for more serious team play, it can be useful when composing a team
Feel free to send more inspiration.
if you have an underlying graph model, where you have different attributes, such as skills, class, etc, and create edges to builds
you could pretty "easily" start extrapolating the data
Right.
i.e. "what are the common treats among top N assassin players who use blade flurry?"
Yeah I was especially thinking about this for investigating most used support gems for a given slice of the data (and a given skill obviously)
Well you can already identify some common traits like: https://i.imgur.com/Dir5z9C.png
@gritty olive here's how signal solved this problem https://signal.org/blog/private-contact-discovery/
A lot easier on the phone than browser
it's gonna be too hard to implement to be worth it so think of it as a thought experiment
OAuth2 seems flawed in quite a few ways unfortunately
still, its better than having people sharing their session tokens that's for sure
@compact isle In what ways in your opinion is it flawed?
it depends too much on the client to handle security
I'm not following, with client you mean on your side or 3rd party tool client?
the consumer
It was a while ago I implemented OAuth2 client-side, but when I did, it doesn't really handle much of the security, but perhaps I am misinterpreting what you mean.
Granted, what the client is required to do is keeping the OAuth token safe. Is that what you mean?
yep that's one of the concerns
less important when the interaction is server -> server
but for applications it's troubling
How so, are you concerned that someone would want to compromise someone's system to gain access to an OAuth token? If so, that could also be the case with servers.
And at that point, it could potentially collect a lot of tokens
well with an application you'd have to run some sort of webserver locally or have a separate system to handle the redirect
that's a big attack vector increase right there
more chances for the client to mess up their implementation
Well, I would imagine that if you have an offline application that would consume Path of Exile user's profile data, a prerequisite is to also host a server that would keep it safe, in my opinion.
I hope ggg's not gonna roll their own thing like with the online api
@compact isle I notified you guys ages ago that it's vulnerable to length extension attack
We've used IdentityServer very successfully, althoguh I suspect it's not something that would integrate well with your current architecture, considering it's a .NET thing. I'm sure there are similar projects for whatever you're using (php?)
first I'm hearing of this ๐
of course
@chrome topaz have you been able to reproduce such an attack?
right yeah I see what you mean
Anyone else seeing the error 14 is not a valid MOD_DOMAIN when extracting mods using PyPoE?
length extension attacks won't work against the API because we don't hash the URL params, we hash their value
we also validate the account names so you can't pad them
they will work if you find a request where len(all account names + hash) adds up to block multiplier
how i was suggested to use it is to sign the request, then embed it in output html so that the client js makes the request
anyway my opinion is better not take chances even though it's very low severity
yeah but it relies on the server not validating inputs before generating the key itself
if the inputs don't validate (ie. you added padding) the server skips them and then the key comparison fails
still very scary how close it comes to being vulnerable, I agree
could just use a different token for the store ๐ค
idc I'm fine with either one whatever makes people feel safer
for the store?
yeah someone mentioned earlier that they were worried about the session token and the accessing the store / purchases
you have each application register a key along side having to use the oauth2 token
this would make sure you can find any rogue applications and shut them down by revoking the application key
well the store wouldn't have an OAuth token
they'd be restricted to certain APIs through scopes
I'm not talking about the future was talking about now
ah
I have a feeling the session id isn't easily split out like that
It takes a little bit to get one
@chrome topaz hmm I think I can get the attack to work, interesting indeed
here's something scary... US ESTA application (Visa waiver program). Mine expired, so I had to apply for a new one.
They added some intersting fields:
ah mine expired recently too
I wonder how long that "optional" will stay optional
@gritty olive which APIs are you using for the web version of Currency Cop? Just the character window ones?
cloudflare doesn't like nginx
cloudflare uses nginx for their servers
Yes but it doesn't like being proxied from nginx
since nginx does a dns resolution and uses the direct ip
pass the Host header?
are you putting nginx in front of cloudflare?
I'm guessing he's using nginx to reverse-proxy requests from the browser to a 3rd party site to avoid CORS issues, and that site's behind cloudflare?
Not really, I'm using NGINX to proxy requests yes, but not from the browser
The other server is behind Cloudflare and cloudflare does not like having it's IPs being used
IPs being used where
So when NGINX uses the proxy_pass directive with a dynamic variable it uses DNS resolution to determine the upstream IP, that upstream IP ends up being the CF IP and the host becomes your host which CF does not like because it wants the Host to be the forwarded host which you cannot make NGINX do dynamically
If you wanted the technical details of it
I forgoe the entire situation and do a different form of connection
that was exactly why I said 'pass the Host header' like 24 hours ago ;)
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
does not work
when doing dynamic NGINX proxy_pass
you can read the source if you don't believe me or try it
or read the thousands of "wtf how do i do this" posts where nginx says "buy NGINX plus" or "try and do it with split traffic, lol good luck"
I'm still confused why you would need nginx proxy passing for your application
The server is built in NGINX
but sure, let me fire up nginx and give it a shot, because I'm pretty sure I've done that before
I'm pretty sure you're thinking of "non-dynamic" proxy_pass usage
and in which case yes
it would work
since you can define a server upstream and give it reference
but when you're dynamically doing it it will not
internal;
rewrite_by_lua "
local req = ngx.req
for k,v in pairs(req.get_headers()) do
if k ~= 'content-length' then
req.clear_header(k)
end
end
if ngx.ctx.headers then
for k,v in pairs(ngx.ctx.headers) do
req.set_header(k, v)
end
end
";
resolver 8.8.8.8;
proxy_http_version 1.1;
proxy_pass $_url;
}```
If you add the host headers to that
It will not add them to the upstream proxy request
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass $_url;```
Is the proxy block as of right now, it will not work
I applaud your interest but this is literally coded into NGINX to not work
source for that fact?
I don't want to spend time finding the bug report again you're free to look for it yourself
Instead I'm using http://w3.impa.br/~diego/software/luasocket/http.html
if you were wondering
listen 8080;
location / {
resolver 8.8.8.8;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass $scheme://$http_host$uri$is_args$args;
}
}```
that's the difference you're not doing it dynamically
there is a difference in what I am doing and you are doing
what URL did you have issues with requesting with your previous solution?
and how did you try to access it?
it's not a matter of "what url" it's a matter of "how are you doing it vs I am doing it"
I cannot do it the way you are because it an internal proxy
that is dynamically built using code
you're not answering my question though
right, im just curious about the end-goal
in the case of "why use $_url" is asynchronous http requests using openresty
yours is not dynamic in the sense you are using internal nginx variables to build your proxy_pass
nginx knows about them
in this scenario it is truly dynamic, nginx doesn't know about it, the request was made, and it modifies it using what it does know, and attempting to override it doesn't work since the request "was already made"
try doing it my way and see the different it requires using openresty
welp, ยฏ_(ใ)_/ยฏ - it feels like you just want to be right, rather than receiving help.
I'm not sure how to explain it better, it's just massively different forms of doing a request using proxy_pass
i.e. good luck!
I don't want help you can't do it xD
lol what?
It's not possible to do, there is no help to give
What you're saying is great if I was using internal nginx variables but I am not
im sorry, but i dont think you're listening to what im trying to say... your end goal
Or if NGINX was building the request, but it is not
but whatever, bud
I already achieved my end goal
I'm way past this xD
method = methods[req.method],
body = body,
ctx = {
headers = req.headers
},
vars = {
_url = req.url
}
})```
If you want to do a deep dive into the difference and why mine is different from yours, this is how you make the request using openresty, good luck
no, i was just trying to be helpful. i'm done
it's like saying "ios is better than android", and not saying why.
Cosockets (async) vs synchronous
it's just an empty opinion, without any insight, so it's hard to trust your insight.
Also not reliant on a proxy location block that is unstable due to programmatic dynamicness
yours is reliant on core NGINX doing the request
mine is reliant on two-factor requests
first factor is the capture of the proxy, then passing to the proxy the request and handling that with lua code, which is outside of the nginx internals and nginx defaults to preset values before lookup and during lookup (the proxy block) doesn't overwrite with whatever is in the block (by technical implementation) since nginx uses what is in it's internals since the request technically already happened
So, using something like sockets.http (synchronous) or resty.http (cosocket async using nginx internals) you use the actual socket libraries to do the request
mitigating a proxy request
so you're doing multiple things at once during the request - and that is why you want it to be asynchronous?
no you want it to be asynchronous because that is the nature of nginx
I'm wondering how much performance gain you're getting out of that, if any if comparing with a well-configured nginx with proper thread pool config.
I've understood that part.
Ok let me ask you, why do you want it to be asynchronous instead of synchronous (I'm presuming you know what the difference is from a low-level perspective)
It blocks the internal context every time you do a synchronous call
and in your case, what's the 'internal context'?
the nginx / open resty context, so any requests to that route are blocked until it finishes
that seems like something you should be able to configure though, using - a thread pool as I mentioned above.
that's not how the internals of nginx work
if you block the main context it blocks all requests
this is why it uses cosockets
internally
to avoid that
modifying the nginx configuration just modifies the cosocket implementation, not any logic that I put into place
so you're saying that with my configuration, I will only be able to server 1 request at the time
no, with yours you will be able to serve a lot because it uses nginx internals to do the request
I could fashion a location block like yours and make requests like that, yes, but that is not what I am doing and the reason is to avoid making requests like that because I want to do things outside of just "proxying"
Plus it would make a synchronous request to that location block which is what I want to avoid as well
I could use lua-resty-http to avoid it, but at that point I can just make a direct request
I want to do things outside of just "proxying"
In the same request, or?
does that matter because the second point kind of makes doing the location block moot
curious.
/api/get_something -> (nginx /proxy-location --- synchronous request to this, but nginx handles it async, but since I am inside of nginx, anyone who makes another request to /api/get_something will be blocked until this finishes)
are you building your backend with Lua and hosting it inside nginx with other words?
basically yes it runs inside of nginx
Hm, interesting. Not the way I personally would go, but still intriguing. What are the benefits that made you go that route?
speed, and ease of development
I was going to do a node or go server, but it's just honestly easier to write in lua
speed in what sense
get the speed of nginx, and development
i'm almost inclined to test, and benchmark vs nginx + kestrel+.netcore 2
yeah im well aware of those
i mean benchmark the same functionality written in lua vs asp.net core with kestrel
lol people put nginx in front of kestrel so idk
well yes, that's for a multitude of reasons
but the RIO (Registered IO) is a very interesting development they're R&D-ing right now
there's little reason nginx would be the bottleneck in a functional application anyways.
yep, which is why developing with nginx somewhat ensures that stays that way
as long as you use cosockets
well its built into nginx
that doesn't guarantee that it's faster than let's say .netcore2, java, erlang, whatever in executing whatever the actual functionality is
not doubting that at all
but the first reason you mentioned was speed
thus the topic, i guess
yeah, those choices are hilarious
is that even supported use case? i'd expect CF to shitlist your nginx server IP pretty quick
that's quite likely, depending on configuration?
im more of an incapsula guy over cloudflare
@chrome topaz supported use case of?
incapsula has a free tier too
how's it better than cf?
i just have had a terrible experience with cloudflare, and a much better after migrating to incapsula
once you're not an average site and want get a shitstorm your way, their teams are gonna try to take advantage of you, and tell you to go enterprise or they'll drop you
i had the complete opposite
you had problems with incapsula u mean?
no i mean i had no problems with CF, I was pushing like a terabyte per day and they never complained
sent me a few emails asking if i'm interested in enterprise plan and i said no
for poetrade or other?
yeah poe.trade
because i think it depends on industry
hehe
now, we did go enterprise with Incapsula out of the gate, because of recent DDoS attempts - and BGP
and when cloudflare just indirectly held our site hostage, we were losing a significant amount of money and reputation per minute, especially since we hold people's funds.
not to pry on your rev stream on poe.trade, but i guess you'd lose a bit if it went down too?
i wouldn't be able to afford any sort of "enterprise" plan anyway
google analytics already "kicked" me out by requesting that i turn on aggressive sampling
yeah, it's all fun and dandy for site with small to medium traffic, but once you go over a threshold, it's business time
and google analytics costs like $150k per year lol
lol yeah, it's not worth it for most SMBs, not us at least
and there are alternatives to Google Analytics
so you were getting, i'm guessing around 800K-1M pageviews per day at peak, @chrome topaz?
heard good things about Heap, Chartbeat
trying to remember the name of the third one
heap free tier "5,000 Sessions per Month"
chartbeat doesn't even have a pricing page
sure, but their premium won't start at 150k
it says that one million page views on chartbeat costs $10
that's honestly ridiculous
i can host my website for cheaper than doing analytics
that's usually the case, hosting is dirt cheap
but the idea is to optimize revenue with the analytics
there are others, but it depends what you want to get out of the analysis, some are more specialized than GA is
maybe i should roll my own
problem with piwik is it's written in php and therefore slow as fuck
do you use custom events and stuff? like, what do you want to get out of the analytics, more than just visits/requests
yeah i track if you have adblock enabled
not much stuff
used to track stuff like which league you searching or what theme you use
we're using App Insights (azure)
but the pricing is a bit harder to figure out, because it's based on data ingestion per day
i can guess it's gonna be a shit ton
lol
let's play a little mind game then, how much data would one assume a page visit would consume
if you're tracking basic stuff
https://mixpanel.com/pricing/ <- that was the 3rd one I was trying to remember the name of
but again, it's a bit more niched than google
does the windows version of Discord use React Native?
interesting
why hasn't anybody made a cross platform html5/js framework yet?
like, desktop and mobile
anyways i'll probably use Electron for ChaosHelper
@polar island can you take feature requests here?
i'd like to be able to search by Free-Prefix or Free-Suffix
even if it would ignore ambiguous results
it would still be useful regardless
guess that goes to @chrome topaz too
ask ggg to add it
i know that affixes aren't exposed but you can still do it from your end for non-ambiguous items, right?
too much effort
i'd rather not go UWP since there are players on platforms that don't support it
what platform?
ok, yeah - i don't care about XP users
i need to support them since the app has an overlay ui for poe itself
like mercurytrade
theres still ppl that play on it
it goes against one of my favorite rules - the 80/20 rule
according to statcounter (global stats), in march 2017 XP comprised of 2.07%
oh, wait that's incorrect sorry, stand by
3.89% in October '17
(windows market share, all other operating systems excluded)
windows xp users should upgrade
they are basically a threat to the internet at this point with unfixed vulns
mostly internet cafes in china and school districts that are still using xp
was close
I also don't support 32-bit in another project (too lazy to drop support for acquisition though)
@chrome topaz what else do you do besides the poe projects?
huh armenia's a continent now, interesting
don't tell china they aren't numba wan
wasn't really thinking about super 3rd world
81 repos lol
nah I mean, do you have a regular job, or do you hustle with projects?
don't need one at the moment
ballin'
pretty sure running a website is a regular job lol
regular employment, 9-5 job is what i meant
the profiling has already started
lol, don't worry
it's almost 50% of your repos
yeah, i don't care too much lol
less personal information on your github than I have
just fun to learn to know who'se behind these usernames a little
but I think @chrome topaz is one of the most private people I've seen here so far.
too afraid of the anti trading cabal
no reason to be a public person on the internet
remember when that currency trading website dude got death threats
I agree that there's no point in spilling your entire life on the Internet, but sometimes it becomes a bit too impersonal with a username... there's something inbetween public figure and anonymous ๐
that's both scary and really lame. it's a game.
yeah, that was nuts
I am only slightly public because I needed my projects to get a real job
and just kind of stuck with it
i'm not really afraid at all, but i haven't had any threats towards me, or anyone in my viscinity though - i guess that could change my opinion
I haven't had anything either, but I haven't created anything that is controversial
You can even check http://store.steampowered.com/hwsurvey/ @simple ravine
Steam looks quite biased
It's mostly automated
I mean, in the sense that Steam probably doesn't cater well to mac people
OSX: 1.56%
skewed ๐
I wonder if
http://prntscr.com/h9qjqw
is gonna drop when battle royal games becomes illegal in china
: P
holy crap that's a large percentage
yeah, it's mostly because of PUBG
i dont get why gamers are so stubborn with windows 7
windows 10 is by far better for performance.
oh wait, china.
never mind.
depends on game, it's less input lag on windows 10 but csgo performs better on windows 7 but a bit higher input lag ๐ค
because windows 10 fucking sucks
i disagree
me too, if you get rid of all the telemetry it's easily the best windows release yet
but if you don't do that it's complete ass
"all the telemetry" yeah
that's what they want you to think
that you got rid of all of it
linux is too wonky, too much unexpected behavior imo, and I can't stand osx
i hope microsoft collapses and we get games on linux or osx or just anything but not windows 10
someone's really mad at windows 10
yep
Electron is complete garbage
I wish I could make WPF apps with .net core tho ๐
it's like twice as fast in certain situations
anyone using the trademacro and having the problem that the league isnt saved? it always reverts to tmpstandard for me after restarting it
@simple ravine i think claiming linux has unexpected behavior is kinda funny when windows updates keep changing things ;)
i know thats not what you meant though
This the "imo" (in my opinion)
The apt dist-upgrade has failed more times than it has succeeded for me as an example
@gritty olive @mortal bone what would you suggest for an html5/js app framework?
@mellow orchid try running it as administrator
Or just fix the file permissions. Running random stuff as admin is the perfect way of ruining your OS installation
@delicate ore even when making the config file read-only after the league has been changed, the macro itself still resets the league, it has nothing to do with the file permissions or something apparently because the file is unchanged and says turmoil league, yet the macro reverts to tmpstandard, probably something in the macro code itself resets the league each startup, completely ignoring the config
@mellow orchid beta or latest release of tm?
beta
hmm, weird
gonna check
@solemn crypt when i run the macro without any .ini files created, after creation the config_trade.ini says SearchLeague=tmpstandard, after changing the league in the options it says SearchLeague=10dayturmoil(ore001) as expected. then i put it on read-only and close the script, reopen it and it tells me that following files were overwritten in the last update: config_trade.ini, now both the config_tade.ini and the config_trade.ini in the backup folder say SearchLeague=10dayturmoil(ore001) and both are read-only, but the options screen says tmpstandard and all searches are made in tmpstandard
step by step for reproducing, if you need it ^^
@simple ravine ah yeah dist-upgrade doesnt work if you have made a lot of changes, as a dev does
thats what a snapshotting file system is for
@delicate ore there isn't much in terms of html/js frameworks for desktop apps. Is there a reason you are opposed to using a compiled language?
@simple ravine wpf in .net core would be amazing...
Would completely change cross platform application development
hey didnt qt just make a new release ;)
But c++
QT#?
doesn't look like an abomination tho?
@mortal bone Wonder if it'd be feasible to create a bridge between XAML and QT# :trollface:
doesnt qt already support an xml definition of the ui
sure, but XAML and its bindings and so forth
yeah the qml thing is not terrible
Didn't know qt# existed...whoops
just use xst or whatever to translate between them :P
xslt
im sure its only like 100k lines
lol
I honestly don't have one @delicate ore otherwise I would use it
Might not be the answer you were looking for but yeah...
I wish application GUI was at the level of HTML / CSS though...
Qt is fine actually, if you use QML
not their native widgets though
and yeah, @mortal bone i'd rather leverage html5/css for gui than use native widgets
NW uses content shell iirc
what's the difference?
embeds the browser into the project
Discord uses CEF instead of Electron
Same for Steam, Spotify, etc
wait, i thought CEF embeds the browser into your app. NW does the same thing?
and also, didn't poeapp say earlier that Discord on desktop uses Electron, while the mobile version uses React Native?
anyone done lexing/parsing of code?
did a project to learn haskell doing lexical analysis, but that is pretty much the extent
@simple ravine I once had to write a Go compiler for the MIPS architecture (BSc project). Most pleasant days of my life...
@lost wigeon Ah cool, yeah I was thinking about using some kind of parser to parse Omega_K2's new python based specification file for the .dat files.
But I think I'll just go the regex route for now, as it's quite niched
uhm if you need it in another format you can just dump it
from PyPoE.poe.file.specification.data import stable
from json import dump
with open('C:/spec.json', 'w') as f:
dump(stable.specification.as_dict(), f)
That's another alternative, for sure ๐ โค
fml
now I just need to figure out a way to automate that wihtout having a dependency on python in my project
I think I'll make an Azure Function that runs on a trigger and compares the json and potentially updates the file in GitHub
running a VM for the sole purpose of that would be a bit too much
serverless computing still requires a server
but yes being able to do a script on a server you don't manage is nice
@chrome topaz did something weird happen with your forum indexing? Lots of people seem to be complaining about old forum threads being brought up on poe.trade... just wondering if we broke something you depended on :/
@compact isle nope, it used to discard threads which are too old w/o edits and does not anymore
hm okay, thanks
yeah people tend to forget about their forum shops
@compact isle it would be nice if the API exposed which league they are playing in
for serverless computing, what are the current providers?
i'm reading about AWS Lambda now, but are there others?
Azure Functions, I'm guessing Google have one too
we use some serverless functions in our statistical ingestion pipeline
works great
saves us hundreds of dollars per month actually
yea
pretty much what's happening with Ethereum and EOS with blockchain / computing
and MaidSafe too, right?
I haven't read about that one
i need to look into ethereum's
lots and lots of information
but the camera should have the option to set whether it's compressed or not
yeah it has