#Improve the Public API

1 messages · Page 1 of 1 (latest)

noble fox
#

The current public API is really quite poor from an objective standpoint. Its extremely slow, has no pagination and we are lacking many endpoints.
We need:
Pagination (with a cursor because the leaderboards update)
Endpoints for specific lookups, this could be an extension of the current endpoint even given the correct filtering parameters. As well a stats endpoint instead of just leaderboard position is very much needed.
I have proven that this is possible with my project https://github.com/damaredayo/better-battlebit-api/ which I threw together in a matter of hours.
This application pulls data from the public api, puts it into a local database and then makes aggregations based on endpoint and parameters given. There are cursors and pagination, the query takes approx. 30ms total. This can very easily be replicated to run on the live database, even if you are using SQL instead of NoSQL.

API V2 is needed.

If you cant be bothered to do it to a good standard, I'll fucking do it.

alpine solstice
#

ummm idk code or server hosting? idk what this is but yeah i agree

noble fox
#

code mate yeah

alpine solstice
#

ah ok

noble fox
#

i think even people that dont work in programming can recognise that taking over 2 seconds and giving ALL of the data instead of just what is needed and having extremely limited data ontop of that is completely unacceptable for a public api

alpine solstice
noble fox
#

precisely

misty bobcat
#

@noble fox maybe, slow things requires same slow parsers that add records to SQL where you can chunk data with WHERE id > lastQueriedId, and accidentally it becomes faster 10 times

noble fox
#

sql is really just a bad design choice for something like this I can't lie

misty bobcat
#

guess so, if you need realtime stats that updated every second.
human brain is not so fast. it could be redesigned with that point.

#

usually customer needs current stats at the moment and he checks it maybe 1-2 times per day/week. so why do you need every second update then?

noble fox
#

the human brain not being so fast should never come into it because that's not what it's about

misty bobcat
#

i conceptually agree to rewriting api, but could be solved for now

noble fox
#

that's a more "normal" lookup

misty bobcat
#

omg, did you check score every game? am little behind the times

noble fox
#

however those requests to just check myself return the entire leaderboard data which is unfathomably slow

noble fox
misty bobcat
#

yes, usually we write tonns of apis in my own projects, and where api's slow - i write proxy-api with parsers by cron every 5 minutes, that copy or even aggregate data my way, then just select it in ready state

#

that usually helps

#

yes, main api could be redesigned, but awaiting redevelopment usually takes years. so better to spend one day to write proxy with php and multi curl

#

as you said before, if you want to save events - the sql is bad thing for. but if you want your personal aggregation - its exactly the point

#

the problem of events - billions of the events (inserts). thats why sql is not a best choice.
but once you have public results of events collection - you could use any db you want

#

guess, currently existing api collects objects to server memory, then every nth-period send it to global stats server via queue like nats or rabbit, then it calculates every batch/period, and only then is available for fetching, you just unable to calculate it immediate cus of tonns of data from each game

noble fox
noble fox
#

but honestly the optimisations are possible without any form of service bus

#

this is a fundamental issue with the design and implementation of the api rather than a few optimisations and bugs here and there

#

the whole thing needs to be scrapped and rewritten

#

preferably NOT in c# like i fucking just know it was written in

#

its 2024, neither go or rust will kill you i promise x