#Users endpoint limit increase request

9 messages · Page 1 of 1 (latest)

abstract fog
#

Hi!

I am a researcher in pure mathematics and I have been working on a side project to predict a blitz or bullet game result using more features than just the corresponding ratings.

The results of the analysis are going to be made available publicly, with a deployment of the model, and comparison with the Glicko-2 expectation.

I am working with the whole database of August 2023 games, in which there are 80 million games using the most common time controls for bullet and blitz. These games were played by approximately 1.6 million users.

I am seeking to get the user endpoint for all of them. I understand that this is not what the API is meant for, but I would kindly ask if an exception can be made. If possible, it would be great to have the limits raised on my fixed ip address, as I understand that the current daily limit is set to 120k.

As a gesture of good will I am in any case going to make a donation to Lichess, thank you for all you do!

kind barn
#

What subset of user data are you interested in?

abstract fog
#

perfs

kind barn
#

I think the standard response is that, apart for the induced stress on the api/server, letting someone mass export user data is a privacy concern. However if you are conducting serious research, you should have enough resources to download and process subsets of the game databases and build the perf history for users yourself.

plucky frigate
#

If you only need to parse the headers, you dont even need to load it with a pgn parser.

abstract fog
#

mmm I didn't think about this. I have streamed the database line by line to get info for the games, keeping only ratings and usernames. I suppose that the rating deviation could be determined using the rating change and the glicko-2 formulas perhaps

plucky frigate
#

Would be a bit more error prone but with enough data i think the error will converge

#

And hell is there enough data