How to model a Real-time Voting System with TTL in Redis

[restive trout]

Hello,

I'm in the process of developing a real-time voting system, where users can vote daily for their preferred games and receive in-game rewards. The system revolves around two primary use cases:

User Voting: Users vote for games daily through a REST API (User can vote every 24 hours). Each game is identified by a unique Snowflake ID (Time-sorted id), and users are identified by their pseudonyms.

Game Polling for Rewards: Games poll the system approximately every 5-10 seconds to retrieve all recent votes cast for them (Max 1000 votes per poll). Once polled, these votes should be immediately removed from the database to ensure they're counted and rewarded exactly once.

Given the high frequency of polling by games and the ephemeral nature of votes, I'm considering using Redis to efficiently handle these operations. I plan to utilize Redis' data structures and TTL capabilities but am seeking advice on the best way to model this system. Specifically:

What Redis data structures would best fit this use case, considering the need for high-performance reads and writes, and the automatic expiration of votes?
How can I ensure atomicity when retrieving and deleting votes to prevent double counting or missing a vote during polling?
Any recommended practices for handling the TTL of votes to optimize performance and resource usage?
I'm looking for insights on how to structure this in Redis to ensure scalability, performance, and reliability, given the real-time nature of the system and the need for immediate processing of votes and rewards.

Thank you in advance for your guidance and suggestions!

[remote vault]

Sets might be a good place to start. Potentially a Set for each game and the members of the Set are the users. Alternatively, it might be good to invert this and have the Sets be the users the and games they vote for are the members. Transactions can be used to make sure things are atomic.

Just some quick, off the cuff ideas.

[restive trout]

So I'd need a Set to retrieve my Game's oldest voters. And then to find out whether or not the user has already voted for a given game, I use a Hash.

I've heard of sorted sets with a score

[restive trout]

Hello ! I have a question.

If I do a transaction with the following two commands, will I have concurrency problems if several instances of my backend execute it?

ZRANGE game:2:voters 0 100

-- Remove the first 10000 votes after having processed them

ZREMRANGEBYRANK game:2:voters 0 100

[remote vault]

You can use the WATCH command with MULTI to make sure no one has messed with the key.

WATCH game:2:voters            # watch for other connections messing with the key
ZRANGE game:2:voters 0 100     # query the key and figure out what to remove
MULTI                          # start the transaction
  ZREM game:2:voters ...       # queue up commands to remove said stuff
  ZREM game:2:voters ...
  ZREM game:2:voters ...
EXEC                           # success if watched key was unmmodified, otherwise failure

If someone messes with game:2:voters, the transaction will fail. Otherwise, it'll work. If it fails, you decide what to do.

[restive trout]

Is it mandatory? Because I'll just need to fetch the first 1000 and then delete them (Only one ZRANGE and only one ZREMRANGEBYRANK)

Why not include the range in the transaction?

[remote vault]

Because the transaction won't return anything until you call EXEC. So you won't know the range to actually remove.

[restive trout]

I'm using the "ioredis" npm package

const voters = await this.redisConnection
      .pipeline()
      .zrange(`game:${gameId}:voters`, 0, 1)
      .zremrangebyrank(`game:${gameId}:voters`, 0, 1)
      .exec();

Won't it work?

[remote vault]

Pipelines aren't necessarily atomic. Transactions are.

(btw... these are excellent questions)

[restive trout]

So to ensure atomicity I'll have to make a multi and run my zrange and zrem, right?

[remote vault]

But the zrange needs to happen outside of the multi because the entire multi block doesn't execute until you call exec.

And the watch needs to happen before you read so you are guaranteed that nobody has changed it between the read and the watch.

And all of this needs to happen on the same connection.

Although technically the zrange wouldn't have to.

But it would be safer if it was.

[restive trout]

With ioredis I can't put the zrange outside the multi

I can't do that:

const voters = await this.redisConnection
      .zrange(`game:${gameId}:voters`, 0, 1)
      .multi()
      .zremrangebyrank(`game:${gameId}:voters`, 0, 1)
      .exec();

I can do this, but I don't think this is the right solution

    await this.redisConnection.watch(`game:${gameId}:voters`);
    const voters = await this.redisConnection.zrange(
      `game:${gameId}:voters`,
      0,
      1,
    );
    await this.redisConnection
      .multi()
      .zremrangebyrank(`game:${gameId}:voters`, 0, 1)
      .exec();

Because if just after the zrange there was a modification, it won't remove the good votes, will it? And they are not using the same connection I think :/

[remote vault]

If after the .zrange() there were a modification then the call to .exec() should throw an error. As long as you are using the same connection, which in this example I think you are. Not sure if ioredis is doing connection pooling or not.

Documentation doesn't mention it but maybe it's happening in the code.