INFO commandstats - cmdstat_client failed_calls | Redis | Page 1

harsh wharf Dec 22, 2023, 9:07 AM

#

I recently deployed a new version of telegraf metric collector and I see that it is now reporting 2 new fields in the commandstats output, one being failed_calls. I am using redis-py and have a question regarding the reported failed CLIENT related calls:
cmdstat_client:calls=114604,usec=552723,usec_per_call=4.82,rejected_calls=0,failed_calls=114604

This is puzzling because my application is running fine and all redis calls are wrapped in exception handling and error logging and nothing is failing. Further I ran MONITOR and no CLIENT calls are being reported which makes this more puzzling. I am just using a fairly standard Redis connection.
redis_conn_decoded = StrictRedis(
password=password,
unix_socket_path=/tmp/redis.sock,
charset='utf-8', decode_responses=True)
Redis is bound to 127.0.0.1 on port 6379 and has socket enabled and the application only uses socket connections. The failed calls seem to coincide with the application use of Redis and they increment almost constantly.
Could someone perhaps enlighten me as to why or what could be responsible for reporting these failed calls?

#

I guess I should add using:
redis-stack-server-6.2.6-v7.rhel8.x86_64.tar.gz
redis_version:6.2.12
redis-py 5.0.1

burnt moss Dec 22, 2023, 12:50 PM

#

CLIENT calls are usually run by the client to configure it with the Redis server. What pops out to me is that redis-py, I think, recently added behavior to call CLIENT SETINFO (so if we are tracing something we at least know what client library was using it) when it's starting up, that command isn't available before 7.2 so I would naturally think that would fail on Redis 6.2.

harsh wharf Dec 22, 2023, 1:53 PM

#

@burnt moss thanks very much! That looks like exactly what it is. 4.6.0 does not have the SETINFO in connection.py but 5.0.1 does which explains it.

burnt moss Dec 22, 2023, 1:56 PM

#

The only thing that gives me pause is that this is happening 100k times? Is it just happening whenever it connects or is it constantly spamming the server?

#

If it's just on connection that should be fine, it's probably pipelined when the connection is coming up and is basically free on Redis' side, but if it's constant spam. . . Probably worth opening an issue in redis-py

harsh wharf Dec 22, 2023, 2:15 PM

#

@burnt moss it seems to have no impact on Redis or the application whatsoever, just increments the failed_calls counter. It happens on every connect. The only reason I discovered this is due to the fact that I am testing a new version of my application and doing a new build and full install of the server from scratch, with all the dependencies and related applications updated to the latest versions. I was leaving updating Redis from 6.2 to 7.2, redis-py and telegraf where updated and after leaving the new build running for a day, I happened to stumble onto the new commandstats failed_calls metric.
Once again thanks for pointing me in the correct direction, I shall open an issue.

#INFO commandstats - cmdstat_client failed_calls