Redis Timeout Issue with TLS enabled in .Net application

@Sunny Sharma Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

I understand that you are facing a Redis timeout issue with TLS enabled in their .Net application. You have provided the web.config file and the below error message you are receiving.

Timeout performing EVAL, inst: 1, mgr: Inactive, err: never, queue: 5, qu: 0, qs: 5, qc: 0, wr: 0, wq: 0, in: 0, ar: 0, clientName: LT114302, serverEndpoint: Unspecified/ "replace redis end point" :6379, keyHashSlot: 1440, IOCP: (Busy=0,Free=1000,Min=12,Max=1000), WORKER: (Busy=1,Free=8190,Min=12,Max=8191)

Looking at the above error message closely, we can infer the below:

queue: 5 --> there are 5 total in-progress operations.
qs: 5 --> All 5 of the above requests have been sent to the server but a response is not yet available.  The response could be:

• Not yet sent by the server
• sent by the server but not yet processed by the client.

The parameter “qs” in the error message tells you how many requests were sent from the client to the server, but have not yet processed a response.  This value keeps growing because StackExchange.Redis uses a single TCP connection and can only read one response at a time.  Even though the first operation timed out, it does not stop the data being sent to/from the server, and other requests are blocked until this is finished. Thereby, causing time outs. One solution is to minimize the chance of timeouts by ensuring that your cache is large enough for your workload and splitting large values into smaller chunks.Another possible solution is to use a pool of ConnectionMultiplexer objects in your client, and choose the “least loaded” ConnectionMultiplexer when sending a new request.  This should prevent a single timeout from causing other requests to also timeout.

A large request/response can cause timeouts. As an example, suppose your timeout value configured on your client is 1 second. Your application requests two keys (for example, 'A' and 'B') at the same time (using the same physical network connection). Most clients support request "pipelining", where both requests 'A' and 'B' are sent one after the other without waiting for their responses. The server sends the responses back in the same order. If response 'A' is large, it can eat up most of the timeout for later requests.
 Such large request/response size can increase load on Redis which in turn increases CPU Usage. Also please note that Long running commands or expensive commands cause spike in CPU Usage. Because Redis command processing is single-threaded, a command that takes time to run will block all others that come after it. You should review the commands that you're issuing to your Redis server to understand their performance impacts.
More Info: https://video2.skills-academy.com/en-us/azure/azure-cache-for-redis/cache-troubleshoot-client#large-request-or-response-size

If the above doesn't help, I would suggest you to follow the below troubleshooting steps:

Troubleshooting steps:
To start investigating why you're receiving timeouts, the first step is to determine what's keeping the operations to not complete in the synctimeout time. Below are some steps to start with:  

1. As a best practice make sure you are using the following pattern to connect using StackExhange Redis client

private static Lazy lazyConnection = new Lazy(() => { return ConnectionMultiplexer.Connect(“cachename.redis.cache.windows.net,ssl=true,abortConnect=false,password=password”); });

public static ConnectionMultiplexer Connection { get { return lazyConnection.Value; } }  

2. It’s highly recommended to have the cache and in the client application in the same Azure region.

3.Are you using the latest version of the StackExchange.Redis NuGet package? Check here to verify if you are using latest version of StackExchange.Redis https://www.nuget.org/packages/StackExchange.Redis/.There are bugs constantly being fixed in the code to make it more robust to timeouts so having the latest version would help.
4.Are you getting network bound on the client? Verify you are not getting network bound on you client. Getting Network bound on the client would create a bottleneck and cause timeouts. Moving to a higher VM size on the client to be lined up with the size/speed of the cache would give you the optimal results.
5.Are you getting CPU bound on the server or on the client? Check if you are getting bound by CPU on your client which could cause the request to not be processed under the syntimeout setting, thus causing a timeout. Moving to a higher machine size or distributing your load would help to control this.Check if you are getting CPU bound on the server from the portal. Requests coming in while Redis is CPU bound would cause those requests to timeout. One thing you could do is to split your data across multiple caches to distribute the load.
6.You can also run slowlog command. This can help you to check slow queries being executed in the Redis.
Please refer to the below documentation for the same: **SLOWLOG – Redis
**
7.Make sure that you implement all the best practices for Azure Cache for Redis to get the better performance:
https://video2.skills-academy.com/en-us/azure/azure-cache-for-redis/cache-best-practices-connection#using-forcereconnect-with-stackexchangeredis

8.Monitor the Redis “info” command and Monitor Command when the redis is approaching its limit then you are likely to start seeing performance issues.

Some useful article:
Azure Redis Timeouts - Client Side Issues
Azure Redis Timeouts - Server Side Issues
Azure Redis Timeouts - Network Issues

Hope this helps.

If you were still not able to resolve it, please open a support ticket by clicking on Get Support on this page https://azure.microsoft.com/en-us/support/options/ . We would need following details to help us investigate:

1. Cache Name
2. Cache Size
3. Date and time of errors (including timezone)
4. Exception messages with full stack trace
5. Subscription ID

**
Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.