How to trace network latency from my AKS to VM in the same network

Mayank 0 Reputation points
2024-09-09T04:55:16.7333333+00:00

Hi there,
We have our services running on AKS and we have one of our middlewares on Azure Virtual machine, they connect to each other through Virtual Network, I occasionally see very high latencies during these as high as few seconds for bytes of data. How can I trace this data and how can I fix this

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,804 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
2,079 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Prrudram-MSFT 24,366 Reputation points
    2024-09-09T05:27:11.3366667+00:00

    Hello @Mayank

    To trace the data and fix the high latency issue, you can follow the below steps:

    1. First, you need to test the network latency between the Azure virtual machine and the AKS cluster. You can use the tools mentioned in the Azure documentation to measure the network latency between two VMs. You can use SockPerf (for Linux) and latte.exe (for Windows) to isolate and measure network latency while excluding other types of latency, such as application latency. These tools focus on the kind of network traffic that affects application performance (namely, Transmission Control Protocol [TCP] and User Datagram Protocol [UDP] traffic).
    2. If the latency is high, you can check the network configuration of the virtual network. You can check if there are any network security groups or custom routes applied to the virtual network that might be causing the high latency. You can also check if the virtual network is peered with any other virtual network and if there is any network congestion.
    3. You can also check the network configuration of the AKS cluster. You can check if the AKS cluster is using the correct virtual network and subnet. You can also check if the AKS cluster is using the correct network security group and if there are any custom routes applied to the AKS cluster.
    4. If the network configuration is correct, you can check the application configuration. You can check if the middleware on the Azure virtual machine is configured correctly to connect to the AKS cluster. You can also check if the AKS cluster is configured correctly to accept connections from the middleware on the Azure virtual machine.
    5. If the application configuration is correct, you can check the performance of the Azure virtual machine and the AKS cluster. You can check if the Azure virtual machine and the AKS cluster have enough resources to handle the workload. You can also check if there are any performance bottlenecks in the application code.

    If you are still unable to identify the issue, you can enable diagnostic logging for the Azure virtual machine and the AKS cluster. You can use Azure Monitor to collect and analyze the diagnostic logs. You can also use Azure Network Watcher to monitor the network traffic between the Azure virtual machine and the AKS cluster.

    I hope this helps you to trace the data and fix the high latency issue. Let me know if you have any further questions.

    If I have answered your query, please click "Accept as answer" as a token of appreciation

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.