TLS certificate verification fails when connecting to VM in the same VNet

Tsoloane Moahloli 5 Reputation points
2023-05-02T11:20:44.32+00:00

Hi

I have a Virtual Machine serving a 3rd party application API in my azure VNET. We also have some microservices deployed in a containerapps environment (Actually 2 of them, 1 for production and 1 for test/uat).

From the UAT environment, I am able to run REST queries successfully (POST) against the service on the VM, but the Production environment fails with a Certificate validation error.

The microservices are coded in Typescript, using Axios as the HTTP client we use to call the service. The Certification validation error I get is:

{"TimeStamp":"2023-05-02T11:13:09.3810315+00:00","Log":"_closed: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3810357+00:00","Log":"socket: TLSSocket {"}
{"TimeStamp":"2023-05-02T11:13:09.3810399+00:00","Log":"_tlsOptions: [Object],"}
{"TimeStamp":"2023-05-02T11:13:09.3810441+00:00","Log":"_secureEstablished: true,"}
{"TimeStamp":"2023-05-02T11:13:09.3810483+00:00","Log":"_securePending: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3810527+00:00","Log":"_newSessionPending: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3810568+00:00","Log":"_controlReleased: true,"}
{"TimeStamp":"2023-05-02T11:13:09.381061+00:00","Log":"secureConnecting: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3810652+00:00","Log":"_SNICallback: null,"}
{"TimeStamp":"2023-05-02T11:13:09.3810695+00:00","Log":"servername: 'ilsliv.gracenineteen.com',"}
{"TimeStamp":"2023-05-02T11:13:09.3810737+00:00","Log":"alpnProtocol: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3810775+00:00","Log":"authorized: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3810816+00:00","Log":"authorizationError: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE',"}
{"TimeStamp":"2023-05-02T11:13:09.3810858+00:00","Log":"encrypted: true,"}
{"TimeStamp":"2023-05-02T11:13:09.3810901+00:00","Log":"_events: [Object: null prototype],"}
{"TimeStamp":"2023-05-02T11:13:09.3810942+00:00","Log":"_eventsCount: 10,"}
{"TimeStamp":"2023-05-02T11:13:09.3810985+00:00","Log":"connecting: false,"}
{"TimeStamp":"2023-05-02T11:13:09.3811027+00:00","Log":"_hadError: false,"}

Any idea how the same docker container deployed in 2 environments can behave so differently? There is nothing special about how the test environment was created - both environments were created using the same az containerapp env create command, with just the name, resource group and subnet being different.

Regards

Tsoloane Moahloli

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,472 questions
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,058 questions
Azure Container Apps
Azure Container Apps
An Azure service that provides a general-purpose, serverless container platform.
325 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Tsoloane Moahloli 5 Reputation points
    2023-05-06T19:49:46.85+00:00

    Hi, I managed to resolve it finally. It was an issue with a custom domain set at the Container App level. I removed it from prod and set one at the app level, and it worked. Luckily only one of our 'apps' in the environment has an ingress, so I set it up on that one. It seems the VM was on the same domain, resulting in the query not leaving that environment.

    What surprises me is that it manifested as a TLS certificate error, especially because the cert is a SAN certificate. A host not found error would have made sense and probably saved me a major headache.

    1 person found this answer helpful.
  2. kobulloc-MSFT 25,731 Reputation points Microsoft Employee
    2023-05-06T20:42:08.8166667+00:00

    Thank you again for posting your solution so that others who may have similar issues can easily reference this, @Tsoloane Moahloli ! Since Microsoft Q&A currently does not allow you to accept your own answer, I've reposted your solution in case you'd like to "Accept" the answer and increase the visibility for other members of the community.

    Issue:

    From the UAT environment which has microservices deployed in a containerapps environment, I am able to run REST queries successfully (POST) against the service on our VM, but the Production environment fails on the same VM with a Certificate validation error (authorizationError: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE').

    Resolution:

    Provided by @Tsoloane Moahloli :

    Hi, I managed to resolve it finally. It was an issue with a custom domain set at the Container App level. I removed it from prod and set one at the app level, and it worked. Luckily only one of our 'apps' in the environment has an ingress, so I set it up on that one. It seems the VM was on the same domain, resulting in the query not leaving that environment. What surprises me is that it manifested as a TLS certificate error, especially because the cert is a SAN certificate. A host not found error would have made sense and probably saved me a major headache.

    0 comments