Troubleshooting BGP peering on Azure ExpressRoute

kaickaic 0

I have set up an ExpressRoute connection with private peering on Azure. Despite enabling and provisioning peering, BGP between the on-prem CE router and the Azure ASN 12076 is down. However, I am able to ping between the CE and PE IP address, and ARP can see the CE MAC address. Do I need to create a gateway or take any other steps to establish BGP peering?

GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-10-31T10:15:51.19+00:00

Hello @kaickaic ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

I understand that you've set up an Azure ExpressRoute connection with private peering and even after enabling and provisioning the peering, BGP between the on-prem CE router and the Azure ASN 12076 is down.

As mentioned in the below Azure ExpressRoute troubleshooting doc,

If the state of a eBGP peering between an MSEE and a CE/PE-MSEE is Active or Idle, check if the assigned primary and secondary peer subnets match the configuration on the linked CE/PE-MSEE. Also check if the correct VlanId, AzureASN, and PeerASN values are used on MSEEs, and if these values map to the ones used on the linked CE/PE-MSEE. If MD5 hashing is chosen, the shared key should be the same on MSEE and CE/PE-MSEE pairs. If you need to change any of these configurations on an MSEE router, see Create and modify routing for an ExpressRoute circuit.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-troubleshooting-expressroute-overview#validate-bgp-and-routes-on-the-msee

Did you validate all your configuration?

Regards,

Gita
kaickaic 0 Reputation points

2023-10-31T10:17:15.9833333+00:00

Adding notes:

the route table summary would show its status as "Connect" but up/down = 0

From the CE side, BGP on cisco router would show its state as "never"

the only take i see is that the onPrem ASN is not private, but configuration in the portal does not throw any error
kaickaic 0 Reputation points

2023-10-31T10:22:02.3333333+00:00

Yes, from the route table summary, the states is shown as "Connect", but up/down =0; vlan ID is matching, ASN is correct for Azur 12076 (see attach screenshot) though on the CE side, the CE IP address is on a SVI instead of a route port with subinterface; My only speculation would be because our on prem ASN is not a private ASN number, but the portal does not throw any error. Would that matter?
GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-10-31T10:22:33.78+00:00

Hello @kaickaic ,

If you are using Public ASN, do you own that ASN?

AS Number requirements for Private peering: Private and public AS numbers. You must own the public AS number if you choose to use one.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-circuit-peerings#peeringcompare

Also, have you already created the ExpressRoute gateway and linked it to a Virtual Network in Azure?

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-howto-linkvnet-portal-resource-manager

Regards, Gita
kaickaic 0 Reputation points

2023-10-31T10:32:25.8366667+00:00

We do own the public ASN but it's configured as private peering. does it allow public ASN in private peering?
GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-10-31T10:35:14.1466667+00:00

Yes, @kaickaic . Private peering allows Public ASN as long as you own it.

On the Cisco router, what is the result of show ip bgp summary?

Also, have you already created the ExpressRoute gateway and linked it to a Virtual Network in Azure?

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-howto-linkvnet-portal-resource-manager
kaickaic 0 Reputation points

2023-10-31T10:49:35.6533333+00:00

Ah ok so that's what i'm missing. bgp summary show "never" as state. I've read somewhere about the expressRoute gateway, but couldn't find the exact tutorial. I remember i need to define GatewaySubnet as name but don't have the exact steps. Does it allow to setup a ExR GW without a Vnet?
GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-10-31T10:57:07.61+00:00

@kaickaic , is the BGP state "never" or the UP/Down shows "never"? AFAIK, the state should show Active or Idle or some number. Could you please share the result of show ip bgp summary?

Does it allow to setup a ExR GW without a Vnet?

No, to create an ExpressRoute gateway, you need an Azure Vnet. Because you will need to create a GatewaySubnet from within the Vnet and then deploy the ExR gateway in that GatewaySubnet.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-howto-add-gateway-portal-resource-manager

Once the ExR gateway is created, you can link the ExR circuit and the Vnet using this gateway.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-howto-linkvnet-portal-resource-manager
kaickaic 0 Reputation points

2023-11-02T03:09:09.8533333+00:00

Thanks again for the pointers. BGP state is Idle, and up/down=never but my problem is that i don't have an ER gteway. I now get it the GatewaySubnet resides in Vnet. May i ask if the GatewaySubnet has any correlation with the private peering subnet? My question is that I'm not certain how the PE ip address of the private peering is associated to the resource group (or where the BGP is terminated at)?
kaickaic 0 Reputation points

2023-11-02T07:07:30.9033333+00:00

I may need to skip ahead of the game.

I've created a vWAN environment with a vHub in the resource group, and have then created a ER gateway in the vHub, and assigned the ER circuit to the ER gateway. The "Hub Connection Status" is "This Hub", and the connection provisioning state is "Succeeded" however the BGP status on prem is still Idle/Active

THe vHub private address space is different than the BGP private peering IP of the ER, does it matter?
GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-11-02T12:48:57.6366667+00:00

Hello @kaickaic , if the BGP state is idle, then we may need to drill into the logs to find why the BGP peering is down between MSEE and CE.

Could you please check the below details:

Is BFP (Bidirectional Forwarding Detection) configured properly?

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-bfd#enabling-bfd

Are the Cisco interfaces configured properly to use eBGP and advertise routes to Microsoft?

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-config-samples-routing#cisco-ios-xe-based-routers

Do you have shared key/MD5 hashing configured? If yes, the shared key should be the same on MSEE and CE/PE-MSEE pairs.

NOTE: The limit is a maximum of 25 alphanumeric characters. Special characters aren't supported.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-howto-routing-portal-resource-manager#private

I've also seen instances where customer configured BGP with the same ASN in local and remote BGP peers. From the customer configuration, the remote peer ASN must be Azure's 12076 ASN.

Please validate the ASN configuration again.

Regards,

Gita
GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-11-06T15:05:16.6266667+00:00

@kaickaic , do you have any updates on this issue?
kaickaic 0 Reputation points

2023-11-07T04:19:21.6933333+00:00

Hi @GitaraniSharma-MSFT , we actually teared down the expressroute and rebuilt them in the new vHub environment, using the same transit IP and connects right away. It was strange and we can't figure out why we can ping across but not able to establish BGP. But it's working now, thank you for the help

1 answer

GitaraniSharma-MSFT 49,256 Reputation points Microsoft Employee

2023-11-07T16:08:38.8333333+00:00

Hello @kaickaic ,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I'll repost your solution in case you'd like to "Accept" the answer.

Issue:

I understand that you've set up an Azure ExpressRoute connection with private peering and even after enabling and provisioning the peering, BGP between the on-prem CE router and the Azure ASN 12076 is down.

What we tried:

I requested you to validate your configuration as mentioned in the below Azure ExpressRoute troubleshooting doc,

If the state of an eBGP peering between an MSEE and a CE/PE-MSEE is Active or Idle, check if the assigned primary and secondary peer subnets match the configuration on the linked CE/PE-MSEE. Also check if the correct VlanId, AzureASN, and PeerASN values are used on MSEEs, and if these values map to the ones used on the linked CE/PE-MSEE. If MD5 hashing is chosen, the shared key should be the same on MSEE and CE/PE-MSEE pairs. If you need to change any of these configurations on an MSEE router, see Create and modify routing for an ExpressRoute circuit.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-troubleshooting-expressroute-overview#validate-bgp-and-routes-on-the-msee

You validated the configuration and confirmed that all is correct.

On the Cisco router, we checked the result of show ip bgp summary and it was showing BGP state as Idle, and up/down=never.

I advised you to validate the below configurations again:

Could you please check the below details:

BFP (Bidirectional Forwarding Detection) configuration: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-bfd#enabling-bfd

Cisco interface configuration to use eBGP and advertise routes to Microsoft: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-config-samples-routing#cisco-ios-xe-based-routers

If shared key/MD5 hashing configured, the shared key should be the same on MSEE and CE/PE-MSEE pairs.

NOTE: The limit is a maximum of 25 alphanumeric characters. Special characters aren't supported.

Refer: https://video2.skills-academy.com/en-us/azure/expressroute/expressroute-howto-routing-portal-resource-manager#private

I've also seen instances where customer configured BGP with the same ASN in local and remote BGP peers. From the customer configuration, the remote peer ASN must be Azure's 12076 ASN. Requested you to validate the ASN configuration again.

You came back with an update saying that you teared down the existing ExpressRoute setup and rebuilt them in a new Virtual WAN environment, using the same transit IP and it connected right away.

Solution:

Rebuilding the ExpressRoute setup within Virtual WAN with the same transit IP and configurations fixed the issue. The ExpressRoute BGP is now UP and connected.

If you have any other questions or are still running into more issues, please let me know.

Thank you again for your time and patience throughout this issue.

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Share via

Troubleshooting BGP peering on Azure ExpressRoute

1 answer