Graph API - Returning 404 in certain data centers when accessing on-prem mailboxes
I recently created a process that reads/writes contacts in users' mailboxes on-prem via Graph. This process works as expected most of the time but certain times it fails with a 404. I've narrowed it down to the datacenter Graph is using. I've verified nothing is causing this issue at the network perimeter and no API calls are actually reaching the Exchange servers when the 404 is returned.
404 - North Central US
DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40fomain.com/contacts?$top=100
HTTP/1.1 404 Not Found
request-id: f245c3b7-4852-4494-aaf9-1424bfd9e41c
client-request-id: f245c3b7-4852-4494-aaf9-1424bfd9e41c
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Central
US","Slice":"SliceC","Ring":"2","ScaleUnit":"000","RoleInstance":"AGSFE_IN_56"}}
Strict-Transport-Security: max-age=31536000
Cache-Control: private
200 - East US
DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40domain.com/contacts?$top=100
HTTP/1.1 200 OK
Vary: Accept-Encoding
request-id: 2d014679-7c0c-4a68-bb3a-26e7cf91af53
client-request-id: 2d014679-7c0c-4a68-bb3a-26e7cf91af53
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"East
US","Slice":"SliceC","Ring":"5","ScaleUnit":"003","RoleInstance":"AGSFE_IN_41"}}
OData-Version: 4.0
Strict-Transport-Security: max-age=31536000
Cache-Control: private
Date: Wed, 25 Nov 2020 20:31:56 GMT
404 - North Europe
DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40domain.com/contacts?$top=100
HTTP/1.1 404 Not Found
request-id: 4ddb30db-e9f7-4a20-bd46-d58d9226fc0b
client-request-id: 4ddb30db-e9f7-4a20-bd46-d58d9226fc0b
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North
Europe","Slice":"SliceC","Ring":"4","ScaleUnit":"005","RoleInstance":"AGSFE_IN_2"}}
Strict-Transport-Security: max-age=31536000
Cache-Control: private
Date: Wed, 25 Nov 2020 21:52:06 GMT