Arc-enabled Linux server data collection storing sst files locally

Matt Smith 26 Reputation points
2022-11-16T13:16:35.873+00:00

We have deployed Arc agents to 12 Linux hosts using the script for multiple hosts with a service principle method. The agent has installed and because the subscription has Defender for Cloud enabled the MDE agent has installed. We see the agent connected with consistent heartbeat entries in the LA workspace and Defender for Endpoint events are being fed into the workspace.

When adding a data collection rule for syslog the data is retrieved to log analytics however the sst files are backing up in /var/opt/microsoft/azuremonitoragent/events/ as if the connection is failing. The mdsd.warn log has these two entries repeated over and over...

2022-11-15T16:11:04.2607920Z: [/source/external/GenevaMonAgent-Shared-CrossPlat/src/XPlatLib/src/MSIToken.cpp:159,GetAzureMSIResponseUsingDefault]Exception occurred [Request canceled by user.] when calling Azure IMDS for MSI token with default identity. URI [/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://monitor.azure.com/] ErrorCode:-2146041343
2022-11-15T16:11:04.2608200Z: [/source/external/GenevaMonAgent-Shared-CrossPlat/src/XPlatLib/src/MSIToken.cpp:159,GetAzureMSIResponseUsingDefault]Exception occurred [Request canceled by user.] when calling Azure IMDS for MSI token with default identity. URI [/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://ingestion.monitor.azure.com/] ErrorCode:-2146041343

Is anyone else having this issue or know of a reason?

Azure Arc
Azure Arc
A Microsoft cloud service that enables deployment of Azure services across hybrid and multicloud environments.
435 questions
Microsoft Sentinel
Microsoft Sentinel
A scalable, cloud-native solution for security information event management and security orchestration automated response. Previously known as Azure Sentinel.
1,151 questions
Accepted answer
  1. Givary-MSFT 32,991 Reputation points Microsoft Employee
    2022-11-29T07:42:08.543+00:00

    @Matt Smith

    Apologies for the delay in reverting back on this issue, I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I'll repost your solution in case you'd like to "Accept" the answer.

    Answered by @Matt Smith

    265117-image.png

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Matt Smith 26 Reputation points
    2022-11-28T09:52:45.14+00:00

    It took a while to figure this out but we have an answer. This problem was caused by SSL traffic inspection on the FortiGate Firewall in front of these machines. Unfortunately FortiGate can only exclude by IP but does have a pre-defined destination group for Azure Monitor.
    Exclusion details are here if you need them https://video2.skills-academy.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-data-collection-endpoint?tabs=PowerShellWindows#firewall-requirements

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.