Problems with Cache

Todd Miller 41 Reputation points
2020-09-16T22:01:20.88+00:00

I am running CM v1910
Lately I have been having widespread trouble with package caching on client machines.
When I push out a package set to download and run, I am having a fail rate of ~10%.
Looking in the CAS.LOG, I can see that the download is working and I can see the content hash succeeded which proves the content is on the client - but it never makes it into the cache.

Download completed for content HLT0048F.8 under context System  ContentAccess   9/16/2020 3:54:11 PM    6984 (0x1B48)
Hash verification succeeded for content HLT0048F.8 downloaded under context System  ContentAccess   9/16/2020 3:54:12 PM    6984 (0x1B48)
Saved Content ID Mapping HLT0048F.8, C:\WINDOWS\ccmcache\f  ContentAccess   9/16/2020 3:54:12 PM    6984 (0x1B48)
CacheManager: ADD new cache entry for id:HLT0048F Version : 8 Size : 33846K RefCount:1 LastRef Minutes : 0 State : ACTIVE PinDuration : 15058 Location : C:\WINDOWS\ccmcache\f  ContentAccess   9/16/2020 3:54:12 PM    6984 (0x1B48)
Download succeeded for download request {C99E38DA-90AA-4A7D-B052-66B1957D905A}  ContentAccess   9/16/2020 3:54:12 PM    6984 (0x1B48)
Error: DeleteDirectory:- Failed to delete Directory  with Error 0x00000003. ContentAccess   9/16/2020 3:54:12 PM    6984 (0x1B48)
Saved Content ID Mapping HLT0048F.8, C:\WINDOWS\ccmcache\f  ContentAccess   9/16/2020 3:54:13 PM    6308 (0x18A4)

But then over in the execmgr.log the content cant be located and looking in CCMCache there is no "f" folder.

I can often just use clientcenter to force the advertisement to rerun and the download/cache/run will succeed -- though it will be cached in the next higher folder ie "g"

Looking in the c:\windows\ccmcache\ it just skips over that failed "f" caching attempt and the files are all just fine in "g"

The security team swears that they have excluded C:\windows\ccmcache from AV scans - that is the first thing I thought of - and there are no records of blocked files in any AV logs.

In execmgr.log when it fails I get this error.

MTC signaled SWD execution request with program id: I-Install software, package id: HLT0048F for execution. execmgr 9/16/2020 3:54:13 PM 6308 (0x18A4)
GetContentRequest failed 0x87d01107 execmgr 9/16/2020 3:54:13 PM 6308 (0x18A4)

when it works, I see

MTC signaled SWD execution request with program id: I-Install Software, package id: HLT0048F for execution. execmgr 9/16/2020 3:57:20 PM 8720 (0x2210)
Sending ack to MTC for task with id: {628DEF35-F85E-4E38-BF3B-D7821935CE4D} execmgr 9/16/2020 3:57:20 PM 8720 (0x2210)
Executing program Install.exe in Admin context execmgr 9/16/2020 3:57:20 PM 8720 (0x2210)

It feels like a timing problem of some kind and it is driving me a little crazy. For large deployments - it is really tedious to have to attempt to rerun on all the fail to cache. I am tempted to just use run from DP on things and avoid the cache all together.

I cant think of any recent changes that might be interfering. I would think if there was an AV problem, then the files would get stopped at the temp folder and would not Hash correctly. That they are hashing right shows the client is downloading the files and the files are making it to the local disk intact and correct. So whats going on with not copying into the assigned ccmcache subfolder?

Anyone with ideas?

Microsoft Configuration Manager Application
Microsoft Configuration Manager Application
Microsoft Configuration Manager: An integrated solution for for managing large groups of personal computers and servers.Application: A computer program designed to carry out a specific task other than one relating to the operation of the computer itself, typically to be used by end users.
472 questions
0 comments No comments
{count} votes

6 answers

Sort by: Most helpful
  1. Todd Miller 1 Reputation point
    2020-10-06T15:58:20.377+00:00

    I know you told me to get lost...

    I have found that changing the advertisement to "as soon as possible" avoids the issue. Since doing that, I have not experienced the perceived race condition where the execution tries before the cache is ready.

    The problem happens when I have an advertisement where there is a single mandatory time in the past and are configured to download and run. Deployments set like that seem to have a failure rate of ~ 10% in my environment. As an experiment, I have set some more recent deployment to "as soon as possible" instead and they do not exhibit this problem. If there were any problems in my configuration, I don't think this small mandatory deadline change would affect success like this. I have recently deployed a package/program to > 500 clients with no similar cache failure. Similarly, I changed one of the deployments that was troublesome over to an "asap" mandatory time and it resolved the issue on the existing package/program.

    I truly think there is a race condition in the code where there is a time date deadline in the past. I think this is a bug. I don't have time to devote to fully testing/troubleshooting it. So I guess I am just offering this as a data point. If there is more confirmation from other customers that they have experienced the same issue, it may be worth investigating. I am on a slightly older ver of CM - build 8913 (aka 1910) Perhaps it will disappear when I move to a newer build.