SCCM troubleshooting: Content Location Request for <Package ID> failed. (Code 0x80040102)
Introduction
This article is aimed at helping anyone else who finds them self in the same situation as when we encountered it there was little to no knowledge or documentation of it despite it being a major issue.
With this article, we hope to bring awareness of the issue and a solution more into the light so that it isn't as hard to resolve in the future.
Resolving this issue took over a week working on it around the clock, involved a Systems Engineer, Microsoft Support, outside Contractors, and quite a few TechNet contributors.. Even this support it was only resolved in that amount of time due to the luck of one TechNet user having encountered it before otherwise all other sources we were not even getting close after a week working on it day and night.
Background
We recently ran into an issue in System Center Configuration Manager that seemed to be a rather rare issue that cause a ton of trouble so I thought I should make the first blog about this issue so that in the future if anyone else were to fall into the same issue they might find the solution with a lot less trouble than we did.
This being our first blog please excuse any shortfalls. We would love to hear feedback, thoughts, or any comments that might help contribute to better writing in the future.
The morning started off with what seemed like a simple enough problem to look at. we were informed by our Service Desk that they were no longer able to image. At the time we didn’t think much of it because none of the other sites reported an outage just our California one. Other sites being Washington DC, London, UK, and two sites in India. However, soon enough they would all start to report the issue.
Error
Message
Failed to Run Task Sequence.
This task sequence cannot be run because the program files for SP10003 cannot be located on a distribution point. For more information, contact your system administrator or Helpdesk operator.
Screenshot
https://i.ibb.co/b2vzB4V/picture1.png
Logfile
With the following log files mentioning:
No Static content server
Content location request for SP1003341:1 failed (Code 0x80040102)
Failed to resolve PackageID=SP100334
Failed to resolved selected task sequence dependencies. Code(0x80040102)
ThreadToResolveAndExecuteTaskSequence failed. Code (0x80040102)
https://i.ibb.co/nR5yP1X/Picture2.png
Troubleshooting
At first, we thought this would be as simple as I had forgotten to distribute content to the Distribution Point but when I went and looked at SP10003 I noticed the content said it was all distributed.
So, we thought okay, maybe it was damaged somehow. With that in mind, we pulled the content off the Distribution Point and redistributed it. From there I ran the Task Sequence only to find now we were getting the same error for a different package. So, we did the same for this package. To my surprise, after that, it was yet again another package.
We will need to go through every package involved in this task sequence till it is resolved. As we went about doing just that we realized we had circled back around to SP100003 again.
Now, this brought us to the instant realization that this was a much bigger problem. So, the first reaction was to open a TechNet Forum on the issue and create a Microsoft ticket.
The next steps that were taken included:
- Rebuilding the packages and deploying them again which resulted in the same loop coming right back to the package we started with.
- Remaking the task sequence and deploying it which resulted in the same error.
- Downloading a new Wim file which Microsoft for some reason suggested.
- Reviewed and made new test boundaries to ensure things were set up right.
- Removed and added Network Access Account back.
- Lots of various other attempts that led to nothing and repeating those attempts to make sure they led to nothing.
TechNet Forum
At the TN Forum, Marin Bakarić Smilevski came out of nowhere to say
“Is content validation enabled for the Distribution Point? Consider disabling validation.”
“You mentioned you created some new packages, resulting in other packages throwing errors, this seems very similar to a problem my team came across. Are you able to perform any NEW package installation via Software Center on a client machine?”
“If the above does resolve the problem, the problem is obviously with content validation. You could set up a new DP with a test boundary, leave the content validation enabled and see what happens. If the problem appears on the new DP, you have something to test with, to find the root cause.”
Solution
The very thing meant to protect and keep my content available was preventing me from getting my content. We tested as he said, and he was right so here are the steps we took to fully resolve it:
- I went to each DP from Administration, right clicked on the DP role and unchecked content verification.
- Use the script found here: https://gallery.technet.microsoft.com/scriptcenter/Remove-Packages-from-af243f41 to remove all content from all Distribution Points
- Then distributed the content back and everything was perfect.
Notes
Some things to note. If you run into this situation it is likely you will want to deploy your Task Sequences again first so start out with your Boot Image, and Operating System, then move to the Task Sequence itself but when that is done click on the Task Sequence then click References at that bottom and look at the Compliance of each package. They should all be 100% or it will not work. From there you will move on to Applications. If you want to see what is going on with the deployment process and maybe speed it up look for a tool called Distribution Point Job Queue Manager. This will show you what is being worked on and you can tell things to start right away.
In the days to follow we contracted an outside company to review things and neither the outside company nor Microsoft could ever explain what when wrong with the Content Validation, but the contacted company did mention they normally turn that off from the get-go anyway. Their thought was that it did not automatically fix anything, users normally alerted them to an issue before it did, it created additional processes, and the risk was clearly higher than the reward given it brought our company to a halt. So this has been left disabled and everything has been good since.
The best we can figure as that the Hash on the content was somehow being changed and so Content Validation was making it unavailable without throwing any errors.