FIM Troubleshooting: Installation Hot Fix Update Rolls Back Without Warning or Error / (SolutionPackUtility)

Today I worked on a hot fix installation issue. The installation goes through and hits a certain point, and then without warning, or error the installation rolls back. I realize that this can be a bit frustrating as to why this happens. I have come across two different reasons that this happens. Here is documentation on one, and how I went through troubleshooting the issue.

Symptoms

The installation goes through and hits a certain point, and then without warning, or error the installation rolls back.

Windows installer verbose log file

I obtained a Windows Installer Verbose Log File through the following steps:

  1. Open an Administrative Command Prompt (Right Click on Command Prompt and select Run As Administrator)
  2. Navigate to where the MSP for the hotfix is located
  3. Type at the command prompt: msiexec /p "[msp file name with path]" /L*v installationlog.txt
  4. Once it rolls back, we should have installationlog.txt
  5. Open the log file and review

In reviewing the Windows Installer Verbose Log File, I searched for “Return Value 3” and found the following information.

CAQuietExec: Retracting ILMPasswordPortal.wsp

CAQuietExec:An exception occurred while running Microsoft.IdentityManagement.SolutionPackUtility.exe: A deployment or retraction is already under way for the solution "ilmpasswordportal.wsp", and only one deployment or retraction at a time is supported.

CAQuietExec: An error occurred while retracting FIM portal solution packs.

CAQuietExec: Error 0xfffffffa: Command line returned an error.

CAQuietExec: Error 0xfffffffa: CAQuietExec Failed

CustomAction PatchRemoveFIMPWDPortal returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)

Action ended 15:53:06: InstallExecute. Return value 3.

 

CAQuietExec: Adding feature for MicrosoftILMPortalCommonDlls.wsp

CAQuietExec: An exception occurred while running Microsoft.IdentityManagement.SolutionPackUtility.exe: Feature 'MSILM2CommonWebConfiguration' (ID: 7c43ce5b-a59b-44f5-9e8a-50bd1b696145) is already activated at scope 'http://localhost'.

CAQuietExec: An error occurred while deploying FIM portal solution packs.

CAQuietExec: Error 0xfffffffa: Command line returned an error.

CAQuietExec: Error 0xfffffffa: CAQuietExec Failed

Action ended 9:28:34: InstallExecute. Return value 3.

The verbose log indicated that the Microsoft.IdentityManagement.SolutionPackUtility had another deployment or retraction already started for the “ilmpasswordportal.wsp”.

CAQuietExec:  Executing SharePoint Worker Process Group Update9511b8d8-fed3-44c7-b901-a5293793d271.



MSI (s) (1C!3C) [13:05:32:956]: Closing MSIHANDLE (175) of type 790531 for thread 4668



MSI (s) (1C!3C) [13:05:32:956]: Creating MSIHANDLE (176) of type 790531 for thread 4668



CAQuietExec:  The SharePoint Worker Process Group Update9511b8d8-fed3-44c7-b901-a5293793d271 job failed with the following error.  This job will be skipped. Failed to connect to an IPC Port: The system cannot find the file specified.

Resolution

Option 1: delete ilmpasswordportal.wsp

To resolve this issue, I removed the “Windows SharePoint Services Solution Deployment for “Ilmpasswordportal.wsp” with the following steps:

  1. Open SharePoint 3.0 Central Administration under Administrative Tools.
  2. Select Operations > Timer Job Definitions
  3. Select Windows SharePoint Services Solution Deployment for “Ilmpasswordportal.wsp”.
  4. Click the Delete button.

Option 2: run administrative timer jobs

When it appeared that the solution pack was stuck in a “deploying” state & causing the FIM timeout, and option 1 failed we executed this command on both servers prior to attempting the install again:

  1. Open elevated command prompt
  2. cd to C:\program files\common files\microsoft shared\web server extensions\12\bin
  3. stsadm -o execadmsvcjobs – note if your running a farm this must be run on all font end servers

This just forces any administrative timer jobs in the queue to run immediately.

SharePoint also caches a lot of information locally to reduce SQL trips, so just as a precaution we also cleared its config cache. Detailed instructions for that can be found here.

Option 3: deactivate/uninstall feature from stsadm

After trying Option 1 and Option 2 we found error in the install log referencing a feature that was already activated: An exception occurred while running Microsoft.IdentityManagement.SolutionPackUtility.exe: Feature

'MSILM2CommonWebConfiguration' (ID: 7c43ce5b-a59b-44f5-9e8a-50bd1b696145) is already activated at scope 'http://localhost'.

This indicates that when it retracted the solution during one of the failed attempts, it probably didn’t fully clean up that feature. We looked in Central Admin in the site settings and didn’t see that feature. What we needed to do next is forced the deactivation and removal:

(notice the feature ID used in this command is the feature ID from the error message. If you encounter this again, change the feature ID as needed):

    1. Open elevated command prompt
    2. cd to C:\program files\common files\microsoft shared\web server extensions\12\bin
    3. Run the following commands:
      1. stsadm -o deactivatefeature -id 7c43ce5b-a59b-44f5-9e8a-50bd1b696145 -force -url http://localhost
      2. stsadm -o uninstallfeature -id 7c43ce5b-a59b-44f5-9e8a-50bd1b696145 -force