How Windows Installer handles file replacement logic for versioned and unversioned files
I ran into an interesting problem yesterday where a team had built an MSI-based setup package, and they were seeing sporadic problems where some of the files failed to install. They enabled Windows Installer verbose logging and this is an example of an entry for one of the files that was failing to install:
MSI (s) (B4:4C) [11:30:07:906]: Executing op: FileCopy(SourceName=eulatxt|eula.txt,SourceCabKey=FILE1,DestName=eula.txt,Attributes=512, FileSize=29239,PerTick=32768,,VerifyMedia=1,,,,, CheckCRC=0,,,InstallMode=58982400,HashOptions=0, HashPart1=-1713153497,HashPart2=58557210, HashPart3=1046945815,HashPart4=871163290,,)
MSI (s) (B4:4C) [11:30:07:906]: File: C:\WINDOWS\system32\eula.txt; Won't Overwrite; Won't patch; Existing file is unversioned but modified
After I saw this, I took a look at the MSI package and I found that this MSI was divided into one component per file, and each component used the one file that it contained as the key path. This led me to understand the root cause of this issue. Windows Installer has a specific set of rules for file versioning and how it decides when to replace files when they already exist in the destination location. Here are the possible permutations for file versioning:
- In the case when the source and destination files are both versioned files, these rules are straightforward - if the key path file does not exist in the destination location or if it exists in the destination location and it is of a lower version, Windows Installer will install the component (and all of the files in it, including the key path). If the key path file exists in the destination location and it is of an equal or higher version, Windows Installer will not install the component but instead only reference counts the component. It becomes trickier when only one of the files is versioned or neither of them are.
- In the case when only one file is versioned, Windows Installer will install the component (and all of the files in it, including the key path file) if the source file is versioned and the destination file is not (meaning it will replace an unversioned file with a versioned one). It will not install the component but instead only reference counts the component if the source file is unversioned and the destination file is versioned (meaning it will not replace a versioned file with an unversioned one).
- In the case when both files are unversioned and have file hashes (as in the scenario I was debugging yesterday), Windows Installer will compare the created time and the last modified time of the destination file. If the times are the same, Windows Installer will compare file hashes of the source and destination files. If the file hashes are different, Windows Installer will install the component (and all of the files in it, including the key path). If the file hashes are the same, Windows Installer will not install the component but instead only reference counts the component. This means that if the key path file already exists in the destination location but has been changed sometime after it has been created, Windows Installer will not update this file, even if it is carrying a different copy of the file as part of the setup package. The intent of this logic is to prevent Windows Installer from overwriting data files that customers may have modified on their machine before running setup
- In the case when both files are unversioned and do not have file hashes, Windows Installer will compare the created time and the last modified time of the destination file. If the times are the same, Windows Installer will install the component (and all of the files in it, including the key path). If the times are different, Windows Installer will not install the component but instead only reference counts the component.
This particular MSI I was investigating was trying to update a text file named eula.txt in %windir%\system32. In some test cases, the file did not exist, in some cases it existed and was identical and in some cases it existed and was different. The 3rd and 4th versioning rules above explains why this problem would only occur on some test machines but not others. This illustrates one of the problems with using an unversioned file as the keypath for a component, especially if the file could be shared by other products on the system. The solution I recommended to the team hitting this problem was to move this eula.txt file to be installed as part of another component that installed to %windir%\system32 and had a binary file as a key path. In general you have to be careful about installing multiple files in one component (because of the Windows Installer component rules), but in this case it is a simple text file, which mitigates some of the drawbacks of including multiple files in a component (complicated servicing and incorrect uninstall behavior being the main risks here).
<update date="9/6/2005"> I listed the logic incorrectly that Windows Installer uses when both the source and destination file are unversioned. I have updated the 3rd bullet above based on the feedback in Stefan's comment. </update>
<update date="4/29/2008"> MSDN links in this post were broken, so I updated them to work again. </update>
<update date="8/3/2016"> Fixed broken link to Rob Mensching's component rules blog post. </update>
Comments
Anonymous
August 31, 2005
The comment has been removedAnonymous
August 31, 2005
The comment has been removedAnonymous
August 31, 2005
The comment has been removedAnonymous
September 01, 2005
Hi Vince - I'm sorry for the hassles you're facing while trying to uninstall SQL 2005. Can you please gather the SQL log files located at %ProgramFiles%Microsoft SQL Server90Setup BootStrapLOGFiles*.*, zip them up and send them to me at aaronste (at) microsoft (dot) com so I can take a more detailed look at what might be causing this uninstall issue?Anonymous
September 06, 2005
The comment has been removedAnonymous
September 06, 2005
Hi Stefan - thank you for the clarification. I will update the text of the main post to reflect this.Anonymous
February 06, 2006
The comment has been removedAnonymous
February 06, 2006
Hi Hochi - You cannot run 2 Windows Installer-based setups at the same time. In this scenario, you will need to use a setup.exe bootstrapper/chainer to launch each of the MSIs. You should include logic in the bootstrapper to detect if the MSI is already installed on the machine and then conditionally launch the MSI if it is detected as being needed.Anonymous
March 29, 2006
Do you have any suggestions about how to deal with .NET application configuration files (e.g. SomeApp.exe.config)? We use these to store version redirects for both .NET applications and COM apps which use .NET assemblies via Interop.
Since these are not versioned and not changed by the user, the Windows Installer will happily overwrite a newer config file with an older one.
I cannot use the EXE as a companion file, since a version change in any of the 4 dependent DLLs listed in the config file could require a config file change without recompiling the EXE.Anonymous
March 31, 2006
Hi Mike - Unfortunately, you have found a case that Windows Installer does not handle very well. You may need to author this logic as a custom action to workaround possible cases where a newer file could be overwritten by an older file. If you are using WiX for your setup (http://wix.sourceforge.net), it has a built-in custom action that you can use to modify XML data. Check out http://www.tramontana.co.hu/wix/lesson5.html#5.11 for more information.Anonymous
April 06, 2006
I am trying to get a MSI to update the bitmap we use as the background. However, the MSI does not seem to want to replace the file the new bitmap has a later modified date then the one currently on the system. It installs the bitmap if it does not exist.Anonymous
April 07, 2006
Hi Kevin - It seems like the BMP should have been updated in that scenario. Do you have a verbose log file from that installation scenario that you could send me? If not, you can use the steps at http://blogs.msdn.com/astebner/archive/2005/03/29/403575.aspx to gather it and you can contact me at Aaron.Stebner@microsoft.com and send it to me so I can take a look.Anonymous
September 06, 2007
One of the features included in the WiX v3.0 source files included in the C# project template in theAnonymous
September 06, 2007
One of the features included in the WiX v3.0 source files included in the C# project template in theAnonymous
September 06, 2007
PingBack from http://msdnrss.thecoderblogs.com/2007/09/06/subtle-problem-with-major-upgrades-in-wix-files-in-the-media-center-sdk/Anonymous
September 06, 2007
PingBack from http://msdnrss.thecoderblogs.com/2007/09/07/subtle-problem-with-major-upgrades-in-wix-files-in-the-media-center-sdk-2/Anonymous
January 28, 2009
Curious, does Installer have equivalent provisions - not to remove what's modified - for other objects - registry entries, ODBC elements, .ini setttings, etc.? it looks like it's only for the files. For registry settings, specifically, it looks like it does not. This is important not as much during removals as during the upgrades.Anonymous
January 28, 2009
The comment has been removedAnonymous
February 10, 2009
The comment has been removedAnonymous
February 11, 2009
The comment has been removedAnonymous
February 11, 2009
The comment has been removedAnonymous
February 12, 2009
The comment has been removedAnonymous
February 12, 2009
The comment has been removedAnonymous
February 13, 2009
The comment has been removedAnonymous
February 16, 2009
Hi Aron, I have noticed one thing here from the file field of the main MSI, b.txt d.exe b.txt 515215 0 10 but the .MSP has only 2 files which are a.exe and b.txt, so I have edited the row in the file field as b.txt a.exe b.txt 515215 0 10 After editing also I have not able to get the correct time on the patched file.Looks like the file is getting replaced but it somehow retains the current time of patch installation. Also I would like to mention that I create a zero kb dummy files (a.exe,b.txt,c.txt,d.exe) in the folder before I patch with latest a.exeand b.txt.Does it make any difference? Thanks, Deepu Abraham KAnonymous
February 16, 2009
The comment has been removedAnonymous
February 16, 2009
The comment has been removedAnonymous
February 17, 2009
Yes, My application follows the entire steps as mentioned in the link correctly.The only difference which I am seeing is that the unversioned file is getting copied but the file time gets changed to same as installation time.Anonymous
February 17, 2009
Hi DeepuAbrahamK - Which file time is being updated to the same time as installation time? Is it the creation time, the last modified time, or both?Anonymous
February 17, 2009
Its the 'b.txt' (file is unversioned) which gets the last modified time only as that of installation time.The 'a.exe' gets copied correctly and shows the correct time (this file is versioned).So only the last modified time of b.txt is updated to the same time as installation time.Anonymous
February 18, 2009
Hi DeepuAbrahamK - Just to make sure I'm understanding your scenario correctly, what you're seeing is that the unversioned file ends up with a last modified time that matches the install time (and not the time the file was modified in your build process). However, the versioned file ends up with a last modified time that matches the time the file was modified in your build process (and not the install time). Is that accurate? Is the creation time for b.txt also set to the install time so that the creation time matches the last modified time? If so, I think that makes sense from Windows Installer's perspective - it is resetting the creation and last modified time so that the next time it needs to update the file, it can tell whether or not the file has been modified by the user.Anonymous
February 18, 2009
Yes, You are right regarding the first paragraph. But the creation time for b.txt is not same as that of the last modified time. Does MSIEXEC modify the file?Anonymous
February 18, 2009
Hi Aron, One more question, Is it necessery to have the old files or original installed files in the INSTALLDIR before I patch? Since I have the file names of the original installed files,I create those files (0 kb files) inside the INSTALLDIR using my code so that MSIEXEC will not shout.Does this cause any problem?Anonymous
February 19, 2009
Hi DeepuAbrahamK - Typically a patch is an update to existing files that your product previously installed. I would expect your original MSI to have installed those files to the user's system before your patch attempts to update them. Creating 0 byte files can be helpful for testing purposes in some cases, but you should focus your patch testing effort on real-world customer scenarios (install your product, which includes version 1 of b.txt, then install your patch, which includes version 2 of b.txt). I don't know enough about the internal implementation details of Windows Installer to answer your other question (does MSIEXEC modify the file). I would expect the modified time of the file to match the time that your build process stamped on it, but that doesn't appear to be the case based on your experience. I'm not sure the exact modified time that the file ends up with in the end matters for this scenario though.Anonymous
February 24, 2009
Hi Aron, Thanks for your help & patience, much appreciated.I have kind of got to the bottom of the problem. There was a callback set in the API MsiSetExternalUI(INSTALLUI_HANDLER,..)and this was doing the trick.While installing/patching the application, callback kicks in.I have changed few things in the callback and it did the trick after I tried to set the date and time of files to be patched as the original installation date and time.While patching on that it patched correctly with the same date and time as that of the build date. Thanks again, Deepu Abraham KAnonymous
June 03, 2010
i am working in Microsoft Patch file creation for my application setup. i have used msimsp exe to create patch file and it is created sucessfully without error but MSP is not overwrite the existing application exe. if older application exe is available in installation path then MSP ignores latest exe installation. if i rename older exe then MSP installs latest exe in application path. Please let me know how to resolve this issue. Regards, P. SaravananAnonymous
June 04, 2010
Hi Saravanan - The first thing I’d suggest doing is gathering a verbose log file from the installation attempt where you apply the MSP and it doesn’t update your application file. The verbose log will typically give you more information about what Windows Installer is doing behind the scenes and it can be helpful to narrow down root causes of this type of issue.Anonymous
June 11, 2010
Hi Aaron, This information is much helpful! I meet similar issue that some unversioned files are modified after installation and they are not updated during applying patch as result. My question: Is there a way to make all unversioned files in patch update directly no matter whether their Created date and Modified date are different or not? My current workaround is to set REINSTALL=FeatureToUpdate and REINSTALLMODE=amus during applying patch. This causes the source installation files are required and all of then are reinstalled. This is not good... Thanks, DewardAnonymous
June 12, 2010
The comment has been removedAnonymous
November 27, 2012
During replacement of versioned files, is the version of the file on the disk actually checked or is the version information read from the msi file? Remember, MSI is declarative. I have a situation where the version column in the MSI has been populated for an unversioned file. When i say unversioned file, it means to say that if you were to check the file properties, you would not see a version. Now i build a patch for this msi. In the upgraded msi, the version is again retained the same as in the target msi. When the patch is applied, the file is not replaced stating that the version of the file on the disk is the same as the one contained in the patch. This means to say that windows instaler service references the version from the catched msi package rather than actually reading the version of the file on the disk at install time. Could you please confirm this? Regards, Kiran HegdeAnonymous
November 28, 2012
Hi Kiran Hegde - I thought that Windows Installer would look at the files on disk in this type of scenario and not rely solely on the data in the MSI. However, I don't have access to the Windows Installer source code, so I don't know that for sure. What is the exact text that you see in the verbose Windows Installer log file in your patch install scenario?Anonymous
November 28, 2012
I see the following text: ; Won't Overwrite; Won't patch; Existing file is of an equal versionAnonymous
November 28, 2012
FileCopy(SourceName=<file name>,SourceCabKey=<file name>.2AE7ACF6_3BDA_4F10_969B_D26E431A3BFF,DestName=<file name>,Attributes=4096,FileSize=77824,PerTick=65536,,VerifyMedia=1,,,,,CheckCRC=0,Version=6.0.0.0,Language=1033,InstallMode=8650752,,,,,,,) MSI (s) (BC:14) [12:06:39:182]: File: <desination file path>o; Won't Overwrite; Won't patch; Existing file is of an equal version This is what i see.Anonymous
November 29, 2012
Hi Kiran Hegde - I'm not sure how to explain this behavior. It is possible that MSI is only doing this for incorrectly marked files (such as an unversioned file marked with a version in the MSI), and/or it might only do this for patch install scenarios. Do you have any cases where a versioned file is marked with a version number that is incorrect in the MSI to test out this theory? You will probably need to post a question on a Microsoft Windows Installer forum and/or contact Microsoft Technical Support to get an official answer about what behavior to expect in this type of scenario.Anonymous
June 16, 2013
The comment has been removedAnonymous
June 18, 2013
Could someone kindly comment about the previous comment? Thanks, Kiran HegdeAnonymous
June 19, 2013
The comment has been removedAnonymous
March 01, 2014
The comment has been removedAnonymous
March 03, 2014
The comment has been removedAnonymous
September 09, 2014
The comment has been removedAnonymous
September 09, 2014
Hi Daivd - My blog post is saying the same thing as the flow chart in the MSDN topic you linked to. The flow chart says "Are the Created and Modified dates of copy-B different?" The No answer is "Replace copy-B with copy-A." That means the replacement will happen if created and modified dates are not different (or as I worded it in my blog post, if they're the same).Anonymous
January 10, 2015
What about the case where you are trying to downgrade? My company wants me to setup our installer to be able to downgrade the product if needs be. In the past they have put out a product version that didn't work for some of our and the customer had to uninstall the new version and install the older version. With our new product they want the customer to just be able to install the old version over the new version. What I am seeing though is that the versioned files on the system are not being removed and replaced with the older, lower versioned files. With windows installer is it even possible to do a downgrade?Anonymous
January 11, 2015
Hi Shawn - Windows Installer does not allow installing an older version of a file over the top of a newer version of a file. You would have to uninstall the newer version of the file first in order to be able to install the older version of the file. Depending on your scenario, it might be an option to author your new MSI to do a major upgrade of the old MSI and automatically uninstall the old MSI behind the scenes as a part of its install process. You can do a web search for "Windows Installer major upgrade" to find more information about that option to see if that might work for you.Anonymous
August 03, 2016
The comment has been removed- Anonymous
August 03, 2016
The comment has been removed
- Anonymous