Using NTFS Junctions to Fix Application Compatibility Issues on 64-bit Editions of Windows

  • [アーティクル]

Executive Summary

This paper describes a simple way to mitigate some types of application compatibility problems with legacy applications installed on 64-bit editions of Windows Vista and newer, including Windows 7 and Windows 8. The technique relies on creating an NTFS junction or symbolic link, effectively “joining” two otherwise separate directory structures. The paper provides technical details on the underlying concepts and on how to implement the mitigation.

Technical Background Discussion

The PC industry is gradually transitioning from an installed base of primarily 32-bit systems to one of primarily 64-bit systems. In the interim, many application developers will continue to build 32-bit versions of their applications or will provide both 32-bit and 64-bit versions. To ease the effort that is involved in porting applications and to help encourage adoption of 64-bit computing, Microsoft provides Windows® 32-bit On Windows 64-bit (WOW64), an emulation layer that enables 32 bit Windows-based applications to run seamlessly on 64-bit Windows.

File System Directories on 64-bit Windows

Windows 64-bit provides “parallel” file system directories and registry keys to support the installation and running of 32- and 64-bit programs on the same computer. For example, while 64-bit programs load the 64-bit version of Kernel32.dll, 32-bit programs trying to load Kernel32.dll get redirected to the directory containing 32-bit Windows components and thus load the 32-bit Kernel32.dll. (64-bit processes can load only 64-bit DLLs, and 32-bit processes can load only 32-bit DLLs.) There are different kinds of redirections involved depending on the directories or registry keys being accessed. 64- and 32-bit processes also get different environment variables, and the results of some operating system APIs differ depending on whether the caller is a 64- or 32-bit process.

On a default installation of 32-bit Windows, most operating system components are found in %windir%\System32, and applications are installed under “%SystemDrive%\Program Files”.

On 64-bit Windows, %windir%\System32 contains core 64-bit operating system components; 32-bit core components are stored in %windir%\SysWOW64. 32-bit programs that try to access file system objects in or under System32 are always redirected to SysWOW64 via file system redirection. That is, even if a 32-bit program contains a hardcoded reference to “C:\Windows\System32\Kernel32.dll”, it will be redirected to “C:\Windows\SysWOW64\Kernel32.dll”. 32-bit programs that are 64-bit aware can call APIs to temporarily disable file system redirection if needed. These programs (including batch files or other scripts processed by a 32-bit host) can also use %windir%\Sysnative to avoid redirection when they must refer to the actual System32 directory.[1] Note that the “Sysnative” alias is defined only for 32-bit processes, and returns a “path not found” error if used by a 64-bit process.

Similarly, Windows provides both “%SystemDrive%\Program Files” and “%SystemDrive%\Program Files (x86)” directories to accommodate 64-bit and 32-bit programs, respectively. One of the benefits of these parallel directory structures is that a product that provides in-process extensions to be consumed by other programs (e.g., Oracle Java, Oracle database client) can install 32-bit components to a subdirectory of “Program Files (x86)” and 64-bit components to the same-named subdirectory under “Program Files”. 32- and 64-bit programs can then use identical code to find the appropriate extensions.

However, the file system redirection that forces 32-bit processes to SysWOW64 is not used in the selection of “Program Files” vs. “Program Files (x86)”. In particular, a 32-bit program that hardcodes a reference to “C:\Program Files” does not get redirected – it accesses the actual “C:\Program Files” directory structure, presumably containing 64-bit programs and components. For this and many other reasons, it is generally a bad programming practice to hardcode file system paths; Windows provides numerous interfaces to determine correct directory paths that should be used instead. The blog post, “FAQ: Where Do I Save Files, and How Exactly Do I Do That?” referenced at the end of this paper describes interfaces and environment variables available to C++, C#, VB. NET, PowerShell, VBScript, JScript, and batch files. The results returned by these mechanisms can depend on whether the caller is a 32-bit or 64-bit process. For example, the value of the %ProgramFiles% environment variable is typically “C:\Program Files” for 64-bit processes, but “C:\Program Files (x86)” for 32-bit processes.

Occasionally, a program may run into problems because it uses a mix of correct interfaces and hardcoded paths. While these references may have appeared to work interchangeably on 32-bit Windows editions[2], they cause “file not found” or other errors on 64-bit Windows. Ideally, the best way to fix the problem is to use correct interfaces consistently in all program components. However, changing source code or redesigning a program is not always feasible. This paper describes a simple mechanism leveraging NTFS junctions that customers can use to try to mitigate these issues.

The Windows NT file system (NTFS) supports multiple types of file-name aliasing, including hard links, symbolic links (also known as “symlinks” or “soft links”), and junctions. Hard links are supported only for files, junctions only for directories, and symbolic links for either.

A hard link allows multiple paths to refer to the same file on a single volume. For example, if you create a hard link named C:\Docs\Spec.docx that refers to the existing file C:\Users\Abby\Documents\Specifications.docx, the two paths link to the same on-disk content and you can make changes to either path. NTFS implements hard links by keeping a reference count on the file data on disk. Each time a hard link is created, NTFS adds a file name reference to the data. Because the file data is not deleted until the reference count is zero, you can delete the original file (C:\Users\Abby\Documents\Specifications.docx in the example) and continue to access the file through its other names (C:\Docs\Spec.docx). The file data shared by hard links includes not only the file’s content and alternate stream data, but also the file’s security descriptor, time stamps, and attributes such as whether the file is read-only, system, hidden, encrypted, or compressed.

By contrast, symbolic links are strings that are interpreted dynamically and can be relative or absolute paths that refer to file or directory locations on any storage device, including ones on a different local volume or even a share on a different system. A symbolic link does not increase the reference count of the original file system object. Deleting the original object deletes the data and leaves the symbolic link pointing to a nonexistent object. File and directory symbolic links have their own permissions and other attributes, independent of the target file system object.

Junctions are very similar to directory symbolic links, except that they can point only to local volumes. Junctions can be used on all supported versions of Windows, while symbolic links were first introduced in Windows Vista and Server 2008 and are therefore not available on XP/2003. Junctions and symbolic links are both based on an NTFS mechanism called reparse points, which contain data that can be used to redirect a file system request to a different location.

Junctions are widely used by Windows Vista and newer for application compatibility. For example, on a default US English installation of Windows 7, the name “C:\Documents and Settings” is actually a junction that points to C:\Users. This allows many programs that reference hard-coded legacy file paths to continue to work. For example, a program that tries to access a picture with the hardcoded path, “C:\Documents and Settings\Abby\My Documents\My Pictures\portrait.jpg” would traverse several of these application compatibility junctions and open the picture at the correct location on Windows 7: C:\Users\Abby\Pictures\portrait.jpg. Junctions are also used for backward compatibility with localized names. For example, Portuguese versions of Windows had an “Arquivos de Programas” directory instead of “Program Files”. Beginning with Windows Vista, all versions of Windows have a “Program Files” directory, with Portuguese installations adding an “Arquivos de Programas” junction pointing to it.

The permissions on these application-compatibility junctions specifically disallow listing the junction content, so that backup programs that are not junction-aware do not back up the same files multiple times through multiple paths. These junctions are also marked Hidden and System, so they do not normally appear in directory listings.

You can create hard links, symbolic links, and junctions in Windows Vista and newer with the mklink command built into Cmd.exe. Non-administrators can create hard links and junctions with mklink (assuming appropriate access rights to the locations involved), but the creation of file or directory symbolic links always requires the Create Symbolic Links privilege, granted by default only to administrators. Note that mklink is not available in Windows XP or Windows Server 2003, nor outside of Cmd.exe – there is no separate mklink.exe program that can be called by PowerShell, for example.

For scenarios where mklink is not available, one can create and view information about junctions using the Sysinternals junction.exe utility. Developers can use the CreateSymbolicLink Win32 API to create symbolic links, and the DeviceIoControl API with the FSCTL_SET_REPARSE_POINT control code to create junctions.

Proposal to use Junctions or Directory Symbolic Links

Junctions or directory symbolic links (symlinks) may be used to solve some of the application compatibility issues on Windows 64-bit mentioned earlier.

For example, say that you have a custom line-of-business program (“MoatMaintenance.exe”) that has been installed to “C:\Program Files (x86)\Contoso”. Most of the app’s components find other components by using relative paths, by building a path using the ProgramFiles environment variable or a similar Windows API, or by looking up a custom “install location” registry value that the app created during installation. However, let’s say that there is a launcher utility that was written with the hardcoded path “C:\Program Files\Contoso\MoatMaintenance.exe” embedded in it, and it always fails with “path not found”.

By creating a junction (or directory symbolic link) “C:\Program Files\Contoso”, that points to “C:\Program Files (x86)\Contoso”, the two directory paths effectively become equivalent. References to any files or subdirectories in or under “C:\Program Files\Contoso” resolve to equivalent object paths under “C:\Program Files (x86)\Contoso”. For example, the utility trying to launch “C:\Program Files\Contoso\MoatMaintenance.exe” will launch “C:\Program Files (x86)\Contoso\MoatMaintenance.exe”. A program trying to open “C:\Program Files\Contoso\Data\ConfigFiles\main.xml” will actually open “C:\Program Files (x86)\Contoso\Data\ConfigFiles\main.xml”. Note that no additional junctions or symlinks need to be created below the “Contoso” directory.

Note that there are no guarantees that this approach will fix any given application, nor that it can even be applied in any given situation. Only thorough testing can prove whether it works. And again, the ideal solution is to ensure that all programs consistently use only recommended coding practices when accessing file system paths. However, a junction or symlink is a simple technique that is viable, relatively easy to understand, and can be worth investigating.

When to Implement

This solution can be applicable when an application works on a 32-bit version of Windows and fails on the 64-bit version. In particular, the failure this solution fixes is when the application expects to find files fails in one of the Program Files directories when those files are installed in the other. The symptoms can include error messages like “file not found”, “path not found”, or “application not correctly installed”, but the application may also fail with a misleading error message or no error message at all. Sysinternals Process Monitor can be used to determine whether the program is looking for files in the wrong directory.

The ideal time to apply the fix is when the application is installed. The fix can be incorporated into the installation package, or applied immediately after installation.

How to Implement

Because both the “Program Files” and “Program Files (x86)” directories grant “write” permissions only to Administrators, only Administrators can create junctions in those locations. Creation of symbolic links always requires administrator rights. Therefore, the junction or symlink should be created by the installation package when the application is installed, as the application installation will also require administrative rights.

First, identify the directory in which the application is installed. Run the installation program, then determine whether a subdirectory was created under “Program Files” or “Program Files (x86)”. The solution will be to create a junction or symlink under the other. If a same-named subdirectory was created under both, then this technique cannot be used. Note that the target must be an application-owned subdirectory, not the “Program Files” or “Program Files (x86)” directories themselves.

For example, if the program’s installation created “C:\Program Files (x86)\Contoso”, then the solution will be to create a junction or symlink called “C:\Program Files\Contoso” pointing to the Contoso subdirectory of “Program Files (x86)”.

The mklink command is the most straightforward way to create the junction or symlink. To use it, however, you must run Cmd.exe. One way to do this is to run mklink from a batch file (file extension .bat or .cmd), which is processed within a Cmd.exe instance. A second way is to pass it as a command line argument to a new Cmd.exe instance, as shown in this example (ignore the line break):

cmd.exe /c mklink /J "C:\Program Files\Contoso" "C:\Program Files (x86)\Contoso"

The /c option tells Cmd.exe “process the following command, then exit”.

This command should then be incorporated into the application installation package, or run immediately after the installation has completed. Note that permissions on the junction or its target should not need to be customized. Standard users should have read and execute permissions by default. There is no need to apply permissions or attributes on these custom junctions similar to those Windows applies to its legacy application compatibility junctions.

Note that the technique described here applies only to 64-bit editions of Windows. Before running the described command, the installation package should determine whether it is running on 32-bit or 64-bit Windows. (I will describe some reliable techniques for doing so in an upcoming document.)

Caveats

  • You cannot create a junction or symbolic link in a directory if that directory already contains an object with that name. For example, if there already is a “C:\Program Files\Contoso” directory and a “C:\Program Files (x86)\Contoso” directory, you cannot create a junction tying them together.
  • You should use the technique described in this paper for your own apps and not for “third party” apps such as Oracle or Java client libraries. (Note that these libraries often install both x86 and x64 variants.)
  • The technique described in this paper may not always work. For example, if an app references a relative path from its install location to load DLLs, it could end up trying to load DLLs of the wrong bitness, depending on which path it believes it is installed in.
  • Deleting or renaming a directory that is referenced by a junction or symlink does not update the junction/symlink. The result would be that the junction/symlink would point to a non-existent location. An application uninstaller should take this into account and delete any such junctions/symlinks associated with the application.

As mentioned earlier, junctions and directory symbolic links are very similar, and either can be used for associating subdirectories of “Program Files” and “Program Files (x86)” on the same volume.

The only significant differences between the two are the way that they are created, and how they appear in Cmd.exe directory listings and Explorer Properties dialogs.

The mklink command to create a junction is:

mklink /J NewName Target

while the command to create a directory symbolic link replaces the /J with /D:

mklink /D NewName Target

DIR, the Cmd.exe directory listing command, labels junctions with <JUNCTION> and directory symbolic links with <SYMLINKD>, as shown in the next two screenshots. The junction’s or symlink’s target is shown in square brackets after the link name. The target may be shown as an absolute or relative path name, depending on how it was created.

clip_image002

clip_image004

Explorer represents both junctions and directory symbolic links as a folder icon with a shortcut icon overlaid. The Properties dialog for a junction is otherwise indistinguishable from that for an ordinary directory, as shown in the following screenshot.

clip_image005

The Properties dialog for a directory symbolic link, however, adds a “Shortcut” tab, as shown in this screenshot:

clip_image006

Alternatives

A naïve but inferior alternative is to install the program to both Program Files directories. E.g., install the program to “C:\Program Files (x86)\subdirectory” and then copy the entire content of subdirectory to “C:\Program Files”. This can fail in even more ways, not the least of which is that any update to one of the directories always needs to be repeated to the other.

Another alternative is to use application compatibility shims, and in particular the CorrectFilePaths (CFP) fix which enables you to modify a file path used by a particular application so that it redirects to a new location. The CFP fix can be configured to be used for specific applications using the Compatibility Administrator of the Application Compatibility Toolkit. The drawbacks of CFP include:

  • CFP is configured for specific executable images. Therefore, you have to know in advance every executable image that might need to be redirected. (By contrast, junctions and symlinks apply to all programs.)
  • CFP operates only on exact text matches at the beginning of file path specifiers. E.g., if an app has been configured with CFP to replace “C:\Program Files\MyApp” with “C:\Program Files (x86)\MyApp”, the redirection will not occur if the app uses “\Program Files\MyApp”, “..\MyApp”, “C:\PROGRA~1\MyApp”, etc.
  • Use of CFP can result in performance degradation.

References

Whitepaper: Best Practices for WOW64
https://msdn.microsoft.com/en-us/windows/hardware/gg463051.aspx

FAQ: Where Do I Save Files, and How Exactly Do I Do That?
https://blogs.msdn.com/b/aaron_margosis/archive/2011/09/25/faq-where-do-i-save-files-and-how-exactly-do-i-do-that.aspx

Windows Internals, 6th Edition, Parts 1 and 2
https://www.microsoft.com/learning/en/us/Book.aspx?ID=15253
https://www.microsoft.com/learning/en/us/Book.aspx?ID=16003

Windows Sysinternals Administrator’s Reference
https://www.microsoft.com/learning/en/us/Book.aspx?ID=13439

MSDN: Hard Links and Junctions
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365006(v=vs.85).aspx

MSDN: Symbolic Links
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365680(v=vs.85).aspx

Sysinternals Junction.exe utility
https://technet.microsoft.com/en-us/sysinternals/bb896768

Sysinternals Process Monitor utility
https://technet.microsoft.com/en-us/sysinternals/bb896645

Using the CorrectFilePaths Fix
https://technet.microsoft.com/en-us/library/cc766201(v=WS.10).aspx

[1] The “Sysnative” alias is defined on Windows Vista, Server 2008 and newer. A hotfix (https://support.microsoft.com/kb/942589) is required to use the “Sysnative” alias on 64-bit editions of Windows XP and Server 2003.

[2] Note that hardcoded paths also tended to fail when the system drive is not C:, or the name “Program Files” has been localized.

Comments

  • Anonymous
    December 11, 2012
    Sysnative hotfix for XP!!

  • Anonymous
    January 09, 2013
    Junctions & Mountpoint support was added in Win2000, hardlinks are supported on NT4 (Not sure about 3.5)

  • Anonymous
    July 11, 2015
    The comment has been removed