Understanding Application and Data Files

  • Article

Applies To: Windows HPC Server 2008

Windows HPC Server 2008 is flexible regarding the organization of task input, output, and error files. You can use the Working Directory, Standard Input, Standard Output, and Standard Error task properties to specify a local or Universal Naming Convention (UNC) file path to any shared location. Tasks can also operate on files stored in the default working directory. In either case, using a central file store on a shared folder, preferably on a file server, is recommended.

Note

When specifying file paths, remember that these files are accessed from the compute node. For example, “C:\Temp” refers to the Temp directory on the compute node that is running the application, not the Temp directory on the head node or on the client computer.

Program files

Use the following guidelines when you specify the program file in the command line for your task:

  • If the application exists on all compute nodes and has been added to the Path environment variable, type only the executable name. For example, type myapp.exe.

  • If the application exists on all compute nodes and has not been added to the Path environment variable, type the full local path to the application on each compute node. For example, C:\Program Files\myapp.exe.

  • If the application is installed on a file share, specify the UNC path to the executable file. For example, type \\server_name\Program Files\myapp.exe.

Data files

By default, the standard input, output, and error files are relative to the working directory of the compute node that is running the application. The default value for the Working Directory task property is the submitting user's home directory on the node (%userprofile%, which typically points to C:\Users\user_name\Documents).

You can use the Working Directory task property to simplify task access to data files on a shared folder. For example, if you set a working directory of \\fileserver\fileshare\ and a Standard Input of somefile.txt, the Standard Input will be read from \\fileserver\fileshare\somefile.txt.

If you do not specify Standard Output and Standard Error files for your task, the results are directed to the HPC Job Scheduler Service database and appear as the task’s output fields in the Task Properties dialog box. The database stores up to 4 KB of data per task. Any additional data beyond 4 KB is truncated. If you specify the Standard Output and Standard Error files, the task output is directed to those locations and is not stored in the HPC Job Scheduler Service database.

Additional considerations

  • When accessing a network share, use the full UNC path instead of using Driver Letter Mappings, since mappings do not persist between different logon sessions.

  • Creating a file store for input, output, and error files is usually a coordinated effort between the cluster administrator and the user, and requires administrator's permissions and oversight over shared resources and the user's specific knowledge of the projects, jobs, and files involved.

Additional references