SQL Server on Linux: How is Delete-On-Close Handled

  • Artikel

Windows provides an option for CreateFile to delete a file when the file is closed (FILE_FLAG_DELETE_ON_CLOSE.)  

Host Extension Handling

Linux does not expose such an option as part of the open syscall.  Instead the host extension remembers that the open request was made with the FILE_FLAG_ON_DELETE option and after closing a file issues the remove syscall on your behalf.

Linux does provide a delete-on-close like capability using the unlink or remove syscalls.  Intentionally, the Host Extension does not open the file and then invoke unlink or remove.  Linux allows a file open syscall, followed by a call to unlink or remove, and as long as a file descriptor is valid the file can be used by the application for reads and writes.  The unlink or remove syscall makes changes to the file system so searching for the file by name returns no entries and when the file descriptor is closed the file is deleted.  

The downside of using unlink or remove, and keeping the file descriptor open, is that the file becomes hidden.    This can add difficulty tracking space usage or other aspects a hidden file descriptor (still in use) can present.

Update Oct 2018 – SQL Server 2019 CTP 2.0

SQL Server 2019 changes the DELTE On CLOSE behavior used on Linux installations.   I continued to work with our Linux partners and we are able to use unlink, right after an open to achieve DELETE ON CLOSE behavior.

I changed the code to call unlink after open completes preventing new access to the file.  Only currently open file descriptors may continue to perform reads, writes and fstat activities.  Attempts to access the file with cat, open, ‘by name’ fail as the file is no longer available for name resolution.

However, ls conveniently continues to list the file as well as display size changes allowing you to monitor disk space.

When the last file descriptor is closed the file honors the ‘delete on close’ behavior and is fully removed from the system.  I even tested the following scenario.

  • Linux VM
  • mounted location on a Windows UNC path
  • open and unlink

The file can be seen from a Windows directory listing.

  • Power down the VM (simulate power outage)

While the VM is stopped I can still see the file in the Windows directory listing.

  • Restart the VM

File is removed on startup of the VM.  The file system journaling intelligently removes the file during startup.

Bob Dorr - Principal Software Engineer SQL Server

Comments

  • Anonymous
    August 02, 2018
    Btw, another approach to the problem is to just launch another process and use it to delete the file.When the system uses procfs, users on Linux can still access the "supposedly deleted file" through the eye of the process still holding handle to that file in /proc//fd
    • Anonymous
      August 02, 2018
      Hate the blog software that treated the "pid" part as HTML tag and eat it.