ReFS Integrity Streams and Data Deduplication

Bernie 21 Reputation points
2020-09-03T18:56:51.633+00:00

We recently had some corruption on one of our file servers and we thought it should have healed itself, though it didn't. We found something interesting that we wanted to get confirmation on. We had a 12 bay server populated with 10 SAS HDD's and 2 SAS SSD's to make a tiered storage spaces virtual disk configured as a mirror. Our volume was formatted as ReFS with Data Deduplication enabled. While all the user files we had the volume had Set-FileIntegrity Enabled, we found the ChunkStore folder and child folders/files did not have this enabled.

In our particular case, the first sign of corruption was a ReFS event stating an object in the chuck store was corrupt and ReFS deleted the file since it could not be repaired. This same event appeared for about 10 chuck objects. Due to deduplication, these same 10 or so objects were apart of 90% of the files on that volume causing all of them to go corrupt.

Does anyone know if the chuck store should have File Integrity enabled, or if this is something automatic perhaps the system processes as part of deduplication?
It appears from the KB's on Integrity Streams that it will only check integrity if the option is enabled on the file.

Windows Server Storage
Windows Server Storage
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.Storage: The hardware and software system used to retain data for subsequent retrieval.
650 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. 2020-09-04T08:25:34.653+00:00

    Hi,

    >we found the ChunkStore folder and child folders/files did not have this enabled.

    How do you found that?

    Because integrity streams carry a performance cost, we recommend leaving integrity streams disabled on performance-sensitive systems.

    For more things, please see:

    https://video2.skills-academy.com/en-us/windows-server/storage/refs/integrity-streams

    https://video2.skills-academy.com/en-us/powershell/module/storage/set-fileintegrity?view=win10-ps

    Best Regards,
    Daniel


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.