NTFS: unallocated data marked as allocated

No operation on a file is allowed to include unallocated (deleted) data into the user-readable area of that file. Otherwise, an unprivileged program could read data from a deleted file even if such access was forbidden when this file was allocated.

But this is not an issue when dealing with files readable by privileged programs only (because such programs can read allocated and unallocated data from a file system directly). However, allocated files containing pieces of unallocated data are very rare (unlike the slack space, such data is a part of file’s data).

One example is the $Extend\$RmMetadata\$Repair file, which can be found on fixed drives (including external drives which report themselves as non-removable) attached to a Windows 10 machine. This file is used to track volume corruptions spotted by the NTFS driver; the file has two data streams used as storage for corruption metadata: $Corrupt and $Verify. According to Microsoft, these streams contain metadata about confirmed and unverified corruptions respectively.

When these streams are allocated, they reference some clusters which aren’t sanitized (wiped) during the allocation operation, so data from these clusters (which is actually deleted data, because these clusters were marked as unallocated before) is available as data in the $Corrupt or $Verify stream.

Here is an example:

In my tests, the total size of such “unallocated” blobs was:

• 2 797 568 bytes for a small drive (4 GiB) without corruption records,
• 2 797 388 bytes for a small drive (4 GiB) with one unverified corruption record (see below),
• 3 919 872 bytes for a large drive (256 GiB) without corruption records.

Here is another example (the drive was filled with the “PTRN” byte pattern prior to the test, one unverified corruption record was written to this stream by the NTFS driver during the test, note the call stack recorded – it could become a good artifact):

It should be noted that when a volume is formatted, the same clusters can be used for the $Corrupt and $Verify streams. So, these streams may also reference corruption records from a previous file system.