Do researchers handle exFAT volumes correctly?

Let’s conduct a simple experiment. In the Ext4 file system, I create two files (“1.txt” and “2.txt”).

touch 1.txt 2.txt

Then, I gather file system metadata (including timestamps) for these files:

sudo debugfs -R 'stat /path_to/the_file/1.txt' /dev/block_device
sudo debugfs -R 'stat /path_to/the_file/2.txt' /dev/block_device

In my case, the output is:

For “1.txt”:

Inode: 23918802   Type: regular    Mode:  0644   Flags: 0x80000
Generation: 2882061095    Version: 0x00000000:00000001
User:  1000   Group:  1000   Project:     0   Size: 0
File ACL: 0
Links: 1   Blockcount: 0
Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
 atime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
 mtime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
crtime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
Size of extra inode fields: 32
Inode checksum: 0x9906b43e
EXTENTS:

For “2.txt”:

Inode: 23921678   Type: regular    Mode:  0644   Flags: 0x80000
Generation: 2369567438    Version: 0x00000000:00000001
User:  1000   Group:  1000   Project:     0   Size: 0
File ACL: 0
Links: 1   Blockcount: 0
Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
 atime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
 mtime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
crtime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
Size of extra inode fields: 32
Inode checksum: 0x9b052019
EXTENTS:

Now, I’m going to store something in these files. I will use the bash shell to append some text to the first file (using the output redirection syntax) and the gedit text editor to modify the second one.

echo 123 >> 1.txt
gedit 2.txt # In the gedit window, I will type "456" and hit Ctrl-S.

And let’s get the timestamps for these files again…

For “1.txt”:

Inode: 23918802   Type: regular    Mode:  0644   Flags: 0x80000
Generation: 2882061095    Version: 0x00000000:00000001
User:  1000   Group:  1000   Project:     0   Size: 4
File ACL: 0
Links: 1   Blockcount: 8
Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x6399eed2:68debfb8 -- Wed Dec 14 18:42:10 2022
 atime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
 mtime: 0x6399eed2:68debfb8 -- Wed Dec 14 18:42:10 2022
crtime: 0x6399ee07:e1dbba28 -- Wed Dec 14 18:38:47 2022
Size of extra inode fields: 32
Inode checksum: 0x4a517d18
EXTENTS:
(0):95716043

For “2.txt”:

Inode: 23860507   Type: regular    Mode:  0644   Flags: 0x80000
Generation: 534549923    Version: 0x00000000:00000001
User:  1000   Group:  1000   Project:     0   Size: 4
File ACL: 0
Links: 1   Blockcount: 8
Fragment:  Address: 0    Number: 0    Size: 0
 ctime: 0x6399eed8:16d4b85c -- Wed Dec 14 18:42:16 2022
 atime: 0x6399eed8:14ec7284 -- Wed Dec 14 18:42:16 2022
 mtime: 0x6399eed8:16d4b85c -- Wed Dec 14 18:42:16 2022
crtime: 0x6399eed8:14ec7284 -- Wed Dec 14 18:42:16 2022
Size of extra inode fields: 32
Inode checksum: 0x2b67e3c3
EXTENTS:
(0):96358507

Do you see anything unusual?

In the first case (“1.txt”), the “ctime” (inode changed) and “mtime” (last modification) fields are updated.

But in the second case (“2.txt”), all four timestamps are updated (including “crtime” and “atime” — created and last access timestamps respectively). Moreover, the inode number isn’t the same as before.

Actually, this is expected behavior! Some text editors, including gedit, replace the file during the save operation. In particular, a temporary file with updated content is written to the drive, then this file is renamed to replace the original file. This is why we see a completely new inode in the second case (“2.txt”).

(It should be noted that an old file, which was replaced during the save operation, or at least some of its metadata can be recovered, just like with any other deleted file.)

Applications do so to perform “atomic” updates to files. This means that users will see either old or new file data, but not file data in the mid-update state (e.g., when a power outage interrupted an ongoing update to file data). (Such an “atomic” update is not really atomic — the underlying file system implementation may lose the file or retain its temporary name during the interrupted rename operation if there is no journaling or copy-on-write metadata support implemented, just like in the exFAT file system.)

To account this behavior, Windows systems use file system tunnelling:

In other words, if you delete some file “File with long name.txt” and then create a new file with the same name, that new file will have the same short name and the same creation time as the original file.

But this feature doesn’t exist in the Linux world. If you create a new file, it’s a new file with its own timestamps. And this behavior can be observed across other file systems supported by Linux, including exFAT. File system drivers have no obligation to preserve the “old” value of the “crtime” field in such cases, userspace applications have no similar obligation too.

Even more, Microsoft says that file system tunnelling merely exists to support old (16-bit) applications:

The idea is to mimic the behavior MS-DOS programs expect when they use the safe save method. They copy the modified data to a temporary file, delete the original and rename the temporary to the original. This should seem to be the original file when complete. Windows performs tunneling on both FAT and NTFS file systems to ensure long/short file names are retained when 16-bit applications perform this safe save operation.

Some think that this feature is a hack:

Also: tunneling is a last-resort hack to help users avoid losing data. It has never been foolproof or robust: it will fail if there is too long a delay between when the file is replaced, or if too many files are replaced in too short a timespan. Put another way, it’s meant and designed for address the fact that programs are careless about how they handle user files, not as a substitute for proper file handling.

So, it’s okay if file system drivers don’t support the tunnelling feature.

However, some researchers seem to be unaware of the “atomic” update feature implemented in some applications and its implications on file system metadata.

In a recent article, “It is about time–Do exFAT implementations handle timestamps correctly?”, the following conclusions are drawn:

1. The MacOS and Linux changes the timestamp for creation when using TextEdit or Gedit to change the content of a file. This means the timestamps are changed by the driver, and it may already be inaccurate before parsing and interpretations of Digital Forensic tools. (Page 12.)
2. If the files are changed by using Gedit in Linux the UTCOffset are set to 0x00 and all timestamps are changed to the time of the update. Therefore, the original created date is lost. (Page 11.)
3. We recommend using X-Ways or FTK Imager to interpret exFAT, and use patterns to identify which OS has been used in order to make an accurate interpretation of the timestamps. (…) Further, we recommend law enforcement to reassess criminal cases where exFAT and timestamps have been an important evidence to make sure innocent persons have not been convicted based on misinterpreted timestamps. (Page 14.)

The first conclusion mentioned above is wrong. The Linux driver isn’t responsible for the discussed updates to the “crtime” field. The gedit application is solely responsible for the observed behavior in the Linux case.

The researchers conducted an experiment similar to described in my post and got the same results (see Table 8, page 9):

But no explanation is given as to why the driver updates only the “mtime” field in the “>>” (output redirection) case (the correct answer is: it’s not the driver, but the application, which writes to the existing file instead of replacing it with a new one).

Next, the “original” timestamp isn’t lost, so the second conclusion mentioned above is inaccurate. A directory entry containing an “old” “crtime” value is marked as unallocated. And forensic examiners can recover the “original” timestamp as long as this directory entry is intact.

The corresponding disk images were released by the researchers and I can use one of them to locate the “original” timestamp for a file modified using the gedit application — the image file examined here is “ExFAT-Experiment-E-Linux-Linux-Overwrite-Files-Manually-NativeExfat.E01“. Note that remnants of temporary files created using the gedit tool are visible too!

On this screenshot (larger), we can see three files marked as bold — a temporary file created by the gedit tool (“/Experiment-0/.goutputstream-8KK4I1“) and two corresponding files, “new” and “old”, the “old” one is marked as deleted (both share the same path: “/Experiment-0/D2022-03-02T16-11-55-tz-0-file100.txt“, but have different timestamps).

We can also see another temporary file and its corresponding “new” file (“/Experiment-0/.goutputstream-6L58I1” and “/Experiment-0/D2022-03-02T16-11-52-tz-0-file1.txt” respectively). Additionally, there is one “new” file (“/Experiment-0/D2022-03-02T16-11-55-tz-0-file99.txt“) and its remnant “old” file name (“/Experiment-0/99.txt (Partial Name)“), the nature of these remnant file names was described here.

As you can see, simply saving a file using the gedit tool doesn’t mean that its “old” file system timestamps are lost. These timestamps are lost when the corresponding unallocated directory entry is reused (overwritten).

Next, the researchers recommend using the FTK Imager tool (along with its commercial alternative, X-Ways, which is out of scope here) to interpret exFAT volumes and, further, they recommend law enforcement to reassess criminal cases involving exFAT file systems and timestamps as important evidence.

The researchers used an old version of the FTK Imager tool: 4.5.0.3. The current one is 4.7.1.2 (I finished this text on December 18, 2022). I will use both versions to see what’s wrong with them…

Let’s open the “ExFAT-Experiment-E-Linux-MacOS-Overwrite-Manually.E01” image in The Sleuth Kit (4.11.1) and in two versions of FTK Imager (4.5.0.3 and 4.7.1.2). Then, let’s see the contents of the root directory.

The Sleuth Kit shows the following list:

r/r 2051: Exp-ExFAT (Volume Label Entry)
r/r 2052: $ALLOC_BITMAP
r/r 2053: $UPCASE_TABLE
d/d 2054: Experiment-0
d/d 2057: Experiment-1
d/d 2060: Experiment-2
d/d 2063: Experiment-3
d/d 2066: .Spotlight-V100
d/d * 2069: .fseventsd
d/d 2072: .TemporaryItems
v/v 62216195: $MBR
v/v 62216196: $FAT1
V/V 62216197: $OrphanFiles

FTK Imager (4.7.1.2) shows the following list:

And the old version (4.5.0.3) gives us:

As you can see, some directories (“.fseventsd“, “.Spotlight-V100“, and “.TemporaryItems“) are absent when the image is opened in the FTK Imager tool.

Here is a related bug report sent to Exterro and their final reply (note: this bug report was sent before the release of macOS Ventura):

This issue is also documented here.

Will you recommend law enforcement reassess cases using a tool that simply ignores some directories (even allocated ones)?

Finally, there is something wrong with the test data released by the researchers. According to the paper, the only macOS version used is Monterey (page 4). But some test images mounted in the macOS operating system (“ExFAT-Base-Experiment-A-MacOS.E01”, “ExFAT-Experiment-B-MacOS-MountedWindows.E01”, and “ExFATExperiment-D-Linux-MacOS-Overwite-Files.E01”) reference Mojave (10.14.5) in the “/.Spotlight-V100/VolumeConfiguration.plist” file.

For example, the “ExFATExperiment-D-Linux-MacOS-Overwite-Files.E01” image has the following data in the “/.Spotlight-V100/VolumeConfiguration.plist” file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Annotations</key>
        <dict>
                <key>Creation_Predicates</key>
                <dict>
                        <key>false</key>
                        <integer>0</integer>
                        <key>fstype.exfat</key>
                        <integer>1</integer>
                        <key>has.uuid</key>
                        <integer>1</integer>
                        <key>interconnect.usb</key>
                        <integer>1</integer>
                        <...>
                        <key>uuid.7b28e33e-5c05-349a-b232-13d444f5a9c0</key>
                        <integer>1</integer>
                </dict>
                <...>
        </dict>
        <key>ConfigurationCreationDate</key>
        <date>2022-03-07T18:28:18Z</date>
        <key>ConfigurationCreationVersion</key>
        <string>Version 10.14.5 (Build 18F132)</string>
        <key>ConfigurationModificationDate</key>
        <date>2022-03-07T18:28:19Z</date>
        <key>ConfigurationModificationVersion</key>
        <string>Version 10.14.5 (Build 18F132)</string>
        <...>
        <key>Stores</key>
        <dict>
                <key>F93DDC7A-9E71-4FF3-AB1E-29577778C20B</key>
                <dict>
                        <...>
                        <key>CreationVersion</key>
                        <string>Version 10.14.5 (Build 18F132)</string>
                        <...>
                        <key>PolicyVersion</key>
                        <string>Version 10.14.5 (Build 18F132)</string>
                </dict>
        </dict>
</dict>
</plist>

---

Be careful with that article.

Update (2023-04-19): a corrigendum was issued.