Microsoft's exFAT is getting absolutely huge performance boost on Linux again thanks to Sony

Microsoft and Linux written on the left and right respectively with a heart icon in the middle

More than a decade ago, in 2012, Microsoft released its 64-bit exFAT (Extended File Allocation Table) file system as the successor to 32-bit FAT32. Seven years later, in 2019, the Redmond giant made a big revelation when it announced support for the exFAT format in Linux.

Fast forward a decade, in 2022, Sony engineer Yuezhang Mo found that reducing repeated traversal of directory entries improved the performance of exFAT by up to 57%. This was pushed in Linux kernel version 6.2.

Mo is back to helping exFAT again, and this time, the latest pull request contains a bunch of exFAT-related improvements, and a performance boost is one of them. He writes:

Description for this pull request:

Fix the random stack corruption and incorrect error returns in exfat_get_block().

Optimize exfat_get_block() by improving checking corner cases.

Fix an endless loop by self-linked chain in exfat_find_last_cluster.

Remove dead EXFAT_CLUSTERS_UNTRACKED codes.

Add missing shutdown check.

Improve the delete performance with discard mount option.

For those wondering, the "discard mount" option refers to the operation of informing the underlying storage device about blocks that are no longer in use. So, when a file system is mounted with the discard option, it enables real-time discard operations. These operations automatically notify the storage device whenever blocks transition from "used" to "free."

The patch explains:

If the discard mount option is enabled, the file's clusters are discarded when the clusters are freed. Discarding clusters one by one will significantly reduce performance. Poor performance may cause soft lockup when lots of clusters are freed.

This commit improves performance by discarding contiguous clusters in batches.

The performance improvement was measured using an 80 GB file, and it took 4 minutes and 46 seconds or close to 287 seconds to delete that. With the new optimization, the task was completed in under 2 seconds, which equates to an improvement of 172 times:

Measure the performance by:

# truncate -s 80G /mnt/file

# time rm /mnt/file

Without this commit:

real 4m46.183s

user 0m0.000s

sys 0m12.863s

With this commit:

real 0m1.661s

user 0m0.000s

sys 0m0.017s