Saturday, November 27, 2021

Storage Format for USB Hard Disk

 I recently bought a 5 TB USB3 hard disk to back up my data (mostly my photo and video files.  I had a data disaster recently, so I bought the EaseUS disaster backup data recovery tool, but it required a huge amount of data to restore).

Out of the box, without paying attention to the factory default format it has (extFAT), the 5 TB space has only about 0.6 TB space left.  I doubt I had that much-used data (as the source of the recovered data is from a 1 TB hard drive).  After doing some investigation (the detail is down below), I found out that the allocation unit size (AUS) of the removable hard disk is pretty big, I think it is too big to storage average files.  So I am in the process of backing up the data to my other hard drive (NTFS, 4k AUS), before I can reformat the drive to have a smaller AUS.

Meanwhile, I am not decided whether to stick with extFAT or change it to NTFS.  I don't really care about portability to other OSes, especially Apple products (sorry MacOS!), as I am pretty much a Windows (plus Linux) user.   Yes, I am all aware that so far Linux supports R/W to NTFS in userspace only (the kernel driver only supports read and some limited write access, but that requires some enabling).  But, the good news is that, according to the recent news, a company called Paragon is willing to make its full-blown proprietary NTFS driver (which fully supports NTFS features) to the Linux community.  The effort is planned to be available starting in Kernel 5.15.  This will really boost the performance and features of NTFS in Linux.[1]

To have 256K AUS seems too expensive.  According to [3] for Test 3 and 4 (Read & Write, 1 GB of data file), extFAT is slightly faster, but for smaller files (Test 1 and 2, 1 GB file size), NTFS prevails.  For duplication or deletion (Test 8 and 9), NTFS is more than 1.60 faster than FAT32 and 1.3 faster than extFAT).

Some useful information:

For  2 TB to 16 TB hard drive, 4 KB AUS is enough for an NTFS-formatted hard drive.[2]


Disk Partition Information:


C:\Windows\System32>diskpart


Microsoft DiskPart version 10.0.22000.1
Copyright (C) Microsoft Corporation.

On computer: ASUS-ROGSTRIX-X

DISKPART> list disk

  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online         3726 GB  1024 KB   *    *
  Disk 1    Online          698 GB      0 B   *    *
  Disk 2    Online          298 GB  1024 KB   *    *
  Disk 3    Online          931 GB  3072 KB        *
  Disk 4    Online         4657 GB      0 B        *
  Disk 5    Online           14 GB      0 B

DISKPART> list volume

  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
  ----------  ---  -----------  -----  ----------  -------  ---------  --------
  Volume 0     X   BACKUP_DISK  NTFS   Simple      3726 GB  Healthy
  Volume 1     Z   SpannedDisk  NTFS   Spanned      996 GB  Healthy
  Volume 2         System Rese  NTFS   Partition    549 MB  Healthy
  Volume 3     C                NTFS   Partition    929 GB  Healthy    Boot
  Volume 4                      NTFS   Partition    544 MB  Healthy
  Volume 5                      FAT32  Partition    299 MB  Healthy    System
  Volume 6                      NTFS   Partition    609 MB  Healthy    Hidden
  Volume 7     I   One Touch    exFAT  Partition   4657 GB  Healthy
  Volume 8         EFI          FAT32  Partition    200 MB  Healthy    Hidden
  Volume 9     E   Win10Pro     NTFS   Removable     14 GB  Healthy

DISKPART> select disk 4

Disk 4 is now the selected disk.

DISKPART> list partition

  Partition ###  Type              Size     Offset
  -------------  ----------------  -------  -------
  Partition 1    System             200 MB    20 KB
  Partition 2    Primary           4657 GB   201 MB


select partition 2

Partition 2 is now the selected partition.


DISKPART> filesystems

Current File System

  Type                 : exFAT
  Allocation Unit Size : 256K
  Flags : 00000000

File Systems Supported for Formatting

  Type                 : NTFS (Default)
  Allocation Unit Sizes: 4096 (Default), 8192, 16K, 32K, 64K, 128K, 256K, 512K, 1024K, 2048K

  Type                 : exFAT
  Allocation Unit Sizes: 512K, 1024K, 2048K (Default), 4096K, 8192K, 16384K, 32768K

DISKPART>





Ref:

[1] Linux boosts Microsoft NTFS support as Linus Torvalds complains about GitHub merges | ZDNet

[2] Default cluster size for NTFS, FAT, and exFAT (microsoft.com)

[3] Flexense - Data Management Software - FAT32 vs. exFAT vs. NTFS USB3 Performance Comparison

No comments:

Post a Comment