Microsoft Structured Storage and Slack Space with SyncWeb

Related Links: Why SyncWeb Uses Microsoft Structured Storage

When using SyncWeb Online Data Backup, plan for approximately 5% extra disk space when determining storage requirements - here's why.

SyncCom and Microsoft Structured Storage: SyncCom leverages a technology called Microsoft Structured Storage for storing all user data (for the reason why, follow this link. An analysis of structure storage demonstrates the ability to create a "mini-file system" within a file (Structure Storage Object). As such, structure storage has similar cluster constraints to that of typical hard drive geometry. In the case of SyncWeb, the structure storage cluster size is 4 Kilobytes (KB).

Slack Space with Hard Drives: Slack space is the amount of disk space that is wasted by having a large cluster size in relation to the actual file size. For example, if a 1KB file is stored on a disk with a cluster size of 4KB (default NTFS cluster size for drives larger than 2GB) there will be 3KB bytes of slack space that And since no two files can occupy the same cluster, that wasted slack space goes unused for the life of the file. The following Graphic depicts this:

The same holds true if a file is only slightly larger than the cluster. For example, a 5KB file written to the cluster will actually occupy 8KB of space as depicted in Figure 2.

Slack Space with SyncWeb: Much like standard hard drive geometry, SyncWeb also uses a small, 4KB cluster size within the Structured Storage file. Therefore, as with standard file system storage, any single file less than 4KB written to the storage object will occupy the full 4KB of space.

Based on the above, it may appear that there could be significant wasted space for anyone backing up many small files. However, while this is certainly true as it applies to typical hard drive geometry, the issue is somewhat mitigated by a technique used by MS Structure Storage called the MiniFAT.

Unlike traditional hard drive cluster geometry based on the FAT or MFT, the MiniFAT permits more than one file to occupy the remaining free space of the cluster as demonstrated in Figure 3 resulting in the "slack space" being reclaimed.

Therefore, while a single backup of a large number of small file may result in what appears to be a significant amount of wasted space, in practice you'll find that over time, the percentage of slack space will decrease as more backups of individual files occur (multiple versions of the same file).

That being said, you will always have some small percentage of slack space since there are many times when numerous small files, such as shortcuts, temp files, small text files, etc, are only backed up once. In addition, the minimum size of a Structured Storage backup file on the server is 20KB regardless of the size of the file being backed up. For this reason, you should plan for approximately 5% slack space when determining storage requirements.