Table of Contents
Windows Cloud Sync Usage Notes
last edited: May 3rd, 2023
These notes are a deep dive into the functionality of the File Fabric's Windows Cloud Sync App. They are based on v18.104.22.168 of the Windows Tools.
What Cloud Sync Is
Cloud Sync is a program supplied by SME as part of SME's Windows Desktop Tooling. It is a component of SME's Enterprise File Fabric (EFF), and can be used with SME's EFF appliance or with SME's cloud-based service.
Use on a Server
Cloud Sync is a desktop tool. It is not designed for use on servers, and SME will only support its use on servers by special arrangement.
What Cloud Sync Does
Cloud Sync ensures that pairs of directories - one local and one in a storage cloud - have either identical contents or similar contents depending on the configuration being used.
When to Use Cloud Sync
Cloud Sync is applicable to a variety of use cases. Here are a few: - User works locally on a Windows desktop and wants to ensure that the her files are also stored in the cloud.
- User works in the cloud and wants to ensure that his/her files are also available on a Windows laptop.
- User works both locally and in the cloud and wants to have the same data in both places.
- User sometimes works offline and wants to ensure that her data is copied to the cloud when connected to the internet.
- Two or more users are collaborating and want to have recent copies of each others' work.
- User wants to bulk upload data from the desktop to the cloud.
As already mentioned Cloud Sync operates on pairs of directories, with each pair consisting of a local directory and a cloud directory. These pairs are referred to as sync pairs. Cloud Sync can manage several sync pairs, each with its own configuration (more on configurations later). The same directory - local or cloud - can participate in more than one sync pair, although there is seldom a requirement to do this.
The sync pair approach is very different from the approach taken by a popular storage provider's desktop sync tool. That tool requires that files you want to upload (sync) to the cloud be placed in a single designated directory. Cloud Sync works by syncing your normal working directories, and lets you sync as many of those directories as you want.
Source and Destination Directories
Cloud Sync works in both directions. Depending on how the sync pair is configured, the local directory can be the source directory for the sync, the destination directory for the sync or both. The same is true for the cloud directory.
You should create overlapping sync pairs. This can lead difficulty troubleshooting issues and unexpected behavior with Cloud Sync.
Two Methods of Executing Syncs
Cloud Sync executes syncs in two ways: interactive sync and automatic sync. With interactive syncs, Cloud Sync synchronises the contents of the two directories in a sync pair when the user initiates a sync command. With automatic syncs, Cloud Sync synchronises automatically, triggered by the passage of time since the completion of the previous synchronisation.
During an automatic sync, Cloud Sync synchronises the contents of the two directories in each sync pair according to how each pair is configured. For automatic syncs Cloud Sync operates only on sync pairs that have been selected by the user for a given execution.
Cloud Sync provides an additional sync’ing option - SyncUp Monitor - that only takes effect when automatic syncs are active. This option is explained later in the Cloud Sync Settings section of these notes.
In addition to the difference in how syncs are launched, there is another important difference between interactive syncs and automatic syncs: When launching an interactive sync, the user can exclude selected files and folders inside the folders that make up the sync pair from the sync process. When syncs are being launched automatically, all of the files and folders within the the local and cloud directories that make up the sync pair are within the scope of the sync.
When the Cloud Sync application is open the program is can be used to launch syncs interactively, and automatic syncs are suppressed. When the Cloud Sync application is closed but the user is still logged in automatic syncs will run. When the user is logged out, Cloud Sync is inactive.
Identical File Names
When performing a Sync Up, Sync Down or Sync All if Cloud Sync encounters files of the same name in the source directory and the destination directory Cloud Sync will not overwrite the newer file with the older file.
For example, assume that a sync pair has been configured for a sync up, and that the file a.txt exists in both the local directory and the cloud directory. If the copy of a.txt in the local directory is older than the copy in the cloud directory then the file will not be sync'd up.
Modes of Synchronisation
All of the options described in this section apply to both interactive and automatic modes of synchronisation.
Conceptually synchronisation between two directories consists of ensuring that they have the same contents. There are, however, several choices to be made when sync'ing two directories. Here are the modes of synchronisation that Cloud Sync supports:
- Sync All - Ensure that the local directory and the remote directory have all of the same files and folders. If any file appears in both directories, copy the newer version to the other directory.
- Sync Down - Treat the cloud directory as the master, and ensure that the local directory has all of the same files and folders as the cloud directory. If a file appears in both directories and the cloud file is newer, the local file is overwritten with the cloud file. If the local directory contains files or folders that the cloud directory does not have, they will be preserved in the local directory.
- Sync Up - Treat the local directory as the master, and ensure that the cloud directory has all of the same files and folders as the local directory. If a file appears in both directories and the local file is newer, the cloud file is overwritten with the local file. If the cloud directory contains files or folders that the local directory does not have, they will be preserved in the cloud directory.
- Clone Down - Treat the cloud directory as the master and ensure that the local directory has all of the same files and folders as the cloud directory. This may require deleting files or folders from the local directory.
- Clone Up - Treat the local directory as the master and ensure that the cloud directory has all of the same files and folders as the local directory. This may require deleting files or folders from the cloud directory.
Sync Deletion Options
When the contents of two directories are being sync’d and the destination directory contains a file that is not in the source directory, the action taken can be controlled with Cloud Sync’s Sync Deletions options:
- Down - This option is only available for Sync Downs and Sync Alls. When this option in checked for a Sync Down, if a file in the local directory does not exist in the cloud directory but did exist in the cloud directory at the time of the most recent previous sync execution, then the file will be deleted from the local directory.
A Sync Down with the Down deletion option enabled is similar to a Clone Down. The difference is that the Clone Down will delete a local file that is not in the cloud directory regardless of whether or when the file was deleted from the cloud directory.
- Up - This option is only available for Sync Ups and Sync Alls. When this option in checked for a Sync Up, if a file in the cloud directory does not exist in the local directory but did exist in the local directory at the time of the most recent previous sync execution, then the file will be deleted from the cloud directory.
A Sync Up with the Up deletion option enabled is similar to a Clone Up. The difference is that the Clone Up will delete a cloud file that is not in the local directory regardless of whether or when the file was deleted from the local directory.
Sync Versioning Options
The EFF provides a file versioning feature which, if activated, retains older versions of a file when a new version is saved. Cloud Sync provides two options for controlling how version files are handled by the sync process:
- Down - This option is only available for Sync Downs and Sync Alls. If it is enabled then version files in the cloud directory will be copied to the local directory. If it is off then version files will not be copied to the local directory.
- Up - This option is only available for Sync Ups and Sync Alls. If it is enabled then, when sync uploads a new copy of a file that already exists in the cloud, the new copy will overwrite the old copy without a version file being made whether or not EFF versioning is active. If it is off then a version file will be created in the cloud if EFF versioning is active.
Automatic Sync Scheduling
Each sync pair can have a sync schedule associated with it. If a sync pair has a sync schedule then the pair will be sync’d automatically when Cloud Sync unless automatic syncs have been suppressed (the Cloud Sync application is open). Sync pars that don’t have sync schedules won’t be sync’d automatically.
The sync schedule choices are: None, Often, Hourly, Daily and Weekly. Often, Hourly, Daily and Weekly refer to the intervals between syncs (Often means 15 minutes). These are the intervals between the conclusion of the most recent automatic sync for the sync pair and the start of the next automatic sync. Note that the actual interval between the starts of two consecutive sync jobs for a pair is the duration of the first sync job plus the configured interval.
Additional Cloud Sync Settings
When the Encrypt and/or Decrypt options are enabled and an encryption password (phrase) has been provided, Cloud Sync will respectively encrypt files using the provide phrase when syncing up, and will decrypt files using the provided phrase when syncing down. It is the user’s responsibility to retain and manage any encryption phrases used by this feature. If the phrase that was used to encrypt a file is lost or cannot be associated with the file for decrypting the file, the encrypted file's contents will not be recoverable.
When this option is enabled, Cloud Sync will log the outcome of each attempt to sync a file. SME strongly recommends that sync logging be enabled.
Other Sync Options - All Sync Pairs
The following two settings apply to all sync. pairs:
Do Cloud Meta Sync before Desktop Sync
The File Fabric maintains a cache of metadata describing the files on the underlying cloud storage. This cache is automatically updated when files are added, removed or changed through SME. If changes are made directly to the contents of the underlying cloud storage, those changes will not be reflected in SME’s metadata cache until the metadata has been re-sync’d with the cloud storage (note that we are referring to a different from of syncing here).
Cloud Sync relies on the metadata in the cache to tell it what files are on the cloud storage. To ensure that the “Cloud Sync is using up to date metadata for each sync, activate the Cloud Meta Sync option.
Retry failed syncs, up to X times
For a variety of reasons an attempt to sync a file may fail while Cloud Sync is syncing a sync pair. Cloud Sync’s strategy in this case is to re-sync the entire sync pair. This setting controls how many times Cloud Sync will retry syncing the pair in the event of failure.
Other Sync Option - Only Sync Pairs With Up Syncs
Use SyncUp Option
The Use SyncUp option applies only to sync pairs that are configured for syncing up from the local directory to the cloud directory (Sync All, Sync Up and Clone Up). When the Use SyncUp option has been selected, Cloud Sync continually syncs up from the local directory to the cloud directory - whenever a change is made to the local directory that would be sync’d up during the next sync execution, Cloud Sync syncs that change immediately to the cloud directory.
If the Use SyncUp feature is in use, we recommend that every sync pair have a scheduled sync.
The Use SyncUp option cannot be activated while the CloudSync program is open.
The Cloud SyncUp Monitor
The Cloud SyncUp Monitor is a program for monitoring and controlling syncs performed by the Use SyncUp option. The Cloud SyncUp Monitor can only be run when CloudSync is closed. Additionally, if the Cloud SyncUp Monitor is running when a scheduled sync begins, the Cloud SyncUp Monitor will be terminated.
When the Cloud SyncUp Monitor program is running, monitoring (detecting files that are ready for upload) by the SyncUp option can be turned on or off with controls in that program. If it is turned off then the SyncUp option will not detect new or changed files that are ready to upload and those files will not appear in the Cloud SyncUp Monitor window. If it is turned on then the SyncUp option will detect files that are ready for upload, and they will appear in the SyncUp Monitor window.
When the Cloud SyncUp Monitor program is running, uploading files that were determined by the SyncUp option to be ready for upload can be turned on or off with controls in the Cloud SyncUp Monitor program. If it is turned off then the files will be listed in the Cloud SyncUp Monitor window in a 'pending' state. If it is turned on then the files will appear briefly in the SyncUp Monitor window and then be uploaded.
When an attempted upload of a file that is listed in the CloudSync Monitor program's window fails, controls in the CloudSync Monitor program can be used to retry or discard the upload.
Turning off the SyncUp option while the Cloud SyncUp Monitor program is running will cause the Cloud SyncUp Monitor program to close.
If Cloud SyncUp Monitor is in use, SME recommends that a scheduled sync be configured for each sync pair that syncs up from the local folder to the cloud folder.
Sync Conflict Detection
Sync conflicts are detected based on last modified times. If the last modified time on the source and the last modified time on the destination are both greater than the last modified time that was recorded during the previous sync, a conflict is detected and no action is taken. The user must resolve the conflict before the next sync.
*Throughout these notes we refer to the storage systems accessed through the File Fabric as cloud storage. In fact, the File Fabric can be used to access both cloud storage and on-premises storage.