M-Stream™ File Transfer Acceleration

Transferring large volumes of data systems can be very difficult. Historically it would take users hours or even days to move very large files. The file would be first downloaded from the source system onto their local desktop (if there’s enough room). Only when that’s successful can an upload to the target server begin and fingers crossed an unreliable network doesn’t derail the transfer.

M-Stream™ File Transfer Acceleration, available with the Enterprise File Fabric™ platform, allows users to transfer files much more quickly. M-Stream transfers files in parallel streams, speeding up large file downloads, uploads, and movement between storage tiers. Additionally, M-Stream operates within the data center rather than a user’s workstation leveraging faster and more reliable networks.

M-Stream File Transfer Acceleration is used when moving and copying files, folders and objects. It may also be used by applications when uploading or downloading files through the Enterprise File Fabric API.

The minimum file size and number of threads used for M-Stream are configurable. The number of segments may be determined based on the total size of the file and the specific target provider.

To change the minimum file size and number of threads, log in as the Appliance Administrator (appladmin) and navigate to Settings > Site Functionality.

The setting M-Stream minimum file size is the size in megabytes above which a file will split into segments for copy/move operations using multiple system threads. For best performance, we recommend a minimum file size of 100 MB. A value of '0' disables M-Stream.

The setting M-Stream number of threads controls the number of threads that will be used during an M-Stream copy or move operation. For best performance, we recommend one thread per CPU core. The default is 2 and the maximum number of threads that can be created is 10.

M-Stream File Transfer Acceleration is supported for storage providers that support multipart uploading and range reads (S3 and OpenStack Swift APIs) as well as file systems that support random I/O such as block-storage providers like CIFS and NFS.

Chunking must be disabled for the target provider. (An Organization Administrator can change from the Dashboard via the Provider Settings.)

See Multipart Uploading (MPU, DLO) for more information on configuring storage providers for M-Stream.