M-Stream™ File Transfer Acceleration

Transferring large volumes of data can be very difficult. Historically it would take users hours or even days to move very large files. The file would be first downloaded from the source system onto their local desktop (if there’s enough room). Only when that’s successful can an upload to the target server begin and fingers crossed an unreliable network doesn’t derail the transfer.

M-Stream™ File Transfer Acceleration, available with the Enterprise File Fabric™ platform, allows users to transfer files much more quickly. M-Stream transfers files in parallel streams, speeding up large file downloads, uploads, and movement between storage tiers.

M-Stream File Transfer Acceleration is activated when moving and copying large files, folders and objects. It's supported both in desktop tools (uploading and downloading) and in the appliance itself (server-to-server copies). It may also be used by applications using the Enterprise File Fabric API.

The minimum file size and number of threads used for M-Stream are configurable. The number of segments may be determined based on the total size of the file and the specific target provider.

To change the minimum file size and number of threads, log in as the Appliance Administrator (appladmin) and navigate to Settings > Site Functionality.

The setting M-Stream minimum file size is the size in megabytes above which a file will split into segments for copy/move operations using multiple system threads. For best performance, we recommend a minimum file size of 100 MB. A value of '0' disables M-Stream.

The setting M-Stream number of threads controls the number of threads that will be used during an M-Stream copy or move operation. For best performance, we recommend one thread per CPU core. The default is 2 and the maximum number of threads that can be created is 10.

M-Stream File Transfer Acceleration is supported for storage providers that support multipart uploading and range reads (S3 and OpenStack Swift APIs) as well as file systems that support random I/O such as block-storage providers like CIFS and NFS.

Chunking must be disabled for the target provider. (An Organization Administrator can change from the Dashboard via the Provider Settings.)

See Multipart Uploading (MPU, DLO) for more information on configuring storage providers for M-Stream.