M-Stream Usage Notes

M-Stream is a File Fabric feature that speeds up the uploading, downloading and copying of very big files. Large files are split into pieces, sent in parallel over multiple streams, and reassembled back into a contiguous file or object at the target. In this way M-Stream maximizes network bandwidth, minimizes network latency, and increases resiliency, particularly for wide area networks.

M-Stream is required to be activated by license key for the enterprise on-premises File Fabric. It is not available for the Software-as-a-Service hosted solution.

M-Stream will function when moving data between any of the following storage providers provided they support range reads.

  • CIFS/SMB
  • Swift API (SwiftStack, OpenStack Swift, Swift based clouds)
  • S3 API (certified on AWS S3, Scality, Cloudian)
  • Filesystem (FS, Local block storage, StorNext, Ceph FS and many others)

M-Stream will also accelerate the downloads to a Windows desktop running the SME Cloud tools version 10.5.10 or later, from one of the above providers.

To enable M-Stream for SME File Fabric follow the steps below:

From Appladmin Account

Go to Settings » Site Functionality, adjust the M-Stream field settings in light of the following information:

  • M-Stream minimum file size
    • If set to 0, M-Stream will be disabled)
    • If set too small, then too much overhead will be added, adversely affecting the performance.
    • If set too large, then full potential of M-Stream will not be utilized
    • Default multipart upload size (This only applies to S3 based storage and needs to be between 5MB and 4GB)
    • Ranged read block size for FS and HPSS providers: (This is in 1803.03 and above. It only applies to FS or HPSS providers and specifies the block size for reading data. In most testing 4096 (4K) to (4M) should be used. Must be a base 2 integer)
    • Write block size for FS and HPSS providers: (Same as point 4, but controls writing to FS or HPSS)
  • M-Stream number of threads
    • Should be set to a max of 10 or ( vCPUs - 2) of web node, whichever is lower
    • Enable Direct Upload: Should be set to Off

From Team Admin Account

Go to Dashboard » Provider Settings:

  • Set Chunking “OFF”
  • Turn MPU/DLO “ON” (If provider supports MPU/DLO)

Go to Policies » Encryption:

  • Disable Encryption. (Accelerated transfer will not be used for files encrypted with SME encryption. SSE encryption or other storage encryption allow accelerated transfer.)

From Cloud Tools » Control Panel » Bandwidth:

  • Enable Allow multistream downloads
  • Pick minimum file size (100 - 4096 MB)
  • Pick up to 4 streams of data (Max 1 per CPU)
  • Determine if Cloud Drive should also use M-Stream
  • Provider support MPU/DLO and range read.
  • File system support Random I/O. (Controlled by the read / write block size)
  • M-Stream requires no user interaction. It works automatically when conditions allow.