Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
m-stream/usage/notes [2020_03_19 10:44] – dan | m-stream:usage:notes [2024_02_09 18:50] – steven | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | # M-Stream | + | # M-Stream |
- | Last updated on March 19, 2020 | + | |
- | M-Stream is a File Fabric feature that speeds up the uploading, downloading and copying of very big files. Large files are split into pieces, sent in parallel over multiple streams, and reassembled back into a contiguous file or object at the target. In this way M-Stream maximizes throughput over available network bandwidth, minimizes network latency, and increases resiliency, particularly for wide area networks. | + | Last updated on Feb 9, 2024 |
- | M-Stream is required to be activated by license key for the enterprise on-premises File Fabric. It is not available for the Software-as-a-Service hosted solution. | + | M-Stream is an Access Anywhere feature that speeds up the uploading, downloading and copying of very big files. Large files are split into pieces, sent in parallel over multiple streams, and reassembled back into a contiguous file or object at the target. In this way M-Stream maximizes throughput over available |
## When is M-Stream Used | ## When is M-Stream Used | ||
- | M-Stream | + | M-Stream |
* CIFS/SMB | * CIFS/SMB | ||
- | * Swift API (SwiftStack, | + | * Swift API (SwiftStack, |
- | * S3 API (certified on AWS S3, Cloudian) | + | * S3 API (certified on AWS S3, Cloudian, IBM COS) |
* Azure Blob | * Azure Blob | ||
- | * Filesystem (FS, Local block storage, StorNext, Ceph FS and many others) | + | * Filesystem (FS, Local block storage, StorNext, Ceph FS and many others) |
- | M-Stream will also accelerate | + | M-Stream will also accelerate |
- | ## Enabling | + | M-Stream |
- | To enable | + | ## M-Stream for Upload |
+ | |||
+ | M-Stream streams parts in parallel directly to storage providers. M-Stream will not be used when an upload is performed outside Access Anywhere, when the file size is not known, or when the file content will be encrypted. | ||
+ | |||
+ | For example when: | ||
+ | |||
+ | * Direct upload is used (when both client and provider have ' | ||
+ | * Chunking is enabled (uncommon) | ||
+ | * File uploaded through FTP (as the file size is not provided) | ||
+ | * Encryption is enabled | ||
+ | * Antivirus scanning is enabled. | ||
+ | |||
+ | ## Enabling M-Stream | ||
+ | |||
+ | To enable M-Stream | ||
### From Appladmin Account | ### From Appladmin Account | ||
- | + | ||
Go to Settings >> Site Functionality, | Go to Settings >> Site Functionality, | ||
- | * M-Stream minimum file size | + | * M-Stream minimum file size: |
- | * If set to 0, M-Stream will be disabled) | + | * If set to 0, M-Stream will be disabled |
* If set too small, then too much overhead will be added, adversely affecting the performance. | * If set too small, then too much overhead will be added, adversely affecting the performance. | ||
- | * If set too large, then full potential of M-Stream will not be utilized | + | * If set too large, then full potential of M-Stream will not be utilized |
- | * Default multipart upload size (This only applies to S3 based storage and needs to be between 5MB and 4GB) | + | * Default multipart upload size (This only applies to S3 based storage and needs to be between 5MB and 4GB) |
- | * Ranged read block size for FS and HPSS providers: | + | * Ranged read block size for FS and HPSS providers: This only applies to FS or HPSS providers and specifies the block size for reading data. We recommend |
* Write block size for FS and HPSS providers: (Same as point 4, but controls writing to FS or HPSS) | * Write block size for FS and HPSS providers: (Same as point 4, but controls writing to FS or HPSS) | ||
- | * M-Stream number of threads | + | * M-Stream number of threads: |
* Should be set to 10 or (no. of web node vCPUs - 2), whichever is lower | * Should be set to 10 or (no. of web node vCPUs - 2), whichever is lower | ||
* Enable Direct Upload: Should be set to Off | * Enable Direct Upload: Should be set to Off | ||
- | |||
### From Team Admin Account | ### From Team Admin Account | ||
Line 49: | Line 61: | ||
* Disable Encryption. (Accelerated transfer will not be used for files encrypted with SME encryption. SSE encryption or other storage encryption allow accelerated transfer.) | * Disable Encryption. (Accelerated transfer will not be used for files encrypted with SME encryption. SSE encryption or other storage encryption allow accelerated transfer.) | ||
+ | |||
+ | Go to Policies >> Security: | ||
+ | |||
+ | * " | ||
## Enabling For Windows Tools | ## Enabling For Windows Tools | ||
Line 54: | Line 70: | ||
From Cloud Tools >> Control Panel >> Bandwidth: | From Cloud Tools >> Control Panel >> Bandwidth: | ||
- | * Enable Allow multistream downloads | + | * Enable Allow multistream downloads |
- | * Pick minimum file size (100 - 4096 MB) | + | * Pick minimum file size (100 - 4096 MB) |
- | * Pick up to 4 streams of data (Max 1 per CPU) | + | * Pick up to 4 streams of data (Max 1 per CPU) |
- | * Determine if Cloud Drive should also use M-Stream | + | * Determine if Cloud Drive should also use M-Stream |
## Notes | ## Notes | ||
- | * Provider must support MPU/DLO and range read. | + | * Provider must support MPU/DLO and range read. |
* File system must support Random I/O. (Controlled by the read / write block size) | * File system must support Random I/O. (Controlled by the read / write block size) | ||
* M-Stream requires no user interaction. It works automatically when conditions allow. | * M-Stream requires no user interaction. It works automatically when conditions allow. | ||
+ | * If a file is being uploaded using M-Stream then MD5 of the file will not be calculated, even if the feature " |