Differences
This shows you the differences between two versions of the page.
| Next revisionBoth sides next revision | |||
| m-stream/usage/notes [2018_08_08 11:30] – created dan | m-stream/usage/notes [2018_08_08 11:47] – dan | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| # M-Stream Usage Notes | # M-Stream Usage Notes | ||
| - | + | ## When is M-Stream used: | |
| - | + | ||
| - | + | ||
| - | When is M-Stream used: | + | |
| M-Stream will function when moving data between any of the following storage providers provided they support range reads. | M-Stream will function when moving data between any of the following storage providers provided they support range reads. | ||
| - | • CIFS/SMB | + | * CIFS/SMB |
| - | • Swift API (SwiftStack, | + | * Swift API (SwiftStack, |
| - | • S3 API (Amazon, Scality, Cloudian, IBM COS, and many many others) | + | * S3 API (Amazon, Scality, Cloudian, IBM COS, and many many others) |
| - | • Filesystem (FS, Local block storage, StorNext, Ceph FS, and many others) | + | * Filesystem (FS, Local block storage, StorNext, Ceph FS, and many others) |
| M-Stream will also accelerate the downloads to a Windows desktop running the SME Cloud tools version 10.5.10 or later, from one of the above providers. | M-Stream will also accelerate the downloads to a Windows desktop running the SME Cloud tools version 10.5.10 or later, from one of the above providers. | ||
| - | Enabling M-Stream on Enterprise | + | ## Enabling M-Stream on Enterprise |
| To enable M-Stream for SME File Fabric follow the steps below: | To enable M-Stream for SME File Fabric follow the steps below: | ||
| - | From Appladmin Account | + | ## From Appladmin Account |
| - | + | ||
| Go to Settings >> Site Functionality, | Go to Settings >> Site Functionality, | ||
| - | + | * M-Stream minimum file size | |
| - | + | * If set to 0, M-Stream will be disabled) | |
| + | * If set too small, then too much overhead will be added effecting the performance. | ||
| + | * If set too large, then full potential of M-Stream will not be utilized | ||
| + | * Default multipart upload size (This only applies to S3 based storage and needs to be between 5MB and 4GB) | ||
| + | * Ranged read block size for FS and HPSS providers: (This is in 1803.03 and above. It only applies to FS or HPSS providers and specifies the block size for reading data. In most testing 4096 (4K) to (4M) should be used. Must be a base 2 integer) | ||
| + | * Write block size for FS and HPSS providers: (Same as point 4, but controls writing to FS or HPSS) | ||
| - | • M-Stream | + | * M-Stream |
| + | * Should be set to a max of 10 or ( vCPUs - 2) of web node, whichever is lower | ||
| + | * Enable Direct Upload: Should be set to Off | ||
| - | o If set to 0, M-Stream will be disabled) | + | ## From Team Admin Account |
| - | o If set too small, then too much overhead will be added effecting the performance. | + | |
| - | o If set too large, then full potential of M-Stream will not be utilized | + | |
| - | o Default multipart upload size (This only applies to S3 based storage and needs to be between 5MB and 4GB) | + | |
| - | o Ranged read block size for FS and HPSS providers: (This is in 1803.03 and above. It only applies to FS or HPSS providers and specifies the block size for reading data. In most testing 4096 (4K) to (4M) should be used. Must be a base 2 interger) | + | |
| - | o Write block size for FS and HPSS providers: (Same as point 4, but controls writing to FS or HPSS) | + | |
| - | • M-Stream number of threads | + | Go to Dashboard >> Provider Setting |
| + | * Set Chunking " | ||
| + | * Turn MPU/DLO " | ||
| - | o Should | + | Go to Policies >> Encryption |
| - | + | * Disable Encryption. (SME encrypted files cannot | |
| - | • Enable Direct Upload: Should be set to Off | + | |
| - | + | ||
| - | From Team Admin Account | + | |
| - | Go to Dashboard >> Provider Setting | + | ## Enabling For Windows Tools: |
| - | • Set Chunking " | + | |
| - | • Turn MPU/DLO " | + | |
| - | Go to Policies >> Encryption | ||
| - | • Disable Encryption. (SME encrypted files cannot be transmitted in parallel. SSE encryption or other storage encryption is fine) | ||
| - | |||
| - | Enabling For Windows Tools: | ||
| - | |||
| From Cloud Tools >> Control Panel >> Bandwidth | From Cloud Tools >> Control Panel >> Bandwidth | ||
| - | + | * Enable Allow multistream downloads | |
| - | + | * Pick minimum file size (100 - 4096 MB) | |
| - | • Enable Allow multistream downloads | + | * Pick up to 4 streams of data (Max 1 per CPU) |
| - | • Pick minimum file size (100 - 4096 MB) | + | * Determine if Cloud Drive should also use M-Stream |
| - | • Pick up to 4 streams of data (Max 1 per CPU) | + | |
| - | • Determine if Cloud Drive should also use M-Stream | + | |
| - | + | ||
| - | Note: | + | |
| - | + | ||
| - | • Provider support MPU/DLO and range read. | + | |
| - | • File system support Random I/O. (Controlled by the read / write block size) | + | |
| - | • M-Stream requires no user interaction. It just works for Provider to Provider movement of data. | + | |
| + | ## Note: | ||
| + | * Provider support MPU/DLO and range read. | ||
| + | * File system support Random I/O. (Controlled by the read / write block size) | ||
| + | * M-Stream requires no user interaction. It just works for Provider to Provider movement of data. | ||