Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
s3compatibleapi [2018_01_30 21:43] – stevens3compatibleapi [2018_08_24 23:26] – steven
Line 1: Line 1:
-====== Amazon S3 compatible API ======+# Amazon S3 compatible API
  
-We provide support for another API that is becoming ubiquitous. In this case it is the Amazon S3 API. This is already used in solutions such as Eucalyptus Walrus, and we have now also added it as entry point for **any files** accessed through a SMEStorage Account. This means you can use the S3 compatible API to access your files on Google Docs, or Google Storage, or RackSpace Cloud Files, or Microsoft OneDrive, or any of the many other Clouds that we support.+We provide support for the Amazon S3 API. This means you can use the S3 compatible object storage API to access files on Google Docs, RackSpace Cloud Files, or Microsoft OneDrive, or any of the many other clouds that we support.
  
-For those interested, moving from a file based structure to a flat file object structure was achieved using the [[http://en.wikipedia.org/wiki/Depth-first_search|depth first algorithm]] within the implementation mapping to our [[http://smestorage.com/?p=static&page=for_developers|API]].+The S3 API is available on our SaaS services and on the appliance.
  
-Below is a practical implementation of using the S3 API.+It can be used by may tools including:
  
-===== 1 Setting up an S3 Tool to work with SMEStorage =====+ * [[https://aws.amazon.com/cli/|awscli]] - Allowing command line access to cloud files 
 + * [[https://rclone.org/|rclone]] - Useful for migrating lots of data 
 + * [[standalone/explorer|]] - Desktop applications for users to easily access S3-compatible and other object storage 
 + * CloudBerry 
 + * Cyberduck
  
 +## Authentication
  
 +To use the S3 API you will need an:
  
-{{:/s3compatibleapi:steps:sme_1_setting_up_an_s3_too.png}}+ 1. Endpoint 
 + 1. Access Key ID 
 + 1. Secret Access Key
  
-Lets look at a practical implementation of the S3 API using a tool that supports S3 compatible endpoints, CloudBerry explorer. We'll use this to access our Google Docs / Drive, although we could have chosen DropBox, or any other number of clouds to access.+### 1. Endpoint
  
 +For our SaaS services use one of the following S3 API endpoints:
  
-===== 2 Creating a New Account =====+ * http://s3.storagemadeeasy.com (US Server) 
 + * https://s3.storagemadeeasy.com (US Server) 
 + * http://s3eu.storagemadeeasy.com (EU Server) 
 + * https://s3eu.storagemadeeasy.com (EU Server)
  
-{{:/s3compatibleapi:steps:sme_2_creating_a_new_accou.png}}+Naturally http endpoints are not secure and should not be used for production data.
  
-Choose to add an account and fill in the endpoint as either:+For appliances contact your administrator for the "Cloud S3 Domain Name".
  
-€˜**//s3.storagemadeeasy.com//**’ (US Server) or '**//s3eu.storagemadeeasy.com//**' (EU Server)+### 2. Access Key ID
  
-and the QueryID should be set as the SME username. You can obtain the API key from logging into SME going to **//My Account//** from the sidebar and copying the API key from where it says **//API secret Key//**+The Access Key ID is your SME or File Fabric username.
  
 +### 3. Secret Access Key 
  
-===== 3 Confirm Connection =====+You can obtain the API secret key by logging into the File Fabric going to **My Account** from the sidebar and copying the key from where it says **API secret key**.
  
-{{:/s3compatibleapi:steps:sme_3_confirm_connection.png}} +## Usefulness
- +
-Now you can choose to access the service using the CloudBerry explorer: +
- +
-===== 4 Accessing Service ===== +
- +
-{{:/s3compatibleapi:steps:sme_4_accessing_service.png}} +
- +
-You can can now browse, download, upload files directly from your SME Clouds which is happening over the S3 API. +
- +
-===== 5 Working with Clouds ===== +
- +
-{{:/s3compatibleapi:steps:sme_5_working_with_clouds.png}} +
- +
-This screenshot shows the root view of the Google Docs folder being access via CloudBerry using the S3 compatible API. Accessing different Clouds is just as easy as changing directories for Clouds that are mapped to an account. +
- +
-===== 6 Usefulness ===== +
- +
-{{:/s3compatibleapi:steps:sme_6_usefulness.png}}+
  
 The S3 API can be useful in a number of scenarios. For example: The S3 API can be useful in a number of scenarios. For example:
Line 54: Line 49:
   * Using S3 Tools and scripts to work with Clouds other than S3   * Using S3 Tools and scripts to work with Clouds other than S3
  
-===== 7 Useful Apps =====+## Restrictions
  
-{{:/s3compatibleapi:steps:sme_7_useful_apps.png}}+SME S3 API does not support multi-part upload.
  
-Useful Apps for interacting with SME over the Amazon S3 API are: +Certain S3 client tools may need you to change a setting to turn off multi-part upload.
-  * [[http://www.cloudberrylab.com/free-amazon-s3-explorer-cloudfront-IAM.aspx|CloudBerry]] (Windows)\\ [[http://cyberduck.ch|Cyberduck]] (Windows /Mac)\\ [[http://storagemadeeasy.com/wiki/cloudmigration/|SME Cloud Migration Assistant]] (Windows)+
  
 +For example you need to do this with Cyberduck.Multipart uploads can be disabled by setting the hidden option s3.upload.multipart to false..   See https://trac.cyberduck.ch/wiki/help/en/howto/s3 for further information.
  
-===== 8 Note! =====+## Implementation
  
-{{:/s3compatibleapi:steps:sme_8_note.png}} +SME supports the following Amazon S3 API Requests:
- +
-SME S3 API does not support multi-part upload (as it makes no sense   as we interacting with back end clouds that do not support it).  +
- +
-Certain S3 client tools may need you to change a setting to turn off multi-part upload. +
- +
-For example you need to do this with Cyberduck.Multipart uploads can be disabled by setting the hidden option s3.upload.multipart to false..   See https://trac.cyberduck.ch/wiki/help/en/howto/s3 for further information.+
  
-SME supports the following Amazon S3 API Requests:\\ \\ + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUT.html|PUT request]]** to create a bucket  
-**PUT request** to create a bucket http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUT.html\\ \\ **GET bucket location** http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGETlocation.html\\ \\ **DELETE delete bucket** http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketDELETE.html\\ \\ **GET** to list buckets http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTServiceGET.html\\ \\ **GET** to list contents of a bucket (objects list) http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGET.html\\ \\ **PUT object** to upload new file http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectPUT.html\\ \\ **DELETE** to delete object from bucket http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectDELETE.html\\ \\ **PUT with header** x-amz-copy-source to copy object http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectCOPY.html\\ \\ **HEAD** to get object metadata http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectHEAD.html+ * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGETlocation.html|GET bucket location]]** 
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketDELETE.html|DELETE bucket]]** 
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTServiceGET.html|GET]]** to list buckets  
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGET.html|GET]]** to list contents of a bucket (list objects) 
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectPUT.html|PUT object]]** to upload new file 
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectDELETE.html|DELETE]]** to delete object from bucket  
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectCOPY.html|PUT with header]]** x-amz-copy-source to copy object  
 + * **[[http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectHEAD.html|HEAD]]** to get object metadata
  
  
  
© Copyright 2024 Nasuni.