# Jibe and AWS Lambda

For general information on Jibe see [[/jibe]].

## Lambda Architecture

Jibe is able to run as a [AWS Lambda](https://docs.aws.amazon.com/lambda/) function processing S3 event notifications. The steps are:

 1. **S3 API** - External applications call Amazon S3 APIs to create and delete objects.
 2. **S3 Events** - Amazon S3 buckets send [S3 Event notifications](https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html) to an [Amazon SQS](https://docs.aws.amazon.com/sqs/) queue. 
 3. **Message** - Our AWS Lambda function receives and unpacks S3 notifications.
 4. **Request Sync** - The function asks the Access Anywhere Server to refresh that object via th the REST API `syncOnProviderEvent`
 5. **Object Sync** - The Access Anywhere Server verifies the object status on S3 and updates metadata.


#### Jibe Sync with AWS Lambda
{{ /jibe:aws-arch-single:aws-arch-single-region.png?600 |}}


S3 Events can only be sent to queues in the same region. For buckets in the other region send events through to an 
 [Amazon Simple Notification Service](https://docs.aws.amazon.com/sns) topic in that region which in turn forwards to the queue in the target region.

## Installation

The AWS Lambda function, SQS queues, buckets and if needed, SNS topics can be configured through the [AWS Management Console](https://console.aws.amazon.com/).

## Manual Installation

These are the steps to configure the Lambda function manually. They can also be useful in troubleshooting any issues with the automated setup.

### Set up a Queue

Create a "Standard" SQS Queue in the region where your S3 buckets are located.

1. Navigate to the [SQS Management Console](https://console.aws.amazon.com/sqs/home).

2. Change regions to where (most of) your S3 buckets are located. 

3. Create a **Standard Queue** with the Name `jibe-source-activity`.

### Queue Permissions

Change the Queue Access Policy to allow the S3 service to send events:

    {
        "Version": "2012-10-17",
        "Id": "example-ID",
        "Statement": [
            {
                "Sid": "example-statement-ID",
                "Effect": "Allow",
                "Principal": {
                    "Service": "s3.amazonaws.com"
                },
                "Action": [
                    "SQS:SendMessage"
                ],
                "Resource": "arn:aws:sqs:us-east-1:410679986536:jibe-source-activity",
                "Condition": {
                    "ArnLike": {
                        "aws:SourceArn": "arn:aws:s3:*:*:*"
                    },
                    "StringEquals": {
                        "aws:SourceAccount": "410679986536"
                    }
                }
            }
        ]
    }


### Send S3 Events to SQS Queue

Select a bucket to configure from the [S3 Management Console](https://s3.console.aws.amazon.com/s3/buckets).

1. Go to the **Properties** tab
2. Scroll to **Event notifications**
3. Select **Add notification**
4. Change **Send to** to **SQS Queue**
5. Select your queue from the drop-down

If you get this error you'll need to add permissions to your SQS Queue.

    Unable to validate the following destination configurations. Permissions on the destination queue do not allow S3 to publish notifications from this bucket. (arn:aws:sqs:us-east-1:1234567890:bucket-activity)

### Send S3 Events to SQS Topic

You can't send S3 events to a queue in another region. Instead, create an SNS topic.

Select a bucket to configure from the [S3 Management Console](https://s3.console.aws.amazon.com/s3/buckets).

1. Go to the **Properties** tab
2. Scroll to **Advanced settings** and select **Events**
3. Select **Add notification**
4. Change **Send to** to **SNS Topic**
5. Select your topic from the drop-down

If you get this error you'll need to add permissions to your SNS Topic.

    Unable to validate the following destination configurations. Permissions on the destination topic do not allow S3 to publish notifications from this bucket. (arn:aws:sns:us-east-1:1234567890:bucket-activity)

### Creating a Lambda Function

Create a config.json file with an Access Anywhere Server endpoint and credentials. For example,

    {
        "apiendpoint" : "https://storagemadeeasy.com",
        "login" : "adminuser@example.com",
        "password" : "Passsword123"
    }

Create a zip of the script and config file:

    zip sme_jibe_sync.zip jibe_core.py config.json

Navigate to the [AWS Lambda Management Console](https://console.aws.amazon.com/lambda/home).

 1. Choose **Create function**.

 2. Choose **Author from scratch**.

 3. Basic information:
    * Function name: sme_jibe_sync 
    * Runtime: python3.8

 4. Permissions:

    * Create a new role and attach these permission policies
        * AWSLambdaRole
        * AWSLambdaSQSQueueExecutionRole

 5. Submit (which creates function)

 6. Verify you are in the Designer view with the Lambda function selected.
 
    * Scroll down to **Function code**.
    * Change Handler to `sme_jibe_sync.lambda_handler`
    * Change Code entry type to Upload a .zip file.
    * Select Upload and locate sme_jibe_sync.zip.
    * Change timeout from 3 to 60 seconds
    * Set throttle to 10
    * Select **Save**, you'll stay on the same page.

 7. Select **Layers**, still within the Designer tab.

 8. Select **Create layer** 

    Add third-party libraries. These packages courtesy of  to [Keith's Layers (Klayers)](https://github.com/keithrozario/Klayers).

        arn:aws:lambda:us-east-1:770693421928:layer:Klayers-python38-requests:5

### Test Lambda Function

Create test using "Amazon S3 Put" template.

### Add Queue

From the [Lambda Console](https://console.aws.amazon.com/lambda) select the function and view the Configuration.

Add a trigger from the Queue:

1. Select **Add trigger**
2. Select **SQS Queue**
3. Choose an SQS queue to read messages from.
4. Enable trigger
5. Then **Add**

The SQS queue will now be visible as a trigger.