How to Receive S3 notifications in a Slack Channel

February 5, 2023 3 min read

Jay Ding

One of the most common scenarios when working with Amazon S3 is to perform specific actions whenever changes (e.g., file uploaded or file deleted) happen in the S3 buckets.

This article shows how to automatically get notified in a Slack channel whenever changes happen in your S3 buckets.

To achieve that, we need one program to receive S3 events and transform them into custom notifications and we also need another program to receive transformed notifications and send them to our Slack channel.

Luckily, Vanus has already provided many out-of-the-box Connectors that enable you to integrate with popular services or applications without writing any codes.

Here is how we can do this within 5 mins.

Go to Vanus Playground (an online K8s playground).
Click “Continue with GitHub” to prepare an interactive K8s environment(Use your GitHub account to sign in).

Wait for preparing the K8s environment (usually less than 1 min). The terminal is ready when you see something like this:

Install Vanus.

kubectl apply -f https://download.linkall.com/all-in-one/v0.6.0.yml

Install vsctl (the command line tool).

curl -O https://download.linkall.com/vsctl/latest/linux-amd64/vsctl
chmod ug+x vsctl
mv vsctl /usr/local/bin

Set the endpoint for vsctl.

export VANUS_GATEWAY=192.168.49.2:30001

Create an Eventbus to store your events.

vsctl eventbus create --name s3-slack-scenario

Set the config file for the S3 Source.

mkdir -p s3-slack-scenario/s3-source;
mkdir -p s3-slack-scenario/slack-sink;
cd s3-slack-scenario/s3-source;

cat << EOF > config.yml
target: http://192.168.49.2:30002/gateway/s3-slack-scenario
aws:
  access_key_id: AKIAIOSFODNN7EXAMPLE
  secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
s3_bucket_arn: "arn:aws:s3:::<buckeName>"
s3_events: ["s3:ObjectCreated:*","s3:ObjectRemoved:*"]
region: "us-west-2"
EOF

NOTE: Remember to replace values of access_key_id, secret_access_key, s3_bucket_arn and region with yours. You also need to ensure the IAM user behind the access_key_id and secret_access_key have certain permissions to allow S3 Source work properly.

Run the S3 Source.

nohup docker run --rm --network=host \
  -v ${PWD}:/vanus-connect/config \
  --name source-aws-s3 public.ecr.aws/vanus/connector/source-aws-s3 > s3.log &

Create a Slack app.

Create a new app on Slack.

Select from scratch to create an app .

Pick a workspace to develop your app in.

Click on permissions in the central menu.

Click Add an Oauth Scope and add chat:write and chat:write.public to the scope.

Install this oauth token to your workspace:

Copy the Bot User OAuth for later usage.

Get the slack-sink yaml file and edit the configuration in it.

cd ../slack-sink;
curl -O https://scenario-utils.s3.us-west-2.amazonaws.com/slack-sink.yaml

vi slack-sink.yaml

NOTE: Remember to replace values of default, app_name, token and default_channel with yours.

The token value should be the Bot User OAuth in previous step.

The default and app name MUST be the same.

Run the Slack Sink.

kubectl apply -f slack-sink.yaml

Create a subscription

vsctl subscription create --name s3-slack-subscription \
  --eventbus s3-slack-scenario \
  --sink 'http://sink-slack:8080' \
  --filters '[
    {
      "exact": {
          "type":"com.amazonaws.s3.ObjectCreated:Put"
      }
    }
  ]' \
  --transformer '{
      "define": {
        "subject" : "$.subject",
        "bucket": "$.data.bucket.name"
      },
      "template": "{\"subject\": \"AWS S3 Notifications\",\"message\":\"Hello, a new file <subject> has been uploaded to your bucket <bucket>.\"}"
    }'

Upload a file to your bucket.

Finally, you can see a notification in your Slack channel.

Now, you’ve successfully received S3 notifications in your Slack channel. Actually, you can integrate with any external services with Vanus and its’ filter and transformer. Learn more about Vanus here.

Vanus is an open-source, cloud-native, Serverless message queue for building EDA applications with Ease.

Try it and have fun with Vanus, guys.