All Apps and Add-ons

Monitor AWS Service Status using Splunk and Syndication Input

marcellodesales
Path Finder

Problem

  • Dependency on AWS Services status

If you depend on Amazon AWS service to operate, you need to keep a close eye on the status of their services. Amazon uses the website http://status.aws.amazon.com/, which provides links to RSS feeds to specific services in specific regions.

  • Potential Errors

Our service uses S3, CloudFront, and other services to operate. We'd like to be informed on any service that might go down during hours of operations, and automate what we should do in case something goes wrong.

  • Splunk Logging

We use Splunk for Logging all of our services.

Requirement

For instance, if errors occurs in the application while writing to S3, we'd like to know if that was caused by a potential outage in AWS.

  • How to monitor the Status RSS feed in Splunk?
  • Is there an HTTP client for that? A background service?
1 Solution

marcellodesales
Path Finder

Solution

How

  1. Ask your Splunk team to install the app "Syndication Input" on the environments you need.

  2. After that, just collect each of the RSS feeds needed and add them to the Settings -> Data Input -> Syndication Feed. Take all the URLs from the Amazon Status RSS feeds and use them as Splunk Data Input, filling out the form with certain interval:

alt text

When you are finished, the Syndication App has the following:

alt text

  1. Use the search for the errors when the occur, adjusting the “since” date so that you can create an alert for the results. I added a day in the past just for display purpose.
  • since should be some start day you will start monitoring AWS. This helps the query to result in any new event when Amazon publishes new errors captured from the text Informational message:.
    • The query should not return anything new because the since will not return any date.
  • Since the token RESOLVED is appended to a new RSS feed item, we exclude them from the alerts.

.

sourcetype=syndication "Informational message:" NOT "RESOLVED" 
   | eval since=strptime("2010-08-01", "%Y-%m-%d") 
   | eval date=strptime(published_parsed, "%Y-%m-%dT%H:%M:%SZ") 
   | rex field=summary_detail_base "rss\/(?<aws_object>.*).rss$" 
   | where date > since 
   | table aws_object, published_parsed, id, title, summary 
   | sort -published_parsed

alt text

  1. Create an Alert with the Query. For instance, to send an email:

alt text

View solution in original post

marcellodesales
Path Finder
0 Karma

marcellodesales
Path Finder

Solution

How

  1. Ask your Splunk team to install the app "Syndication Input" on the environments you need.

  2. After that, just collect each of the RSS feeds needed and add them to the Settings -> Data Input -> Syndication Feed. Take all the URLs from the Amazon Status RSS feeds and use them as Splunk Data Input, filling out the form with certain interval:

alt text

When you are finished, the Syndication App has the following:

alt text

  1. Use the search for the errors when the occur, adjusting the “since” date so that you can create an alert for the results. I added a day in the past just for display purpose.
  • since should be some start day you will start monitoring AWS. This helps the query to result in any new event when Amazon publishes new errors captured from the text Informational message:.
    • The query should not return anything new because the since will not return any date.
  • Since the token RESOLVED is appended to a new RSS feed item, we exclude them from the alerts.

.

sourcetype=syndication "Informational message:" NOT "RESOLVED" 
   | eval since=strptime("2010-08-01", "%Y-%m-%d") 
   | eval date=strptime(published_parsed, "%Y-%m-%dT%H:%M:%SZ") 
   | rex field=summary_detail_base "rss\/(?<aws_object>.*).rss$" 
   | where date > since 
   | table aws_object, published_parsed, id, title, summary 
   | sort -published_parsed

alt text

  1. Create an Alert with the Query. For instance, to send an email:

alt text

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...