Splunk App for AWS: How to create an alert to trigger every time the amount of security group rules changes?

[r1tual]

I am trying to get Splunk Enterprise to alert on changes to a security group with information that is being fed in from an AWS account using the Splunk App for AWS. I have the information being presented into Splunk with the following search:

aws-description-resource((aws_account_id="*"),  (region="**") , "ec2_security_groups")` | spath path=rules{} output=ingress  | spath path=rules_egress{} output=egress  | stats count(ingress) as ingress, count(egress) as egress   | eval count=ingress+egress   | nadefault count

This results in a data set that looks like this:

ingress     egress      count
302         302         604

Now the problem with this is I am trying to get an alert generated every time the amount of security group rules changes. This information is then being fed into the Slack Notification App in order to produce a Splunk Bot post in a particular Channel. So far I have not been able to do this with any combination of custom alerts and searches. Any help with this would be greatly appreciated as this is a necessary integration for our organization.

[czhang_splunk · Accepted solution]

hi r1tual, could you please double check whether you have configured AWS Config in App? If so, there should be some events with sourcetype="aws:config:notification". Every time you create/delete/modify the resource in AWS, there will be a aws:config:notification event logged the diff of your modification.

[czhang_splunk]

Is that OK to use AWS Config Notification to monitor the SG changes? Basically you can use sourcetype="aws:config:notification" "configurationItem.resourceType"="AWS::EC2::SecurityGroup" to detect whether there is any update of SGs.

[r1tual]

I tried doing this by using aws config like you said but that does not return any data with the search query that you gave. Still not sure how to do this.

[czhang_splunk]

hi r1tual, could you please double check whether you have configured AWS Config in App? If so, there should be some events with sourcetype="aws:config:notification". Every time you create/delete/modify the resource in AWS, there will be a aws:config:notification event logged the diff of your modification.

[r1tual]

Thank you very much this is exactly what I was looking for. Now I am just working on outputting the results to slack channel with an alert and I will be good to go.

[czhang_splunk]

Would you please mark this as the answer? Thanks!