Cisco Umbrella Log Management with App: Splunk Add-on for AWS

[kennymann]

Has anyone been able to successfully setup Cisco Umbrella(OpenDNS, Cloud) and the AWS Splunk Add-on?

I have setup Cisco Umbrella using the Amazon S3. The bucket is syncing data according to Cisco Umbrella.
https://support.umbrella.com/hc/en-us/articles/115004685266

My problems start with the configuration of Splunk Addon Inputs error:

S3ResponseError: 403 Forbidden <?xml version="1.0" encoding="UTF-8"?> <Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>

[hrithiktej]

Umbrella suport replied to me the same that due to permission issues it is not possible to ingest the logs using this app. So i followed the work around mentioned in this article

https://support.umbrella.com/hc/en-us/articles/360001388406-Configuring-Splunk-with-a-Cisco-managed-...

and now i have it all working

his article covers the basics of getting Splunk up and running so it is able to consume the logs from your Cisco-managed S3 bucket. You will:

1) Set up your Cisco-managed S3 bucket in your dashboard.

2) Create a cron job to retrieve files from the bucket and store them locally on your server.

3) Configure Splunk to read from a local directory.

[hmorales1]

When you are running this on your Splunk instance is it downloading the csv.gz files?

[hrithiktej]

Yes below are the examples of files getting downloaded, and i am downloading it to my syslog server and then from there the splunk UF reads and forwards to my splunk indexer

-rw-r--r--. 1 root root 628 Mar 15 14:50 2019-03-15-14-40-2593.csv.gz
-rw-r--r--. 1 root root 683 Mar 15 14:50 2019-03-15-14-40-32fa.csv.gz
-rw-r--r--. 1 root root 844 Mar 15 14:50 2019-03-15-14-40-7f7d.csv.gz
-rw-r--r--. 1 root root 930 Mar 15 14:50 2019-03-15-14-40-b3ab.csv.gz
-rw-r--r--. 1 root root 798 Mar 15 14:50 2019-03-15-14-40-dcd8.csv.gz

[hmorales1]

Awesome, I'm getting the same thing!

Question how are you cleaning these folders up? are you running a cron job?

[hrithiktej]

Nope i manually clean up after every 30 days. You can setup a cron job for 30 days cleanup. Because in umbrella bucket we have set the log retention for 30 days, so if clean up anything it downloads complete 30 days of log folders again, so once anything gets older than 30 days i delete them.

[BonMot]

Cisco doesn't allow listing on the root of the bucket, and the Splunk AWS app doesn't allow you to type in a bucket_id. Luckily you can manually enter the bucket id in local/inputs.conf. Here is an example that sets a sourcetype compatible with TA-cisco-umbrella. This assumes your S3 bucket is

s3://YOUR-CISCO-S3-BUCKET/YOUR-INCREDIBLY-LONG-FOLDER-NAME

Everything in ALL-CAPS should be replaced. Don't forget the trailing slash (/) after the key_name

[aws_s3://ANY-NAME-YOU-WANT]
start_by_shell = false
aws_account = YOUR-ACCOUNT-NAME-CREATE-IN-UI
sourcetype = opendns:s3
initial_scan_datetime = default
max_items = 100000
max_retries = 3
polling_interval = 30
interval = 30
recursion_depth = -1
character_set = auto
is_secure = True
host_name = s3.amazonaws.com
ct_blacklist = ^$
ct_excluded_events_index =
key_name = YOUR-INCREDIBLY-LONG-FOLDER-NAME/
bucket_name = YOUR-CISCO-S3-BUCKET

[hrithiktej]

@ David rose

TRY THIS workaround it works
The work around mentioned in this article

https://support.umbrella.com/hc/en-us/articles/360001388406-Configuring-Splunk-with-a-Cisco-managed-...

and now i have it all working

I am downloading logs onto my syslog servers with a cron job to run after every 10mins and from there i have the splunk reading them.

If u need help with cron job let me know

[david_rose]

Thanks. If I can't use the existing aws add on, i might as well create a modular input. Just trying not to invent the wheel if I don't have to.

[david_rose]

I also tried this with no luck. Logs always show 403 Forbidden. settings are verified by working with s3browser.

[hrithiktej]

Hi BonMot,

Thanks for your post but i tried exactly this and it is still not working? Anything else you can suggest?

[kennymann]

After opening a ticket with Cisco Umbrella. This is the answer I received.

"Hello Kenny,
Currently it's not possible to retrieve Cisco managed S3 log files with Splunk. This is due to the restrictive permissions on the bucket itself. The Splunk AWS module expects to be able to list all buckets, which it cannot - as the Cisco Managed buckets are restricted to that singular bucket. However, you can access the logs with the Amazon cli tool. And other third party tools like S3 Browser."

[nickhills]

That’s rubbish!
You may need to change the access policy for your Splunk user so that it has list* and get* permissions on the bucket containing the logs, but there is no reason a bucket in your s3 account can not be retrieved by Splunk with the correct policy settings.

[rajeevlalla]

I think he's referring to the Cisco provided S3 instance i.e. managed/owned by Cisco Umbrella service.