Deployment Architecture

How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions

Richy_s
Path Finder

We are utilizing Splunk Ingest actions to copy data to an S3 bucket. After reviewing various articles and conducting some tests, I've successfully forwarded data to the S3 bucket, where it's currently being stored with the Sourcetype name. However, there's a requirement to store these logs using the hostname instead of the Sourcetype for improved visibility and operational efficiency. Although there isn't a direct method to accomplish this through the Ingest actions GUI, I believe it can be achieved using props and transforms. Can someone assist me with this?

Tags (1)
0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@Richy_s- There is currently no option for it I think.

But you can suggest Splunk team to include it for future release of Splunk at https://ideas.splunk.com/

 

I hope this helps!!! Kindly upvote if it does!!!

0 Karma

Richy_s
Path Finder

Hi @VatsalJagani 

 

Yes I raised a case with Splunk support and they confirm they do not have such capability in place and I advised the to add it to their future enhancements list.  I hope this will be considered.  Appreciate your response.

isoutamo
SplunkTrust
SplunkTrust

If I recall correctly there was same questions some time ago, but I cannot found it now.

Anyhow the answer was same on that time too.

Maybe you can use the next solutions as a work around?

Is it possible that you will change sourcetype to be e.g. <host>:<original sourcetype> for those events which you are forwarding to AWS's S3 buckets? In that way you full will your requirements to store those based on hostname?

0 Karma

Richy_s
Path Finder

Hi @isoutamo 

 

I am currently using Splunk ingest actions feature to route the logs to S3 bucket and it doesn't have the capability to include <host>:<original sourcetype> for the events. 

Thank you for taking time to reply to my query.

0 Karma
Get Updates on the Splunk Community!

Detecting Brute Force Account Takeover Fraud with Splunk

This article is the second in a three-part series exploring advanced fraud detection techniques using Splunk. ...

Buttercup Games: Further Dashboarding Techniques (Part 9)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Buttercup Games: Further Dashboarding Techniques (Part 8)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...