topic Re: How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions in Deployment Architecture

How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions

Richy_s — Thu, 09 Jan 2025 11:05:29 GMT

We are utilizing Splunk Ingest actions to copy data to an S3 bucket. After reviewing various articles and conducting some tests, I've successfully forwarded data to the S3 bucket, where it's currently being stored with the Sourcetype name. However, there's a requirement to store these logs using the hostname instead of the Sourcetype for improved visibility and operational efficiency. Although there isn't a direct method to accomplish this through the Ingest actions GUI, I believe it can be achieved using props and transforms. Can someone assist me with this?

Re: How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions

VatsalJagani — Thu, 23 Jan 2025 06:22:30 GMT

@Richy_s- There is currently no option for it I think.

But you can suggest Splunk team to include it for future release of Splunk at https://ideas.splunk.com/

I hope this helps!!! Kindly upvote if it does!!!

Re: How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions

isoutamo — Thu, 23 Jan 2025 22:42:29 GMT

If I recall correctly there was same questions some time ago, but I cannot found it now.

Anyhow the answer was same on that time too.

Maybe you can use the next solutions as a work around?

Is it possible that you will change sourcetype to be e.g. <host>:<original sourcetype> for those events which you are forwarding to AWS's S3 buckets? In that way you full will your requirements to store those based on hostname?

Re: How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions

Richy_s — Fri, 24 Jan 2025 13:51:28 GMT

Hi @VatsalJagani

Yes I raised a case with Splunk support and they confirm they do not have such capability in place and I advised the to add it to their future enhancements list. I hope this will be considered. Appreciate your response.

Re: How to store logs to AWS S3 bucket with the hostname using Splunk Ingest actions

Richy_s — Fri, 24 Jan 2025 13:52:56 GMT

Hi @isoutamo

I am currently using Splunk ingest actions feature to route the logs to S3 bucket and it doesn't have the capability to include <host>:<original sourcetype> for the events.

Thank you for taking time to reply to my query.