topic Re: Sending data to AWS S3 from Splunk in Splunk Enterprise in Splunk Enterprise

Sending data to AWS S3 from Splunk in Splunk Enterprise

faisalshani001 — Tue, 16 Feb 2021 07:22:24 GMT

Hi Members, So I am quite new to splunk and I need to send the splunk search results to AWS S3 bucket. I have tried some apps from splunkbase but they are not working. (APP NO 5273 & Event Push by Deductiv).

Can someone guide me here what approach I should follow to make such a pipeline?
(Since we are working on just of POC we cant use the Splunk DSP, I am looking for an open source or free approach with minimal cost) .

Thanks 🙂

Re: Sending data to AWS S3 from Splunk in Splunk Enterprise

alonsocaio — Tue, 16 Feb 2021 12:54:03 GMT

Hi @faisalshani001,

One approach you can try is to export your data using Splunk REST API. Since you need to export the search results, I guess this works for you: Exporting Large Results Sets to CSV | Splunk , and you can export on CSV, JSON or RAW format. Also, I think that using the API is pretty simple, since you can use python requests or event curl to create and export your search.

After saving the results into a file, you can upload it to the S3 bucket.

Re: Sending data to AWS S3 from Splunk in Splunk Enterprise

faisalshani001 — Tue, 16 Feb 2021 13:39:31 GMT

Thanks for replying @alonsocaio . But I need to create an automated pipeline which should send data to AWS S3 automatically. Means when one writes the SPL query on splunk searc & reporting bar the result should be exported to an csv file and send to AWS S3. Any idea how to create this pipeline?