Getting Data In

Archive Splunk buckets to AWS S3?

sarnagar
Contributor

How can I send splunk cold buckets to S3?
We have our on-premises splunk and send Splunk data to S3 for longer storage.

I came across this Hadoop Data Roll that sends the splunk data to S3A filesystem. This looks something to deal with Hadoop+S3 , which Im not quite aware of. I'm very new to AWS. I thought Splunk can send data directly to S3 for archival. Isnt that possible?

The document says to provide some provider parameters. Can someone please elaborate on this? Does this mean I need to have Hadoop installed on S3?

link text

Tags (3)

ByteFlinger
Engager

It seems difficult to find documentation on the coldToFrozen script API.

Any chance you can give an example on how one would setup an indexes.conf with a couple of indexes using the cold2frozen.py?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...