Splunk Enterprise

Archive Raw Received Data?

Bob_Diepenbrock
Engager

I am new to administrating Splunk Enterprise Server. I'm guessing the answer is obvious to some, but I'm getting confused trying to figure out a solution from the documentation.

We are using Splunk Enterprise Server v 9.2.1 stand-alone on an isolated network. We primarily collect and report on multiple systems' audit logging.  The server is set up and I can see ingested logs arriving and create reports on the data. But I need one more thing.

I must archive all the original data exactly as it is received on the TCP receiver and copy it to offline storage for safe keeping.

I need to be able to re-ingest the raw data at some future date, but that seems pretty straightforward.

How can I do this?  Is there some way I can grab the data being received on my TCP port listener in RAW form or some magic I need to do with some indexer or forwarder->Receiver string?  

I'm sure I'm not the first person to need this... How do others accomplish this stuff?

Thank You!

Labels (1)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

Splunk has designed to archive data from buckets, not from collection phase. If you are running on AWS then you could try to archive data before it’s indexed by ingest action, but I think that you are running it in on premise? 
The best option is use archive script in indexes.conf for archiving buckets. If this is not an option to you, then you have two option.

  1. setup some props + transforms.conf files to duplicate that data e.g. to syslog server and use it to store and archive it. But as splunk use UDP to send syslog feed, you will lose some events time by time.
  2. Use some other tool to collect and archive those events and send those also to splunk by that tool

r. Ismo

0 Karma

Bob_Diepenbrock
Engager

Good afternoon,

Yes, I am most assuredly not on AWS, but running an on-premise solution.  This means that I cannot archive off to S3 buckets, which are an AWS thing (for the most part).

For your suggested solutions, can you point me towards the relevant documentation or add some additional details that might get me started on the right path?   My gut reaction is that option 1 is likely the solution of choice.  The Splunk configuration "props + transforms.conf" part has me scratching my head a bit, though I think I got it from the rsyslog part onward.

Thanks!

 

 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...