Archive Raw Received Data?

[Bob_Diepenbrock]

I am new to administrating Splunk Enterprise Server. I'm guessing the answer is obvious to some, but I'm getting confused trying to figure out a solution from the documentation.

We are using Splunk Enterprise Server v 9.2.1 stand-alone on an isolated network. We primarily collect and report on multiple systems' audit logging.  The server is set up and I can see ingested logs arriving and create reports on the data. But I need one more thing.

I must archive all the original data exactly as it is received on the TCP receiver and copy it to offline storage for safe keeping.

I need to be able to re-ingest the raw data at some future date, but that seems pretty straightforward.

How can I do this?  Is there some way I can grab the data being received on my TCP port listener in RAW form or some magic I need to do with some indexer or forwarder->Receiver string?  

I'm sure I'm not the first person to need this... How do others accomplish this stuff?

Thank You!

[isoutamo]

Hi

Splunk has designed to archive data from buckets, not from collection phase. If you are running on AWS then you could try to archive data before it’s indexed by ingest action, but I think that you are running it in on premise? 
The best option is use archive script in indexes.conf for archiving buckets. If this is not an option to you, then you have two option.

1. setup some props + transforms.conf files to duplicate that data e.g. to syslog server and use it to store and archive it. But as splunk use UDP to send syslog feed, you will lose some events time by time.
2. Use some other tool to collect and archive those events and send those also to splunk by that tool

r. Ismo

[Bob_Diepenbrock]

Good afternoon,

Yes, I am most assuredly not on AWS, but running an on-premise solution.  This means that I cannot archive off to S3 buckets, which are an AWS thing (for the most part).

For your suggested solutions, can you point me towards the relevant documentation or add some additional details that might get me started on the right path?   My gut reaction is that option 1 is likely the solution of choice.  The Splunk configuration "props + transforms.conf" part has me scratching my head a bit, though I think I got it from the rsyslog part onward.

Thanks!

[isoutamo · Accepted solution]

https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Replicate_a_subse... should helps you.