Splunk Enterprise

Archive Raw Received Data?

Bob_Diepenbrock
Engager

I am new to administrating Splunk Enterprise Server. I'm guessing the answer is obvious to some, but I'm getting confused trying to figure out a solution from the documentation.

We are using Splunk Enterprise Server v 9.2.1 stand-alone on an isolated network. We primarily collect and report on multiple systems' audit logging.  The server is set up and I can see ingested logs arriving and create reports on the data. But I need one more thing.

I must archive all the original data exactly as it is received on the TCP receiver and copy it to offline storage for safe keeping.

I need to be able to re-ingest the raw data at some future date, but that seems pretty straightforward.

How can I do this?  Is there some way I can grab the data being received on my TCP port listener in RAW form or some magic I need to do with some indexer or forwarder->Receiver string?  

I'm sure I'm not the first person to need this... How do others accomplish this stuff?

Thank You!

Labels (1)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

Splunk has designed to archive data from buckets, not from collection phase. If you are running on AWS then you could try to archive data before it’s indexed by ingest action, but I think that you are running it in on premise? 
The best option is use archive script in indexes.conf for archiving buckets. If this is not an option to you, then you have two option.

  1. setup some props + transforms.conf files to duplicate that data e.g. to syslog server and use it to store and archive it. But as splunk use UDP to send syslog feed, you will lose some events time by time.
  2. Use some other tool to collect and archive those events and send those also to splunk by that tool

r. Ismo

0 Karma

Bob_Diepenbrock
Engager

Good afternoon,

Yes, I am most assuredly not on AWS, but running an on-premise solution.  This means that I cannot archive off to S3 buckets, which are an AWS thing (for the most part).

For your suggested solutions, can you point me towards the relevant documentation or add some additional details that might get me started on the right path?   My gut reaction is that option 1 is likely the solution of choice.  The Splunk configuration "props + transforms.conf" part has me scratching my head a bit, though I think I got it from the rsyslog part onward.

Thanks!

 

 

0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...