Getting Data In

Splunk Forwarder migration

mehmettecer
Explorer

Hi guys,

Here is my issue:
I have 2 rsyslog servers that are in production in redundant setup. Other servers forward same logs to these servers at the same time. Due to UDP drops, log files are slightly different on both servers.

I have Forwarder setup on only one of them, lets call it Server A, now I need to retire Server A and replace it with Server B.

My question here is, when Server B comes up, will Splunk start indexing all the log files as if it was a new server ? If yes, how can I avoid duplicate indexed data in ? Is there are Splunk best practice for such situation ? I know splunk forwarder remembers where it is was left while monitoring files, is there a way to transfer that monitoring offset data to the new server so once Server B comes up it starts monitoring from where it left off ?

Thanks

Tags (1)
0 Karma

jasonnadeau
Explorer

Can you expand on your rsyslog configuration a little bit? My rsyslog creates a directory hierarchy based upon the source IP address for the log message, then creates a new log file per day for each unique IP address.

+/syslog
+-/192.168.1.1
---/2011-10-04.log
+-/192.168.1.10
---/2011-10-04.log
---/2011-10-03.log

If you need to replace Server A with Server B on a scheduled date and time you may want to move the older data out of "view" of splunk monitored directories just before you fire up Splunk on Server B, then you can shutdown Splunk on Server A. At that point the log data will be ingested via Server B, and once you are satisfied you are getting all the required logs you can decommission Server A. If things appear to be going left you can always turn Splunk back on server A and it will pickup logs from where it left off.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...