Getting Data In

Splunk Forwarder migration

mehmettecer
Explorer

Hi guys,

Here is my issue:
I have 2 rsyslog servers that are in production in redundant setup. Other servers forward same logs to these servers at the same time. Due to UDP drops, log files are slightly different on both servers.

I have Forwarder setup on only one of them, lets call it Server A, now I need to retire Server A and replace it with Server B.

My question here is, when Server B comes up, will Splunk start indexing all the log files as if it was a new server ? If yes, how can I avoid duplicate indexed data in ? Is there are Splunk best practice for such situation ? I know splunk forwarder remembers where it is was left while monitoring files, is there a way to transfer that monitoring offset data to the new server so once Server B comes up it starts monitoring from where it left off ?

Thanks

Tags (1)
0 Karma

jasonnadeau
Explorer

Can you expand on your rsyslog configuration a little bit? My rsyslog creates a directory hierarchy based upon the source IP address for the log message, then creates a new log file per day for each unique IP address.

+/syslog
+-/192.168.1.1
---/2011-10-04.log
+-/192.168.1.10
---/2011-10-04.log
---/2011-10-03.log

If you need to replace Server A with Server B on a scheduled date and time you may want to move the older data out of "view" of splunk monitored directories just before you fire up Splunk on Server B, then you can shutdown Splunk on Server A. At that point the log data will be ingested via Server B, and once you are satisfied you are getting all the required logs you can decommission Server A. If things appear to be going left you can always turn Splunk back on server A and it will pickup logs from where it left off.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...