will there be loss of data during splunk upgrade process?

[remy06]

Hi,like to clarify more on the upgrade process..

1) During the process, where the splunk indexer will be off for some time,will there be loss of data between that period? I currently have servers using forwarders and udp to send data to our indexer..

2) In these scenario,if I were to switch from a forwarder of different architecture to another(heavy to universal,or universal to heavy),does it re-index all the events or only starting from the latest?

My concern is if there will be loss of data during the indexer upgrade process and if so how can I retain them. Also will there be duplicate events if I am to switch the forwarders type.

Thanks.

[hazekamp · Accepted solution]

remy,

Data will definitely not be indexed while the indexer is off. It is the responsibility of the forwarder to cache the data locally or maintain state (in the case of Splunk "monitor://" file monitors). I am confused by the mention of "using forwarders and udp to send data". Are you using forwarders to collect UDP and forward to the indexer? Are the indexers collecting UDP directly?

In general if you are using Splunk forwarders to collect data via "monitor://" inputs, Splunk, in the event the forwarder cannot contact the indexer, will maintain state and pick up where it left off. Other data (i.e. TCP, UDP, or scripted inputs) is maintained by the forwarder based on several output settings. To name a few:

maxQueueSize = [<integer>|<integer>[KB|MB|GB]]
dropEventsOnQueueFull = <integer>
useACK = [true|false]

If you are indexing data directly (on the indexer) via a "udp://" inputs statement, data will be dropped if splunkd is stopped.

Finally, most of these issues are mitigated with the introduction of a mult-indexer environment which allows forwarders to auto load balance. I would also not recommend indexing UDP data directly using Splunk. Instead data could be collected via syslogd and indexed via "monitor://" or collected via a Splunk forwarder and indexed via "splunktcp://".

[hazekamp]

Got it. Data will be dropped silently for servers w/o Splunk forwarder sending udp syslog directly to an indexer collecting with "udp://" stanza. Consider collecting w/ syslogd on the indexer and then indexing via "monitor://".

[remy06]

I mean that we have certain servers(win) installed with splunk forwarders,as well as other servers(linux) w/o splunk forwarder that is sending syslog via udp.sry for the confusion.