Installation

will there be loss of data during splunk upgrade process?

remy06
Contributor

Hi,like to clarify more on the upgrade process..

1) During the process, where the splunk indexer will be off for some time,will there be loss of data between that period? I currently have servers using forwarders and udp to send data to our indexer..

2) In these scenario,if I were to switch from a forwarder of different architecture to another(heavy to universal,or universal to heavy),does it re-index all the events or only starting from the latest?

My concern is if there will be loss of data during the indexer upgrade process and if so how can I retain them. Also will there be duplicate events if I am to switch the forwarders type.

Thanks.

Tags (1)
0 Karma
1 Solution

hazekamp
Builder

remy,

Data will definitely not be indexed while the indexer is off. It is the responsibility of the forwarder to cache the data locally or maintain state (in the case of Splunk "monitor://" file monitors). I am confused by the mention of "using forwarders and udp to send data". Are you using forwarders to collect UDP and forward to the indexer? Are the indexers collecting UDP directly?

In general if you are using Splunk forwarders to collect data via "monitor://" inputs, Splunk, in the event the forwarder cannot contact the indexer, will maintain state and pick up where it left off. Other data (i.e. TCP, UDP, or scripted inputs) is maintained by the forwarder based on several output settings. To name a few:

maxQueueSize = [<integer>|<integer>[KB|MB|GB]]
dropEventsOnQueueFull = <integer>
useACK = [true|false]

If you are indexing data directly (on the indexer) via a "udp://" inputs statement, data will be dropped if splunkd is stopped.

Finally, most of these issues are mitigated with the introduction of a mult-indexer environment which allows forwarders to auto load balance. I would also not recommend indexing UDP data directly using Splunk. Instead data could be collected via syslogd and indexed via "monitor://" or collected via a Splunk forwarder and indexed via "splunktcp://".

View solution in original post

hazekamp
Builder

remy,

Data will definitely not be indexed while the indexer is off. It is the responsibility of the forwarder to cache the data locally or maintain state (in the case of Splunk "monitor://" file monitors). I am confused by the mention of "using forwarders and udp to send data". Are you using forwarders to collect UDP and forward to the indexer? Are the indexers collecting UDP directly?

In general if you are using Splunk forwarders to collect data via "monitor://" inputs, Splunk, in the event the forwarder cannot contact the indexer, will maintain state and pick up where it left off. Other data (i.e. TCP, UDP, or scripted inputs) is maintained by the forwarder based on several output settings. To name a few:

maxQueueSize = [<integer>|<integer>[KB|MB|GB]]
dropEventsOnQueueFull = <integer>
useACK = [true|false]

If you are indexing data directly (on the indexer) via a "udp://" inputs statement, data will be dropped if splunkd is stopped.

Finally, most of these issues are mitigated with the introduction of a mult-indexer environment which allows forwarders to auto load balance. I would also not recommend indexing UDP data directly using Splunk. Instead data could be collected via syslogd and indexed via "monitor://" or collected via a Splunk forwarder and indexed via "splunktcp://".

hazekamp
Builder

Got it. Data will be dropped silently for servers w/o Splunk forwarder sending udp syslog directly to an indexer collecting with "udp://" stanza. Consider collecting w/ syslogd on the indexer and then indexing via "monitor://".

0 Karma

remy06
Contributor

I mean that we have certain servers(win) installed with splunk forwarders,as well as other servers(linux) w/o splunk forwarder that is sending syslog via udp.sry for the confusion.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...