Hi there
We use a configuration of splunk light forwarders and a splunk index server. The light forwarders using tcp to write data to the index server. Now we've encountered, some lines of logfiles are missing on index side. Splunk logfiles (on light forwarder and index server) don't show any errors.
An example of lost data is:
Original Logfile:
06.01.2011 05:10:46 INFO Worldcheck import: 6,000 (22/sec)
06.01.2011 05:11:24 INFO Worldcheck import: 7,000 (26/sec)
06.01.2011 05:11:57 INFO Worldcheck import: 8,000 (29/sec)
06.01.2011 05:12:33 INFO Worldcheck import: 9,000 (27/sec)
06.01.2011 05:13:00 INFO Worldcheck import: 10,000 (37/sec)
06.01.2011 05:13:09 INFO Worldcheck import: 10,394 (43/sec)
On side of splunk index server we have for the same time:
06.01.2011 05:10:46 INFO Worldcheck import: 6,000 (22/sec)
06.01.2011 05:11:57 INFO Worldcheck import: 8,000 (29/sec)
06.01.2011 05:13:00 INFO Worldcheck import: 10,000 (37/sec)
06.01.2011 05:13:09 INFO Worldcheck import: 10,394 (43/sec)
As you can see in original logfile there are 6 lines output, on splunk index side only 4 lines output got received. Means 30% loss!
Gives me a bit bad taste, because of some 30 million events a day I'm not able to proof every event got recorded.
Does anyone have an idea how to ensure data from lightforwarder are written on splunk index server for sure?
output.conf on lightforwarder looks like:
[tcpout]
disabled = false
defaultGroup = group1_58499
[tcpout:group1_58499]
disabled = false
server = splunk:58499
[tcpout:RouteSplunkLogs]
disabled = true
server = splunk:58499
Kind regards,
Aaron
... View more