Getting Data In

Where do I start to look for Forwarder issues?

justin_deutsch
Explorer

I have a a number of light weight forwarders pointing to a single heavy forwarder point which in turn points to a single search head/indexer. The heavy forwarder has DB Connect installed.

Versions
Light Weight Forwarders: 6.1.0
Heavy Forwarder: 6.0.1
Search Head/Indexer: 6.0.1
(yes I know that we should upgrade the heavy forwader and search head/indexer)

Up until 31 Oct everything was working fine and then some events have stopped being forwarded to the indexer. For example, I can see that the database I am monitoring is generating the batch files on the forwarder and they are being processed in the logs on the server, but I can't see them when I search the index.

I would like to know is where I should start looking to see why the events aren't being forward. Any help would be appreciated.

Tags (2)
1 Solution

fabiocaldas
Contributor

You can start by reading splunkd.log files on your forwarders and indexer, it's can be found at $SPLUNK_HOME$/var/log/splunk folder. This log use be very helpful.

In my case I use a pair of applications to help me identify problems. The first app is s.o.s (Splunk on Splunk) installed on my search head and I have s.o.s add-on installed in all my Splunk Instance provinding an overall view about all machines. I also use another app to help me understand better my forwarders, it's called Forwarder Health.

View solution in original post

fabiocaldas
Contributor

You can start by reading splunkd.log files on your forwarders and indexer, it's can be found at $SPLUNK_HOME$/var/log/splunk folder. This log use be very helpful.

In my case I use a pair of applications to help me identify problems. The first app is s.o.s (Splunk on Splunk) installed on my search head and I have s.o.s add-on installed in all my Splunk Instance provinding an overall view about all machines. I also use another app to help me understand better my forwarders, it's called Forwarder Health.

View solution in original post

.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!