Getting Data In
Highlighted

Why did a Splunk forwarder stop sending to log packet?

Explorer

It was working fine until 1 month ago.
There was no Splunk forwarder and network configuration change.
No packets from the forwarder to the indexer.
How can I solve the problem?

0 Karma
Highlighted

Re: Why did a Splunk forwarder stop sending to log packet?

Legend

Hi @lifekis,
what's tha last date you received logs?
if you have logs until 31/12/2019 and not more after, probably the problem is the timestamp parsing: if you have a date in format dd/mm/yyyy, Splunk usually read as format mm/dd/yyyy, so it correctly read them after the 13th of each month, but take the wrong format until the 12th of each month.
You can check this problem verifying if you have logs of 7th of January on the 1st of july.

To solve the problem, you have to set the correct TIME_FORMAT in props.conf.

Ciao.
Giuseppe

View solution in original post

0 Karma
Highlighted

Re: Why did a Splunk forwarder stop sending to log packet?

Explorer

I haven't received it since December 13, 2019 (GMT + 9). There is a log in the forwarder but I don't know why not forwarding it.

0 Karma
Highlighted

Re: Why did a Splunk forwarder stop sending to log packet?

Legend

Hi @lifekis,
ok, let start dubugging:
at first check if you have internal logs from that server (index=_internal host=your_host).
If you haven't them, the problem is on the connection, so test the route (from the UF telnet ip_splunk_indexer 9997).
if Nok verify firewall route between servers, if Ok check in outputs.conf on UF if the address of the Indexers is correct.

If instead you have Splunk internal logs but not the logs you want, check in inputs.conf if the path and the filename of the monitor stanza is still correct.

Ciao.
Giuseppe

0 Karma
Highlighted

Re: Why did a Splunk forwarder stop sending to log packet?

Explorer

Thank you for answer! I checked the disconnection log in splunkd.log.

0 Karma
Highlighted

Re: Why did a Splunk forwarder stop sending to log packet?

Explorer

Here are the steps I would take to troubleshoot -
(1) Login to the server to make sure splunk process is running
(2) Check that your outputs.conf is still on the system
(3) Try to do a network connectivity test to your HF/Indexer
(4) Check splunkd.log to see if there are any errors

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.