Getting Data In

SPLUNK INDEX Discovery

christay
New Member

Hi Guys,

I have configured using index discovery for my forwarder which are forwarding my firewall logs.
I saw from my splunkd.log it seems like the connection to my indexer is successful however, i can't see any logs from my indexer dashboard.

x.x.x.x is my 1st index server
y.y.y.y is my 2nd index server

logs

05-15-2019 03:59:05.238 +0800 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997, pset=0, reuse=0. using ACK.
05-15-2019 03:59:10.784 +0800 WARN TailReader - Could not send data to output queue (parsingQueue), retrying...
05-15-2019 03:59:35.133 +0800 INFO TcpOutputProc - Closing stream for idx=x.x.x.x:9997
05-15-2019 03:59:35.133 +0800 INFO TcpOutputProc - Connected to idx=y.y.y.y:9997, pset=0, reuse=0. using ACK.
05-15-2019 03:59:40.785 +0800 INFO TailReader - ...continuing.
05-15-2019 03:59:45.785 +0800 WARN TailReader - Could not send data to output queue (parsingQueue), retrying...
05-15-2019 04:00:08.725 +0800 INFO TailReader - ...continuing.
05-15-2019 04:01:00.462 +0800 INFO ArchiveProcessor - Handling file=/var/log/fortigate/fortigate.log-20190515.gz
05-15-2019 04:01:00.462 +0800 INFO ArchiveProcessor - new tailer already processed path=/var/log/fortigate/fortigate.log-20190515.gz
05-15-2019 04:01:04.850 +0800 INFO TcpOutputProc - Closing stream for idx=y.y.y.y:9997
05-15-2019 04:01:04.850 +0800 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997, pset=0, reuse=0. using ACK.
05-15-2019 04:03:04.453 +0800 INFO TcpOutputProc - Closing stream for idx=x.x.x.x:9997
05-15-2019 04:03:04.453 +0800 INFO TcpOutputProc - Connected to idx=y.y.y.y::9997, pset=0, reuse=0. using ACK.
05-15-2019 04:04:04.270 +0800 INFO TcpOutputProc - Closing stream for idx=y.y.y.y:9997
05-15-2019 04:04:04.270 +0800 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997, pset=0, reuse=0. using ACK.

The indexes over at the index servers are not updated with any latest event as well.
Any idea how i can troubleshoot on this issue ?

Thanks

Tags (1)
0 Karma

codebuilder
Influencer

Is your forwarder running as the splunk user? Files and directories under /var/log/ are typically owned by root.
Try changing the user on your forwarder, or change the ownerships of your directory under /var/log/

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

christay
New Member

I have changed the ownership of /var/log to splunk user, but the result is still the same.
The splunkd.log doesn't share much insight to the issue as well....

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

A Four-Part Event Series: Full Stack Observability For the AI Era

As AI reshapes applications, infrastructure, and the way teams operate, the traditional boundaries of ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...