Splunk Enterprise

How to search the internal logs for the remote agent via Splunk portal ?

Hemnaath
Motivator

Hi All,

How to search the internal logs of the remote agent (UF) node via Splunk Portal ? 

I am trying to troubleshoot why the logs are not ingested into Splunk from the remote agent node, I did simple search query from the search head console.

index="_internal"  sourcetype="splunkd.log"  host="test1" 

but unable to get any result, so please do let me know how to search the internal log details from the search head portal.

When I log into the UF server I can see the following information  Error | Warn | Info details  from the splunkd.log  but my intension is to check the same from the Splunk console .

Kindly guide me on the same.

 

Labels (1)
0 Karma

johnhuang
Motivator

Try these to see if you  get any results:

index="_internal" sourcetype="splunkd*" host="test1" 

index="_internal" test1

0 Karma

PickleRick
SplunkTrust
SplunkTrust

The UF's logs get ingested to _internal index by means of the UF forwarding them as any other log so if your connection/forwarding is not working you'll not get _internal events from that forwarder.

As @richgalloway said - check forwarder's logs on forwarder's side - you're mostly interested in splunkd.log file. And of course perform the typical network-level troubleshooting.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The query you used is a good one.  If it returns no results then either you don't have access to _internal or the UF's logs are not being forwarded to the indexer(s).

Sign in to the server on which the UF is running and use OS tools to examine the logs to learn why it's not forwarding them.

Check your firewalls, too.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Buttercup Games Tutorial Extension - part 9

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Buttercup Games Tutorial Extension - part 8

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Introducing the Splunk Developer Program!

Hey Splunk community! We are excited to announce that Splunk is launching the Splunk Developer Program in ...