Splunk Enterprise

How to search the internal logs for the remote agent via Splunk portal ?

Hemnaath
Motivator

Hi All,

How to search the internal logs of the remote agent (UF) node via Splunk Portal ? 

I am trying to troubleshoot why the logs are not ingested into Splunk from the remote agent node, I did simple search query from the search head console.

index="_internal"  sourcetype="splunkd.log"  host="test1" 

but unable to get any result, so please do let me know how to search the internal log details from the search head portal.

When I log into the UF server I can see the following information  Error | Warn | Info details  from the splunkd.log  but my intension is to check the same from the Splunk console .

Kindly guide me on the same.

 

Labels (1)
0 Karma

johnhuang
Motivator

Try these to see if you  get any results:

index="_internal" sourcetype="splunkd*" host="test1" 

index="_internal" test1

0 Karma

PickleRick
SplunkTrust
SplunkTrust

The UF's logs get ingested to _internal index by means of the UF forwarding them as any other log so if your connection/forwarding is not working you'll not get _internal events from that forwarder.

As @richgalloway said - check forwarder's logs on forwarder's side - you're mostly interested in splunkd.log file. And of course perform the typical network-level troubleshooting.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The query you used is a good one.  If it returns no results then either you don't have access to _internal or the UF's logs are not being forwarded to the indexer(s).

Sign in to the server on which the UF is running and use OS tools to examine the logs to learn why it's not forwarding them.

Check your firewalls, too.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...