Deployment Architecture

How to determine the Forwarder that is sending data to a particular Index? Not Indexer!

Pawlub1
Engager

So I am troubleshooting missing data from hosts, I have the index name that is missing the data, and so I would like to track down the suspect forwarder that is not sending the data to the index. 

I am not interested in the indexer server. 

 

Labels (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Splunk generally works based on push principle which means that forwarders just send what they want and splunk happily accepts that. There is no configuration items saying that some particular forwarder should send only events to one index or something like that.

So you either must know that beforehand, have it stored in some lookup and verify as @somesoni2 wrote or check which hosts _used to_ send to a given index (provided the data hadn't rolled out of the index already) and stopped.

0 Karma

somesoni2
Revered Legend

I would assume that 'host' field in the Splunk is matching your forwarder server name.

To find out which hosts are not sending data for a particular index, you'd need a lookup (or list ) of all hosts that are expected to send data to an index, then you can cross compare the list/lookup with hosts that are actually sending data.  Something like this

| tstats count where index=yourIndexName by host
| append [| inputlookup yourhostlist.csv | table host | eval count=0]
| stats max(count) as count by host 
| where count=0

 

If you can't create a lookup and have smaller list of hosts, you could do something like this

 

| tstats count where index=yourIndexName by host
| append [| makeresults| eval host=split("host1 host2 .. list all your hosts space seperated" | table host | mvexpand host| eval count=0]
| stats max(count) as count by host 
| where count=0
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Pawlub1,

you could run something like this:

index=*
| stats values(index) AS index count BY host

in this way you have the list of each destination index for each host.

Ciao.

Giuseppe

Pawlub1
Engager

Yes, that was helpful and sorry for my delayed confirmation. 

0 Karma

Pawlub1
Engager

Hi Guiseppe, 

I appreciate your quick response, but your script shows hosts sending data to an index.

I need a script that displays hosts sending to a forwarder than sent an index. 

hosts > forwarder > index

Ciao, Paul

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Pawlub1,

sorry, let me understand: do you want to know an intermediate forwarders passing through data?

if this is your requirement it isn't possible now.

I proposed to add this information to Splun ideas (https://ideas.splunk.com/ideas/EID-I-1731) and it's "Under consideration", if you agree that's important vote for it.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...