Getting Data In

Heavy forwarder connections to Splunk indexers by checking internal logs for any type of issues.

phanikumarcs
Explorer

i have to create an alert to monitor any issue happens for HF to Indexers, by checking internal logs. I am using this spl. Need suggestions or correct SPL.

index=_internal source=*metrics.log group=tcpin_connections hostname="*hf*"

Labels (1)
Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @phanikumarcs ,

I'd simplify your search to search eventual erros:

index=_internal hostname="*hf*"

whot do you want to monitor?

Ciao.

Giuseppe

0 Karma

phanikumarcs
Explorer

Yes, but we have to setup a alert for example if any issues will get based on warns or errors it will trigger

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @phanikumarcs ,

as I said, define your Use Cases, then you could create your searches.

e.g. you could create an alert for the queues:

index=_internal  source=*metrics.log sourcetype=splunkd group=queue 
| eval name=case(name=="aggqueue","2 - Aggregation Queue",
 name=="indexqueue", "4 - Indexing Queue",
 name=="parsingqueue", "1 - Parsing Queue",
 name=="typingqueue", "3 - Typing Queue",
 name=="splunktcpin", "0 - TCP In Queue",
 name=="tcpin_cooked_pqueue", "0 - TCP In Queue") 
| eval max=if(isnotnull(max_size_kb),max_size_kb,max_size) 
| eval curr=if(isnotnull(current_size_kb),current_size_kb,current_size) 
| eval fill_perc=round((curr/max)*100,2) 
| bin _time span=1m
| stats Median(fill_perc) AS "fill_percentage" perc90(fill_perc) AS "90_perc" max(max) AS max max(curr) AS curr by host, _time, name 
| where (fill_percentage>70 AND name!="4 - Indexing Queue") OR (fill_percentage>70 AND name="4 - Indexing Queue")
| sort -_time

then you could check the disk space,

or whtelse you like.

Anyway: define your Use Cases

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...