Splunk Search

How to create a search to detect if no increase in the count of fields?

jakeoftrades
Explorer

hi,

can anyone help me how should I query the counts of kafka_datatype  of those stream_type which Im going to set an alert if there will be no increase within  3 hours within only a time period of time 7am-8pm 

I have this query :

 

index="pcg_p4_datataservices_prod" sourcetype="be:monitoring-services"
| setfields a=a
| rex "^[^\|\n]*\|\s+(?P<kafka_datatype>\w+)\s+\-\s+(?P<kafka_count>.+)"
| search kafka_datatype IN (PRODUCED, CONSUMED)
| search stream_type IN (Datascore_Compress, Datascore_Decompress, Eservices_Eload, Eservices_Ebills)
| eval service_details=stream_type
| timechart span=3h limit=0 sum(kafka_count) by service_details

 



I tried to add in the query for the earliest/latest (as -180m I guess for 3hrs) but not showing any result that Im aiming to set it  to detect that if for 3 hours if no increase in the count of kafka_datatype of those stream_type which Im going to set the alert:

 

index="pcg_p4_datataservices_prod" sourcetype="be:monitoring-services"
earliest=-180m
latest=now
| setfields a=a
| rex "^[^\|\n]*\|\s+(?P<kafka_datatype>\w+)\s+\-\s+(?P<kafka_count>.+)"
| search kafka_datatype IN (PRODUCED, CONSUMED)
| search stream_type IN (Datascore_Compress, Datascore_Decompress, Eservices_Eload, Eservices_Ebills)
| eval service_details=stream_type
| timechart span=3h limit=0 sum(kafka_count) by service_details

 

Labels (2)
Tags (2)
0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...