Splunk Search

Getting alerts on sudden increase in traffic.


I have a custom script that collects stats on a custom HW appliance every minute and forwards it to our splunk system.

And has following style data:



log_type="throughput_data", local_time="2020/09/09 19:01 CST", server_ip="", host_name="host2", host_ip="", version="13", model="M1000", serial_no="1234234", ssl_card="No", total_traffic="93700", app_traffic="17524", cpu="15", ssl="0", http="258",connections="1", sql="0", sql2="0"
log_type="throughput_data", local_time="2020/09/09 19:01 CST", server_ip="", host_name="host5", host_ip="", version="13", model="M2000", serial_no="12342342", ssl_card="No", total_traffic="0", app_traffic="0", cpu="3", ssl="0", http="0",connections="0", sql="0", sql2="0"




I have a 2 parter question:

  1. How do I go about generating an alert when the app_traffic has a sudden spike or out of usual spike.
    EG: normally the app_traffic hovers around 500 and there was a sudden increase to 10000.

    Just having this will make my team happy, but I do not believe that is the proper solution we need

  2. Is there a way I can go about and create a dataset/lookup for each models supported datasheet values and generate an alert when that models certain values go up.
    EG: Model M1000 can do total app_traffic of 10000 and have an alert be generated when it reaches 90% of that value; in this case 9000.
    1. Can this be split do alert if either app_traffic or total_traffic or CPU or SSL reach 90% of the set limit in the data set

      I believe this will help us scale and be better for future use cases and making a business use case for management.




Labels (2)
0 Karma


Thanks, Now I have a strong case to get our 6.5.2 upgraded to latest version.

0 Karma


I think below conf video will help you. finding outliers.


If this helps, give a like below.

You should look this conf presentation. https://conf.splunk.com/files/2016/slides/time-after-time-comparing-time-ranges-in-splunk.pdf
It gives you to good basement for this challenge.
r. Ismo
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...