Splunk Search

Detect abnormalities with splunk

indeed_2000
Builder

Hi

I have key value that call (duration) in my application log that show duration of each job done.

each day when I get maximum duration it has false positive because it is natural to become high duration in some point.

It’s not normal when it continues high duration.

e.g. 
normal condition:

00:01:00.000 WARNING duration[0.01]
00:01:00.000 WARNING duration[100.01]
00:01:00.000 WARNING duration[0.01]

 

abnormal condition:

00:01:00.000 WARNING duration[0.01]
00:01:00.000 WARNING duration[100.01]
00:01:00.000 WARNING duration[50.01]

00:01:00.000 WARNING duration[90.01]
00:01:00.000 WARNING duration[100.01]
00:01:00.000 WARNING duration[0.01]

 

1-how can I detect abnormal condition with splunk? (Best way with minimum false positive on hug data)

2-which visualization or chart more suitable to show this abnormal condition daily? this is huge log file and it is difficult to show all data for each day on single chart.

Any idea?

 Thanks,

 

Labels (5)
0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!