Splunk Search

How to filter numeric field with where clause?

fgysin
Explorer

So, our application logs duration times of logged method calls as ..dT=XXXms.. and I would like to use this for nice splunk graphs.

This works brilliantly if I use a query like this (in advanced charting view)

eventtype="app" dT | timechart avg(dT)

My Problem is, that the application rarely logs absurdly high duration times going up to several years - clearly a bug of the logging framework we are using.

These high dT values sadly totally screw up my nice timechart graphs, and mess with statistics. How can I filter out these values?
I already tried filtering those log statements using a where clause, but so far this has not worked for me - result set stays empty.

eventtype="app" dT | where dT<3600000 | timechart avg(dT)

Any ideas would be much appreciated!

Tags (3)
0 Karma
1 Solution

MuS
Legend

Hi fgysin,

you can use the filter in your base search like this:

 eventtype="app" dT<3600000 | timechart avg(dT)

cheers, MuS

View solution in original post

MuS
Legend

Hi fgysin,

you can use the filter in your base search like this:

 eventtype="app" dT<3600000 | timechart avg(dT)

cheers, MuS

fgysin
Explorer

Awesome stuff, much appreciated.

0 Karma

strive
Influencer

eventtype="app" dT | eval dT = tonumber(substr(dT,0,len(dT)-2)) | where dT<3600000 | timechart avg(dT)

fgysin
Explorer

Ah I see. So how would I remove the ms? With the rex command?

0 Karma

MuS
Legend

ahh I see, your field is like dT=XXXms ... so remove the ms first and then you can filter 😉

MuS
Legend

take this run everywhere example:

 index=_internal earliest=-2h@h latest=-1h@h kb | where kb<128 | stats count
 index=_internal earliest=-2h@h latest=-1h@h kb<128 | stats count

both will return the same count. Is this dT field numeric or a string?

0 Karma

fgysin
Explorer

Hmm, that does not work for me... The is graph still plotting average values which lie in the millions and billions.

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...