Dashboards & Visualizations

rangemap with too many values

tyronetv
Communicator

have this search:

application response sourcetype=log1 OR sourcetype=log2 (host = host1 OR host = host2 OR host = host3 OR host = host4 ) | rex "(A|a)pplication response.*time was\s+(?P<app_response>\d+)\s" | rangemap field=app_response "A. Less than 0.25 seconds"=0-249 "B. More than 0.25 but less than 0.5 seconds"=250-500 "C. More than half-second but less than a second"=500-1000 default="D. More than a second" |stats count by range

Should work, right? If I run it with just "stats count" I get 55,127 returns.

If I run it with rangemap I get 77,484 with 22,377 going to the "default" category.

If I do the search and and only search for items over 1000 ms I get zero ( "search app_response>1000").

So, why the extra bad numbers? What am I doing wrong?

Tags (1)
0 Karma
1 Solution

lguinn2
Legend

Rangemap is a strange command - it is actually a custom command and written as a Python script. I would try this instead:

application response sourcetype=log1 OR sourcetype=log2 (host = host1 OR host = host2 OR host = host3 OR host = host4 ) 
| rex "(A|a)pplication response.*?time was\s+(?P<app_response>\d+)\s" 
| where app_response >= 0
| eval appResponseCategory = case(
          app_response<250,"A. Less than 0.25 seconds",
          app_response>=250 AND app_response<500,"B. More than 0.25 but less than 0.5 seconds"
          app_response>=500 AND app_response<1000,"C. More than half-second but less than a second"
          "1"=="1","D. One second or more" )

Note that I eliminated events that did not have an application response time - this may be where your "extra" default events were arising. Also, I made sure that the categories did not overlap, as your original categories did at 500 (one-half second). Finally, I think that the case function will out-perform the rangemap command.

View solution in original post

lguinn2
Legend

Rangemap is a strange command - it is actually a custom command and written as a Python script. I would try this instead:

application response sourcetype=log1 OR sourcetype=log2 (host = host1 OR host = host2 OR host = host3 OR host = host4 ) 
| rex "(A|a)pplication response.*?time was\s+(?P<app_response>\d+)\s" 
| where app_response >= 0
| eval appResponseCategory = case(
          app_response<250,"A. Less than 0.25 seconds",
          app_response>=250 AND app_response<500,"B. More than 0.25 but less than 0.5 seconds"
          app_response>=500 AND app_response<1000,"C. More than half-second but less than a second"
          "1"=="1","D. One second or more" )

Note that I eliminated events that did not have an application response time - this may be where your "extra" default events were arising. Also, I made sure that the categories did not overlap, as your original categories did at 500 (one-half second). Finally, I think that the case function will out-perform the rangemap command.

tyronetv
Communicator

This was the fix. I had used case before (another question) but was suggested I use rangemap. For this search using case is the clear winner.
Thanks.

0 Karma

tyronetv
Communicator

2013-03-01 12:35:28,878 INFO [ler-HTTPThreadGroup-17042] RID=1362170128682-2299470 c.r.t.i.s.e.applicationImageArchiveConnection - application response time was 138 milliseconds.

2013-03-01 14:35:22,040[ndler-HTTPThreadGroup-681] INFO dis.service.application.ImageArchiveConnection - [1] application response time was 128 milliseconds.

2013-03-01 12:35:21,950 INFO [ler-HTTPThreadGroup-17053] RID=1362170121771-2299465 c.r.t.i.s.e.applicationImageArchiveConnection - application response time was 124 milliseconds.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Can you provide anonymized sample data?

0 Karma

tyronetv
Communicator

Maybe I wasn't clear. The default bucket has 20,000+ count that do not exist in the search. Read my initial post. Search pipe to count gives 50K+ results. Search pipe to rangemap gives 70K+ results.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The default bucket contains all events that do not belong in another bucket. That's those >1000, those <0, and those with no value.

0 Karma

tyronetv
Communicator

They have "no value" that I can find. There should be no counts in the default bucket. But, to more specifically answer, if you look at the query, I believe the default should include values > 1000

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

What duration value(s) do those in the default bucket have?

0 Karma

tyronetv
Communicator

No, the issue is that the 'default' bucket has 20,000+ that don't exist in the primary search string.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Are there by any chance thousands with exactly 500ms duration? Those get rangemapped twice due to an overlap at 500.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...