Splunk Search

Help with multiple search to create negative number

indeed_2000
Motivator

Hi
I have three search commands that need to combine result and show on a timechart.

I have to count "Success",  "Fail", "Total", "Other"

Here is my SPL:

 

index="myindex" TURNOFF NOT DISPOSE AND "A[*00]" AND "Packet Processed:" source="/data/app.log*"| bin _time span=1h | stats count AS Success by _time
| join _time
[search index="myindex" TURNOFF NOT DISPOSE AND "Packet Processed:" source="/data/app.log*"| rex "R\[(?<R>\d+)"| search A!=*00| bin _time span=1h | stats count AS Failed by _time ]
| join _time
[search index="myindex" TURNOFF NOT DISPOSE AND "Packet Processed:" AND "A[*]" source="/data/app.log*"| bin _time span=1h | stats count AS Totals by _time ]

| eval TotalSF=Success+Failed | eval Other=Totals-TotalSF | fields - Totals TotalSF| addtotals

 

it works correctly on a specific time range (like 2 or 3 hours) but when I set the time to Yesterday and it created a negative number (for field that I call Other).

 

indeed_2000_0-1656740552633.png

Any idea?
Thank

Labels (5)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

In addition to @ITWhisperer comments, your 3 searches should be a single search. It's not clear what your 'Other' condition is without seeing your data, but you should be able to use an eval statement to create a variable to represent the state of the event and then just sum the types up as needed, something like 

index="myindex" TURNOFF NOT DISPOSE AND "Packet Processed:" source="/data/app.log*"
| bin _time span=1h
| eval Success=if(match(_raw, "A\[[^\]]*00\]"), 1, 0)
| eval Failed=if(match(_raw, "Failed_Condition_Regex"), 1, 0)
| eval Other=if(match(_raw, "Other_Condition_Regex"), 1, 0)
| stats count AS Totals sum(Success) as Success sum(Failed) as Failed sum(Other) as Other by _time 

 This would be more efficient and faster than using two joins

 

indeed_2000
Motivator

actually it's not regex, it's some search command, any other idea?

current:

 

| eval Failed=if(match(_raw, "Failed_Condition_Regex"), 1, 0)
| eval Other=if(match(_raw, "Other_Condition_Regex"), 1, 0)

 

 

expected:

1- 

 

TURNOFF NOT DISPOSE AND "A[*00]" AND "Packet Processed:" 

 

2-

 

 TURNOFF NOT DISPOSE AND "Packet Processed:" search A!=*00

 

3-

TURNOFF NOT DISPOSE AND "Packet Processed:" AND "A[*]"

 

0 Karma

bowesmana
SplunkTrust
SplunkTrust

All 3 search conditions have the base

index="myindex" TURNOFF NOT DISPOSE AND "Packet Processed:" source="/data/app.log*"

then success is

"A[*00]"

which maps (with my interpretation of your data) onto the Success regex of

| eval Success=if(match(_raw, "A\[[^\]]*00\]"), 1, 0)

Failure is

A!=*00

and your "Totals" is

"A[*]"

As @ITWhisperer your use of wildcards in your search is not useful, as the wildcard can be greedy. 

Your success search is looking for the string A[*00] in the raw event where * is a wildcard and can match anything between brackets.

Your failure example is looking for the FIELD A which does NOT contain *00 - note this is different to the opposite of the success search, which is looking in RAW field.

Your Totals example is looking for the string A[*] in the raw event, again using potentially greedy wildcards.

If you can provide an example of your data that represent each of these 3 potential data types we can better advise.

 

Tags (1)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You are right that search does not use regular expressions but it does use wildcards and it is the wildcards which are giving you false positives.

To improve the accuracy of wild-carding, regular expressions (regex) as used by the match function https://docs.splunk.com/Documentation/SCS/current/SearchReference/ConditionalFunctions#match.28.26lt... will help eliminate the false positives (which are leading to your negative results).

Without seeing your actual events (or at least de-sensitised representations of them), it is difficult to recommend a solution, beyond the ideas we have already given you.

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

To be fair, it isn't the multiple search that creates the negative number, it is the calculation that does that!

The reason the calculation creates a negative number is because of the way you are determining successes and failures. It probably the successes that are counting false positives due to the wild card in the search, although without seeing the actual events, it is difficult to verify this.

For example, an event like this

blah blah A[123] B[200]

would be picked up as a  success because "A[*00]" will match to "A[123] B[200]", etc.

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...