Splunk Search

expect value

indeed_2000
Motivator

Hi
I have log file like this:

09:04:04.042 module1:  F[6]L: IN
09:04:01.417 module1:  F[6]L: OUT
09:04:01.418 module4:  F[6]L: IN
09:04:01.419 module4:  F[6]L: OUT
09:04:01.420 module12: F[6]L: IN
09:04:01.421 module2:  F[6]L: IN
09:04:01.422 module41: F[6]L: IN
09:04:01.426 module12: F[6]L: OUT
09:04:01.427 module50: F[18]L: IN
09:04:01.428 module52: F[20]L: IN
09:04:01.429 module50: F[18]L: OUT
09:04:01.435 module52: F[20]L: OUT

as you see every module had (IN) value after while (OUT).

Now I want to define something to expect (OUT) value for each (IN) due to the Fingerprint and Module.
For example in above log file:
1- group them by F (F value means fingerprint)
2- group them by modules
3- detect any F had (IN) but no (OUT). example module2, module41 with F[6] had IN (input) but never had OUT (output).

Any recommendation?

Thanks,

0 Karma

woodcock
Esteemed Legend

Like this:

| makeresults
| eval _raw="Time,module,fingerprint,direction
9:04:04.042,module1,F[6]L,IN
09:04:01.437,module1,F[6]L,OUT
09:04:01.427,module4,F[6]L,IN
09:04:01.422,module4,F[6]L,OUT
09:04:01.381,module12,F[6]L,IN
09:04:01.371,module2,F[6]L,IN
09:04:01.338,module41,F[6]L,IN
09:04:01.381,module12,F[6]L,OUT
09:04:01.338,module50,F[18]L,IN
09:04:01.381,module52,F[20]L,IN
09:04:01.338,module50,F[18]L,OUT
09:04:01.381,module52,F[20]L,OUT"
| multikv forceheader=1
| eval _time = strptime(Time, "%H:%M:%S.%3N")
| sort 0 - _time

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| stats dc(direction) AS dc values(direction) AS directions BY fingerprint module
| where dc<2
0 Karma

indeed_2000
Motivator

Thank you @woodcock this is exactly what i want.

indeed_2000
Motivator

run your solution on this part of log:
09:04:01.042,module1,F[6]L,IN
09:04:01.437,module1,F[6]L,OUT
09:04:01.438,module4,F[6]L,IN
09:04:01.439,module4,F[6]L,OUT
09:04:01.481,module2,F[6]L,IN
09:04:01.482,module2,F[6]L,IN
09:04:01.483,module2,F[6]L,IN
09:04:01.484,module2,F[6]L,OUT
09:04:01.485,module2,F[6]L,OUT
09:04:01.488,module50,F[18]L,IN
09:04:01.489,module52,F[20]L,IN
09:04:01.490,module53,F[18]L,OUT
09:04:01.491,module52,F[20]L,OUT"

result:
09:04:01.488,module50,F[18]L,IN
09:04:01.490,module53,F[18]L,OUT

expected result:
09:04:01.483,module2,F[6]L,IN
09:04:01.488,module50,F[18]L,IN
09:04:01.490,module53,F[18]L,OUT

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mehrdad_2000,
you could run something like this:

index=my_index
| rex "^\d+:\d+:\d+\.\d+\s+(?<module>[^:]*):\s+(?<fingerprint>[^:]*):\s+(?<value>\w+)"
| stats dc(value) AS dc_values values(value) AS value BY module fingerprint
| where dc_values<2 AND value="IN"
| table module fingerprint

Ciao and Happy New Year.
Giuseppe

0 Karma

indeed_2000
Motivator

Thank you and Happy New Year 🙂
this is large file I can't define dc value, it has lots of this kind of events.

I expect every single value that hadn't output.

Any idea?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mehrdad_2000,
if you have many events it isn't a problem!
if you could have more than 2 events for the same module and fingerprint, you have to use the transaction command that's a very slow command

 index=my_index
 | rex "^\d+:\d+:\d+\.\d+\s+(?<module>[^:]*):\s+(?<fingerprint>[^:]*):\s+(?<value>\w+)"
 | transaction module fingerprint startswith=": IN"
 | where eventcount<2
 | table module fingerprint

Ciao and Happy New Year.
Giuseppe

0 Karma

indeed_2000
Motivator

run your solution on this part of log:

 09:04:01.042,module1,F[6]L,IN
 09:04:01.437,module1,F[6]L,OUT
 09:04:01.438,module4,F[6]L,IN
 09:04:01.439,module4,F[6]L,OUT
 09:04:01.481,module2,F[6]L,IN
 09:04:01.482,module2,F[6]L,IN
 09:04:01.483,module2,F[6]L,IN
 09:04:01.484,module2,F[6]L,OUT
 09:04:01.485,module2,F[6]L,OUT
 09:04:01.488,module50,F[18]L,IN
 09:04:01.489,module52,F[20]L,IN
 09:04:01.490,module53,F[18]L,OUT
 09:04:01.491,module52,F[20]L,OUT"

result:
09:04:01.481,module2,F[6]L,IN
09:04:01.482,module2,F[6]L,IN
09:04:01.488,module50,F[18]L,IN

expected result:
09:04:01.483,module2,F[6]L,IN
09:04:01.488,module50,F[18]L,IN
09:04:01.490,module53,F[18]L,OUT

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...