Splunk Search

If 3 consecutive files are found > 1KB in size, send an email alert

chaitalynavare
Engager

How can I get results only when 3 consecutive files exceeds 1 KB limit?

I tried this with below Query however not getting expected results -

index=idx1 sourcetype=src1 | eval raw_len=(len(_raw)/1024) | stats sum(raw_len) as filesize by source | streamstats window=3 values(eval(filesize>=1)) as finalcheck by source

Can someone please help me on this?

Thank you in Advance.

Tags (1)
0 Karma
1 Solution

micahkemp
Champion

Try this (though I'm not saying it's a clean or efficient solution):

index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval is_greater_1k=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(is_greater_1k) AS last_is_greater_1k
| eval is_new_greater_1k_group=if(isnull(last_is_greater_1k) OR (is_greater_1k=0 AND last_is_greater_1k=1), 1, 0)
| streamstats sum(is_new_greater_1k_group) AS greater_1k_group
| streamstats count AS files_in_greater_1k_group BY greater_1k_group
| search files_in_greater_1k_group>=3

View solution in original post

micahkemp
Champion

Try this (though I'm not saying it's a clean or efficient solution):

index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval is_greater_1k=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(is_greater_1k) AS last_is_greater_1k
| eval is_new_greater_1k_group=if(isnull(last_is_greater_1k) OR (is_greater_1k=0 AND last_is_greater_1k=1), 1, 0)
| streamstats sum(is_new_greater_1k_group) AS greater_1k_group
| streamstats count AS files_in_greater_1k_group BY greater_1k_group
| search files_in_greater_1k_group>=3

View solution in original post

chaitalynavare
Engager

Thank you @micahkemp for the hint. I have modified the above query and able to fix the problem. The below query is working query where 3 consecutive files > 1KB size can be found -

index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval filesize_1=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(filesize_1) AS filesize_2
| streamstats current=f window=1 last(filesize_2) AS filesize_3
|eval finalcheck=case(filesize_1=="1" AND filesize_2=="1" AND filesize_3=="1","YES")
|search finalcheck="YES"

0 Karma

koshyk
Super Champion

how do you define "consecutive" files? is it in alphabetical order or files within same directory?

A quick way to count size of file if its indexed is

index=_internal source="/opt/splunk/var/log/splunk/license_usage.log" s="somefile"| stats sum(b) by s

Then you can combine saying three files within the directory > 1KB, do something etc.

0 Karma

chaitalynavare
Engager

Here consecutive files mean files within same directory as in below fashion. If 3 consecutive files are having output as "1" as in "Expected OUTPUT" column in table then trigger alert.

I am getting output as below -

            source                      filesize               finalcheck                   
C:\splunkfile\20171116-01.txt       7.521494            1                           
C:\splunkfile\20171116-02.txt       2.948234            1                           
C:\splunkfile\20171116-03.txt       0.702148                                        
C:\splunkfile\20171116-04.txt       1.938470            1                           
C:\splunkfile\20171116-05.txt       0                                               
C:\splunkfile\20171116-06.txt       1.712890            1                           
C:\splunkfile\20171116-07.txt       2.954095            1                           
C:\splunkfile\20171116-08.txt       4.965805            1                           
C:\splunkfile\20171116-09.txt       0.944326            1                           
C:\splunkfile\20171116-10.txt       1.399411            1

Expected output is as below -

                 source                 filesize            Expected OUPPUT
C:\splunkfile\20171116-01.txt       7.521494                    0
C:\splunkfile\20171116-02.txt       2.948234                    0
C:\splunkfile\20171116-03.txt       0.702148                    0
C:\splunkfile\20171116-04.txt       1.938470                    0
C:\splunkfile\20171116-05.txt       0                           0
C:\splunkfile\20171116-06.txt       1.712890                    1
C:\splunkfile\20171116-07.txt       2.954095                    1
C:\splunkfile\20171116-08.txt       4.965805                    1
C:\splunkfile\20171116-09.txt       0.944326                    0
C:\splunkfile\20171116-10.txt       1.399411                    0
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.