Splunk Search

If 3 consecutive files are found > 1KB in size, send an email alert

chaitalynavare
Engager

How can I get results only when 3 consecutive files exceeds 1 KB limit?

I tried this with below Query however not getting expected results -

index=idx1 sourcetype=src1 | eval raw_len=(len(_raw)/1024) | stats sum(raw_len) as filesize by source | streamstats window=3 values(eval(filesize>=1)) as finalcheck by source

Can someone please help me on this?

Thank you in Advance.

Tags (1)
0 Karma
1 Solution

micahkemp
Champion

Try this (though I'm not saying it's a clean or efficient solution):

index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval is_greater_1k=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(is_greater_1k) AS last_is_greater_1k
| eval is_new_greater_1k_group=if(isnull(last_is_greater_1k) OR (is_greater_1k=0 AND last_is_greater_1k=1), 1, 0)
| streamstats sum(is_new_greater_1k_group) AS greater_1k_group
| streamstats count AS files_in_greater_1k_group BY greater_1k_group
| search files_in_greater_1k_group>=3

View solution in original post

micahkemp
Champion

Try this (though I'm not saying it's a clean or efficient solution):

index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval is_greater_1k=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(is_greater_1k) AS last_is_greater_1k
| eval is_new_greater_1k_group=if(isnull(last_is_greater_1k) OR (is_greater_1k=0 AND last_is_greater_1k=1), 1, 0)
| streamstats sum(is_new_greater_1k_group) AS greater_1k_group
| streamstats count AS files_in_greater_1k_group BY greater_1k_group
| search files_in_greater_1k_group>=3

chaitalynavare
Engager

Thank you @micahkemp for the hint. I have modified the above query and able to fix the problem. The below query is working query where 3 consecutive files > 1KB size can be found -

index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval filesize_1=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(filesize_1) AS filesize_2
| streamstats current=f window=1 last(filesize_2) AS filesize_3
|eval finalcheck=case(filesize_1=="1" AND filesize_2=="1" AND filesize_3=="1","YES")
|search finalcheck="YES"

0 Karma

koshyk
Super Champion

how do you define "consecutive" files? is it in alphabetical order or files within same directory?

A quick way to count size of file if its indexed is

index=_internal source="/opt/splunk/var/log/splunk/license_usage.log" s="somefile"| stats sum(b) by s

Then you can combine saying three files within the directory > 1KB, do something etc.

0 Karma

chaitalynavare
Engager

Here consecutive files mean files within same directory as in below fashion. If 3 consecutive files are having output as "1" as in "Expected OUTPUT" column in table then trigger alert.

I am getting output as below -

            source                      filesize               finalcheck                   
C:\splunkfile\20171116-01.txt       7.521494            1                           
C:\splunkfile\20171116-02.txt       2.948234            1                           
C:\splunkfile\20171116-03.txt       0.702148                                        
C:\splunkfile\20171116-04.txt       1.938470            1                           
C:\splunkfile\20171116-05.txt       0                                               
C:\splunkfile\20171116-06.txt       1.712890            1                           
C:\splunkfile\20171116-07.txt       2.954095            1                           
C:\splunkfile\20171116-08.txt       4.965805            1                           
C:\splunkfile\20171116-09.txt       0.944326            1                           
C:\splunkfile\20171116-10.txt       1.399411            1

Expected output is as below -

                 source                 filesize            Expected OUPPUT
C:\splunkfile\20171116-01.txt       7.521494                    0
C:\splunkfile\20171116-02.txt       2.948234                    0
C:\splunkfile\20171116-03.txt       0.702148                    0
C:\splunkfile\20171116-04.txt       1.938470                    0
C:\splunkfile\20171116-05.txt       0                           0
C:\splunkfile\20171116-06.txt       1.712890                    1
C:\splunkfile\20171116-07.txt       2.954095                    1
C:\splunkfile\20171116-08.txt       4.965805                    1
C:\splunkfile\20171116-09.txt       0.944326                    0
C:\splunkfile\20171116-10.txt       1.399411                    0
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...