How can I get results only when 3 consecutive files exceeds 1 KB limit?
I tried this with below Query however not getting expected results -
index=idx1 sourcetype=src1 | eval raw_len=(len(_raw)/1024) | stats sum(raw_len) as filesize by source | streamstats window=3 values(eval(filesize>=1)) as finalcheck by source
Can someone please help me on this?
Thank you in Advance.
Try this (though I'm not saying it's a clean or efficient solution):
index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval is_greater_1k=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(is_greater_1k) AS last_is_greater_1k
| eval is_new_greater_1k_group=if(isnull(last_is_greater_1k) OR (is_greater_1k=0 AND last_is_greater_1k=1), 1, 0)
| streamstats sum(is_new_greater_1k_group) AS greater_1k_group
| streamstats count AS files_in_greater_1k_group BY greater_1k_group
| search files_in_greater_1k_group>=3
Try this (though I'm not saying it's a clean or efficient solution):
index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval is_greater_1k=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(is_greater_1k) AS last_is_greater_1k
| eval is_new_greater_1k_group=if(isnull(last_is_greater_1k) OR (is_greater_1k=0 AND last_is_greater_1k=1), 1, 0)
| streamstats sum(is_new_greater_1k_group) AS greater_1k_group
| streamstats count AS files_in_greater_1k_group BY greater_1k_group
| search files_in_greater_1k_group>=3
Thank you @micahkemp for the hint. I have modified the above query and able to fix the problem. The below query is working query where 3 consecutive files > 1KB size can be found -
index=idx1 sourcetype=src1
| eval raw_len=(len(_raw)/1024)
| stats sum(raw_len) as filesize by source
| eval filesize_1=if(filesize>1, 1, 0)
| streamstats current=f window=1 last(filesize_1) AS filesize_2
| streamstats current=f window=1 last(filesize_2) AS filesize_3
|eval finalcheck=case(filesize_1=="1" AND filesize_2=="1" AND filesize_3=="1","YES")
|search finalcheck="YES"
how do you define "consecutive" files? is it in alphabetical order or files within same directory?
A quick way to count size of file if its indexed is
index=_internal source="/opt/splunk/var/log/splunk/license_usage.log" s="somefile"| stats sum(b) by s
Then you can combine saying three files within the directory > 1KB, do something etc.
Here consecutive files mean files within same directory as in below fashion. If 3 consecutive files are having output as "1" as in "Expected OUTPUT" column in table then trigger alert.
I am getting output as below -
source filesize finalcheck
C:\splunkfile\20171116-01.txt 7.521494 1
C:\splunkfile\20171116-02.txt 2.948234 1
C:\splunkfile\20171116-03.txt 0.702148
C:\splunkfile\20171116-04.txt 1.938470 1
C:\splunkfile\20171116-05.txt 0
C:\splunkfile\20171116-06.txt 1.712890 1
C:\splunkfile\20171116-07.txt 2.954095 1
C:\splunkfile\20171116-08.txt 4.965805 1
C:\splunkfile\20171116-09.txt 0.944326 1
C:\splunkfile\20171116-10.txt 1.399411 1
Expected output is as below -
source filesize Expected OUPPUT
C:\splunkfile\20171116-01.txt 7.521494 0
C:\splunkfile\20171116-02.txt 2.948234 0
C:\splunkfile\20171116-03.txt 0.702148 0
C:\splunkfile\20171116-04.txt 1.938470 0
C:\splunkfile\20171116-05.txt 0 0
C:\splunkfile\20171116-06.txt 1.712890 1
C:\splunkfile\20171116-07.txt 2.954095 1
C:\splunkfile\20171116-08.txt 4.965805 1
C:\splunkfile\20171116-09.txt 0.944326 0
C:\splunkfile\20171116-10.txt 1.399411 0