I have done a search as below to create a table in Dashboard to list the top 20 users that upload files the most to cloud storage services and their accessed cloud storage service URLs then get the number of file uploads for each user base on that listed 20 users and theirs accessed URLs.
There is a problem that the search shows different results sometimes but when I rerun the search it will return the same result which is probably the correct one even though I do not change anything.
Has anyone seen the same symptom like this so far? Is there anything in my queries that possibly affects the search or there might be a cache problem?
(time range:last month)
index=proxy sourcetype="XXX" filter_category="File_Storage/Sharing"
[ search index=proxy sourcetype="XXX" filter_category="File_Storage/Sharing"
| eval end_time=strftime(_time, "%Y-%m-%d %H:%M:%S")
| eval bytes_in=bytes_in/1024/1024/1024
| eval bytes_in=round(bytes_in, 2)
| table end_time,user,url,bytes_in
| sort - bytes_in
| head 20
| fields user url ]
| eval end_time=strftime(_time, "%Y-%m-%d %H:%M:%S")
| eventstats count(eval(bytes_in>0)) as Number_File_Uploads by user url
| table end_time,user,src,src_remarks01,url,bytes_in,Number_File_Uploads
| eval bytes_in=bytes_in/1024/1024/1024
| eval bytes_in=round(bytes_in, 2)
| sort - bytes_in
| head 20
| rename "end_time" as "Access date and time", "src" as "IP address", "src_remarks01" as "Asset information", "bytes_in" as "BytesIn(GB)"
Subsearches are limited to 50,000 events - if you have more than 50,000 events, your search can give odd results.
In your case, do you actually need a subsearch? Try this
index=proxy sourcetype="XXX" filter_category="File_Storage/Sharing"
| eval end_time=strftime(_time, "%Y-%m-%d %H:%M:%S")
| eval bytes_in=bytes_in/1024/1024/1024
| eval bytes_in=round(bytes_in, 2)
| eventstats count(eval(bytes_in>0)) as Number_File_Uploads by user url
| table end_time,user,src,src_remarks01,url,bytes_in,Number_File_Uploads
| sort - bytes_in
| head 20
| rename "end_time" as "Access date and time", "src" as "IP address", "src_remarks01" as "Asset information", "bytes_in" as "BytesIn(GB)"
Subsearches are limited to 50,000 events - if you have more than 50,000 events, your search can give odd results.
In your case, do you actually need a subsearch? Try this
index=proxy sourcetype="XXX" filter_category="File_Storage/Sharing"
| eval end_time=strftime(_time, "%Y-%m-%d %H:%M:%S")
| eval bytes_in=bytes_in/1024/1024/1024
| eval bytes_in=round(bytes_in, 2)
| eventstats count(eval(bytes_in>0)) as Number_File_Uploads by user url
| table end_time,user,src,src_remarks01,url,bytes_in,Number_File_Uploads
| sort - bytes_in
| head 20
| rename "end_time" as "Access date and time", "src" as "IP address", "src_remarks01" as "Asset information", "bytes_in" as "BytesIn(GB)"
I tried with your search which is thought should work too but it shows the number of downloads 0, meanwhile with the previous search it shows with number >0. (The other columns's values are same between both searches which is no problem)
Sorry I actually wanted it to be bytes_out>10000000 so I changed it a little bit with your search and it works!
Many thanks. Accepted your solution.
| eval bytes_out=bytes_out/1024/1024/1024
| eventstats count(eval(bytes_out>10000000/1024/1024/1024)) as Number_File_Uploads by user url
| table end_time,user,src,src_remarks01,url,bytes_out,Number_File_Uploads
| sort - bytes_out
| eval bytes_out=round(bytes_out, 2)
| head 20
Please show the exact search you are using to give this result