i don't have an active license but i want to file a bug report for this logic string/number issue i noticed in both "search and where"
| metadata type=hosts index=_internal
| stats max(recentTime) as latest by host
| eval last_heartbeat=strftime(latest,"%F %T")
| eval time_now=now()
| eval "last_log_seconds"= ( time_now - latest )
| stats count by host last_log_seconds, last_heartbeat
| where last_log_seconds > "3600"
| fields - count
run that in your splunk environment and you'll see that it returns invalid results, but when you remove the quotes around the number it works fine.
| metadata type=hosts index=_internal
| stats max(recentTime) as latest by host
| eval last_heartbeat=strftime(latest,"%F %T")
| eval time_now=now()
| eval "last_log_seconds"= ( time_now - latest )
| stats count by host last_log_seconds, last_heartbeat
| where last_log_seconds > 3600
| fields - count
Splunk Cloud
Splunk Version
7.0.8.5
Splunk Build
c3e02dedf40a
@mattlucas719 this is not a bug, this is an expected behavior of number represented as string. For your use case you should be comparing last_log_seconds>3600
.
Try out the following two run anywhere searches.
1) Sorts by numeric data i.e. 1,2,10,20
| makeresults
| eval data="1,10,2,20"
| makemv data delim=","
| mvexpand data
| eval data_string="str".data
| sort data
2) Sorts by string data i.e. 1,10,2,20
| makeresults
| eval data="1,10,2,20"
| makemv data delim=","
| mvexpand data
| eval data_string="str".data
| sort data_string
On similar lines as above if you apply
filter | search data>=2
, it will do numeric filter, returning 2,10,20
:
| makeresults
| eval data="1,10,2,20"
| makemv data delim=","
| mvexpand data
| search data>=2
and filter | search data>="2"
, will do string filter, returning 2,20
:
| makeresults
| eval data="1,10,2,20"
| makemv data delim=","
| mvexpand data
| search data>="2"
Hope this clarifies behavior.
PS: While it does not matter much with metadata command however, as a performance optimization suggestion, you should apply the filter before stats. Try the following search with metadata:
| metadata type=sourcetypes index=_internal
| stats max(recentTime) as last_heartbeat by sourcetype
| eval "last_log_seconds"= ( now() - last_heartbeat )
| search last_log_seconds > 3600
| stats count by sourcetype, last_log_seconds, last_heartbeat
| fields - count
| fieldformat last_heartbeat=strftime(last_heartbeat,"%F %T")
If you are maintaining a lookup/kvstore of all hosts, you can refer to Splunk Documentation to do something similar using tstats and addinfo command as well (example uses expected_hosts
lookup):
| tstats latest(_time) as latest_time where index=_internal by host
| addinfo
| eval latest_age = info_max_time - latest_time
| fields - info_*
| inputlookup append=t expected_hosts
| fillnull value=9999 latest_age
| dedup host
| where latest_age > 42