While investigating a search to find maximum cpu usage per day for a particular process, I was going to "< search|multikv|etc>|stats max(pctCPU) as MaxPctCPU by date_mday". Except, default datetime fields are not getting generated, this date_mday doesn't exist.
Odd.
If I do a simple "index=os sourcetype=ps" search, timestamp field shows empty, yet _time is populated. Of course, without timestamp, date_* doesn't get populated. Searching for all sourcetypes gives same thing.
Something specific in the app config? I haven't found it so far.
Added:
For this exercise, I have a process 'foo'. Said process 'foo' runs on a farm of hosts. Also, there can be multiple processes of 'foo' running. httpd could be an example.
So, for the first part:
index="os" sourcetype="ps" host="foohosts*" |
``multikv fields pctCPU, COMMAND, host |
search COMMAND=foo |
stats sum(pctCPU) as sumpctCPU by _time,host |
timechart avg(sumpctCPU) as avgpctCPU by host
Where we use stats to sum up the cpu hit for 'foo' for the event and then feed that to timechart, where for a particular day it will average it into 30 minute buckets. Let's say we just tell Splunk to search 'Yesterday'.
To find the maximum value of the day, we can add:
| stats max() as *
or
timechart span=1d max() as *
and for a single day, they will give you equivalent single row of data, the maximum observed value of the calculated 30 minute averages for Yesterday. Useful if you want to get daily observed maximums of the averaged buckets to, say, plot trendlines over months to get estimates of when capacity runs out.
If I tell Splunk to do the last 7 days, however, I get a different value for Yesterday's max. It's close, but not the same value. Why would it be different?
ADDED FURTHER:
Further exploring, if I used earliest and latest to define the ranges instead of the menu selector, I got more consistent results. And, credit goes to the solution to my original problem. However, just tossing this out there in case someone else runs across this behavior.
(the < pre> tags don't seem to respect the asterisks above that should be in max() and follow as )
... View more
