Following @martin_mueller's R-rated suggestion and help from R-rated app author @rfujara_splunk😉 as well as a frantic search for cheap interpolation, the following is a recipe to analyse event count.
| timechart count
| appendpipe [
| stats count
| addinfo
| eval temp=info_min_time."##".info_max_time
| fields temp count
| makemv temp delim="##"
| mvexpand temp
| rename temp as _time
] | timechart max(count) as COUNT
| fillnull
| eventstats count as TOTAL
| r "output=transform(input,FFT=Mod(fft(COUNT)),Freq=((1:TOTAL)-1)/(TOTAL*X_span))"
Application notes
You need to install the R app. See @martin_meuller's answer above.
For event counts, gaps should be interpreted as 0. The largest part of the above search is to do just that, thanks to @somesoni2's answer to my question.
The eventstats to obtain TOTAL is superficial and a waste of computation. There should be a better way to do this within R.
The above only outputs modulus of the transformation because counts are all real numbers. You can output the complex numbers by ridding Mod() from the above. (Interestingly, although Splunk lacks complex number arithmetics, its stats functions accepts complex numbers. Maybe it takes the real part and discards imaginary part as NaN.)
Freq is a dummy sequence for interpretation, expressed in hertz. You can chart over Freq , for example.
Maximum frequency you can analyse is 0.5/ span . span in both timechart calls must be equal.
Beware of an undesirable side effect of timechart used to fill gaps: It forces an extra interval.
A few F(FT)-words
As discrete Fourier transform goes, you only look at half of the output sequence (positive frequencies) when inputs are all real.
When analyzing (all-positive) event counts, output at frequency 0 is meaningless, as this component contains the strong DC bias.
fft() uses a square sampling window. Spectrum leakage could diffuse your analysis especially when dealing with black-and-white data such as event counts.
R-rated notes
Object input from Splunk is in "data frame” class. You need to “transform" it into arrays that most R functions deal with. The transform() function in the above has nothing to do with Fourier transformation. The latter is performed in fft() function.
In addition to fields you pass to R, input also passes certain Splunk internal fields as X-rated objects. In the above, X_span is span in the last stats function ( timechart ); you also have access to X_time which corresponds to _time in Splunk. (This is perhaps not limited to R app.)
The above doesn’t address how to separate data series into R arrays then output transformed objects. That will be my end goal. But it’s a good start.
... View more
