Hi,
We are using Linux Auditd App to monitor and track all Audit events. One of the panels for "Anomalous Event Volume" works correctly on the Indexer(Currently on older version of 6.3.2) but shows "N/A" on the Search Head(Version 6.4.0). After doing some testing, it looks like the "predict" command being used is giving slightly different results in the latest version as compared to 6.3.2., which appears to be breaking the search. Here are the details:
Ran the following search on both Indexer and the SH
| tstats count WHERE [|inputlookup auditd_indicies] [|inputlookup auditd_sourcetypes] BY _time span=1h | predict count as prediction upper95=upper lower95=lower future_timespan=0
On Indexer(Version 6.3.2)
On Search Head(Version 6.4.0)
On the Search Head, the command adds "prediction" for both lower and upper column, which in turn breaks the subsequent eval function for range as it is still expecting columns "lower" and "upper". Here is the complete command used for this panel:
| tstats count WHERE [|inputlookup auditd_indicies] [|inputlookup auditd_sourcetypes] BY _time span=1h | predict count as prediction upper95=upper lower95=lower future_timespan=0 | eval range=upper-lower | eval difference=case(count>lower AND count<upper, 0, count<lower, round((count-lower)/range,1), count>upper, round((count-upper)/range,1)) | search difference=* | table _time difference
Why is it appending text (prediction)
after the columns and is this what's causing the query to fail?
Thanks,
~ Abhi
I've successfully replicated this on splunk-6.4.1-debde650d26e.x86_64, so it appears to be a bug in the predict command. Could you please open a support ticket with Splunk?
In the interim, you could add renames between the predict and eval commands like so:
... | rename upper(prediction) as upper | rename lower(prediction) as lower | ...
I've successfully replicated this on splunk-6.4.1-debde650d26e.x86_64, so it appears to be a bug in the predict command. Could you please open a support ticket with Splunk?
In the interim, you could add renames between the predict and eval commands like so:
... | rename upper(prediction) as upper | rename lower(prediction) as lower | ...
Hello
Turns out this is not a bug but an intended change. I have verified that (and double checked) with the product managers. The changes were introduced in 6.4.
The documentation team are working to update the docs to ensure this is clear that there was a specific and intended change.
Thank you!!
Hi stmcmahon, could you please provide the rationale given by the PMs for changing this functionality?
I updated the app to v2.0.3 more than two weeks ago, but it's still going though the certification process.
That works perfectly. I'll open a support ticket with Splunk to report this.
Thanks again,
~ Abhi
Any update from support?
Hi - perhaps a dumb question, but if the bug is within the App, what would Splunk developers do? Or, is my understanding incorrect and the defect is actually in Core Splunk?
Please let me know
Hi stmcmahon, it's certainly a bug with core Splunk and not the app, but I do appreciate Abhi's raising the question because then other customers experiencing the issue after upgrading to Splunk 6.4.1 can discuss here. It's definitely not related to the app because the issue can be replicated by using the predict command against any dataset.
Hi stmcmahon,
It appears that the app's search is breaking because now the upper and lower columns have additional text (prediction) being appended which was not there in previous versions. I believe if we could find out whether this is intended functionality change or not, then that might help us answer the question.
~ Abhi