Splunk Search

when using predict fields become null

Path Finder

so when I use the predict command my fields become null

index=summary source="summaryevents2"
origsource=pnr
ms
level=ERROR OR mslevel=error
NOT event=no-event
| timechart span=5m sum(count) as count
| predict count as prediction algorithm=LLP future
timespan=200 holdback=0
| eval residual=count-round(prediction,0)
| streamstats window=200 current=true median(residual) as medianofresidual
| eval absdev=(abs(residual - medianofresidual))
| streamstats window=200 current=true median(abs
dev) as medianabsdev
| eval upperbound=(medianofresidual + medianabsdev * 20)
| eval anomaly=if(residual > upper
bound,1,0)

so this is my query and I want to add

|table event, anomaly, count

but for some reason the "event" field is null. can anyone explain why?

0 Karma
1 Solution

Champion

Hi,

You are applying a table command after a timechart and prediction command. There is actually, no event command from your timechart - your timechart (and the subsequent fields like residual,anomaly etc.) are all based on evals of count. There is no way events are going to be captured AFTER you apply a timechart command and just use the count , based on the timechart for further processing.
For example, if i write something like this | eval event="vvvv" | table |table event, anomaly, count , then you will receive an event field with vvv as the field values, see what I mean?
It is difficult to say what actually you want under the event field , but if I just slightly modify your timechart command to something like this - |timechart span=5m sum(count) as count, values(_raw) as event and then run the rest of your query (including |table event,anomaly,count) you WILL get values under the event field (event =raw) in this case.
My example is bad and the query takes a long time to execute, try running this for last 15 mins. What i did was select the _audit index so that you can use the code as it is, run this code for the lats 15 minutes and see the otuput `index="
audit" | timechart span=5m sum(count) as count, values(raw) as event
| predict count as prediction algorithm=LLP5 future
timespan=200 holdback=0
| eval residual=count-round(prediction,0)
| streamstats window=200 current=true median(residual) as medianofresidual
| eval absdev=(abs(residual - medianofresidual))
| streamstats window=200 current=true median(abs
dev) as medianabsdev
| eval upperbound=(medianofresidual + medianabsdev * 20)
| eval anomaly=if(residual > upper
bound,1,0)|table event, anomaly, count`

So, either you need to include what you mean by event in the timechart(your prediction for count will remain unscathed ) OR define something using an eval for the event to pick values

View solution in original post

0 Karma

Champion

Hi,

You are applying a table command after a timechart and prediction command. There is actually, no event command from your timechart - your timechart (and the subsequent fields like residual,anomaly etc.) are all based on evals of count. There is no way events are going to be captured AFTER you apply a timechart command and just use the count , based on the timechart for further processing.
For example, if i write something like this | eval event="vvvv" | table |table event, anomaly, count , then you will receive an event field with vvv as the field values, see what I mean?
It is difficult to say what actually you want under the event field , but if I just slightly modify your timechart command to something like this - |timechart span=5m sum(count) as count, values(_raw) as event and then run the rest of your query (including |table event,anomaly,count) you WILL get values under the event field (event =raw) in this case.
My example is bad and the query takes a long time to execute, try running this for last 15 mins. What i did was select the _audit index so that you can use the code as it is, run this code for the lats 15 minutes and see the otuput `index="
audit" | timechart span=5m sum(count) as count, values(raw) as event
| predict count as prediction algorithm=LLP5 future
timespan=200 holdback=0
| eval residual=count-round(prediction,0)
| streamstats window=200 current=true median(residual) as medianofresidual
| eval absdev=(abs(residual - medianofresidual))
| streamstats window=200 current=true median(abs
dev) as medianabsdev
| eval upperbound=(medianofresidual + medianabsdev * 20)
| eval anomaly=if(residual > upper
bound,1,0)|table event, anomaly, count`

So, either you need to include what you mean by event in the timechart(your prediction for count will remain unscathed ) OR define something using an eval for the event to pick values

View solution in original post

0 Karma

Path Finder

it works!! thanks a lot for the very detailed answer

0 Karma

SplunkTrust
SplunkTrust

When you can't figure out where a field gets "lost", drop one command at a time from the query until the field appears. The last command dropped is the culprit.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

yup did that and it gets lost after the predict command in the query. I'm just wondering why or is there any way to get those fields back?

0 Karma