when using predict fields become null

[kiamco]

so when I use the predict command my fields become null

index=summary source="summary_events_2"
orig_source=pnr
ms_level=ERROR OR ms_level=error
NOT event=no-event
| timechart span=5m sum(count) as count
| predict count as prediction algorithm=LLP future_timespan=200 holdback=0
| eval residual=count-round(prediction,0)
| streamstats window=200 current=true median(residual) as median_of_residual
| eval abs_dev=(abs(residual - median_of_residual))
| streamstats window=200 current=true median(abs_dev) as median_abs_dev
| eval upper_bound=(median_of_residual + median_abs_dev * 20)
| eval anomaly=if(residual > upper_bound,1,0)

so this is my query and I want to add

|table event, anomaly, count

but for some reason the "event" field is null. can anyone explain why?

[Sukisen1981 · Accepted solution]

Hi,

You are applying a table command after a timechart and prediction command. There is actually, no event command from your timechart - your timechart (and the subsequent fields like residual,anomaly etc.) are all based on evals of count. There is no way events are going to be captured AFTER you apply a timechart command and just use the count , based on the timechart for further processing.
For example, if i write something like this | eval event="vvvv" | table |table event, anomaly, count , then you will receive an event field with vvv as the field values, see what I mean?
It is difficult to say what actually you want under the event field , but if I just slightly modify your timechart command to something like this - |timechart span=5m sum(count) as count, values(_raw) as event and then run the rest of your query (including |table event,anomaly,count) you WILL get values under the event field (event =_raw) in this case.
My example is bad and the query takes a long time to execute, try running this for last 15 mins. What i did was select the _audit index so that you can use the code as it is, run this code for the lats 15 minutes and see the otuput index="_audit" | timechart span=5m sum(count) as count, values(_raw) as event
| predict count as prediction algorithm=LLP5 future_timespan=200 holdback=0
| eval residual=count-round(prediction,0)
| streamstats window=200 current=true median(residual) as median_of_residual
| eval abs_dev=(abs(residual - median_of_residual))
| streamstats window=200 current=true median(abs_dev) as median_abs_dev
| eval upper_bound=(median_of_residual + median_abs_dev * 20)
| eval anomaly=if(residual > upper_bound,1,0)|table event, anomaly, count

So, either you need to include what you mean by event in the timechart(your prediction for count will remain unscathed ) OR define something using an eval for the event to pick values

[kiamco]

it works!! thanks a lot for the very detailed answer

[richgalloway]

When you can't figure out where a field gets "lost", drop one command at a time from the query until the field appears. The last command dropped is the culprit.

[kiamco]

yup did that and it gets lost after the predict command in the query. I'm just wondering why or is there any way to get those fields back?