collect can collect events in the future, the issue is how the collect command handles _time. It will NOT use the _time field as _time. It has different behaviour depending on whether it's run as a ...
See more...
collect can collect events in the future, the issue is how the collect command handles _time. It will NOT use the _time field as _time. It has different behaviour depending on whether it's run as a scheduled saved search or an add hoc search. The docs on collect are really bad and buggy. Using addtime is also problematic. We use this process via a macro when using collect and you need specific control over _time | eval _raw=printf("_time=%d", _time)
| foreach "*"
[| eval _raw=_raw.case(isnull('<<FIELD>>'),"", ``` Ignore null fields ```
mvcount('<<FIELD>>')>1,", <<FIELD>>=\"".mvjoin('<<FIELD>>',"###")."\"", ``` Handle MV fields just in case ```
``` Concatenate the field with a quoted value and remove the original field ```
!isnum('<<FIELD>>') AND match('<<FIELD>>', "[\[\]<>\(\){\}\|\!\;\,\'\"\*\n\r\s\t\&\?\+]"),", <<FIELD>>=\"".replace('<<FIELD>>',"\"","\\\"")."\"",
``` if no breakers, then dont quote the field ```
true(), ", <<FIELD>>=".'<<FIELD>>')
| fields - "<<FIELD>>" ]
| fields _raw
| collect index=bla addtime=f testmode=f It will ignore null fields, it will write unquoted fields when they do not contain major breakers (this allows for more performant searching using TERM() search techniques) and it will join multivalue fields together with ### You can also use this similar but slightly different approach | foreach *
[ eval _raw = if(isnull('<<FIELD>>'), _raw, json_set(coalesce(_raw, json_object()), "<<FIELD>>",'<<FIELD>>'))]
| table _time _raw or you can use output_mode=hec, which I believe will get time correct.