I have the following table
Timestamp
2021-08-09 12:26:55.7852
2021-08-09 12:26:56.2278
2021-08-09 12:26:56.2278
2021-08-09 12:26:56.3939
2021-08-09 12:26:39.2861
2021-08-09 12:26:40.3430
2021-08-09 12:26:41.3482
2021-08-09 12:26:41.4832
2021-08-09 12:26:56.8794
2021-08-09 12:26:57.8846
2021-08-09 12:26:58.9398
2021-08-09 12:26:59.9450
2021-08-09 12:26:59.9700
2021-08-09 12:26:59.9700
2021-08-09 12:27:00.8201
2021-08-09 12:27:00.8401
2021-08-09 12:27:01.0352
2022-03-30 10:09:25.6406
2022-03-30 10:09:25.8007
2022-03-30 10:09:26.8109
2022-03-30 10:09:27.5961
2022-03-30 10:09:27.5961
I have extracted timestamp manually using regex instead of default timestamp. I have different device_ids. Each device_id will have logfile.
I have the following macro query, to remove the events which is decreasing(seconds value are decreasing for the same date marked as bold)
index="xxxx" source="*$Device_ID$*xxxx*"
| eval Device_ID=mvindex(split(source,"/"),5)
| rex field=_raw "(?<timestamp>[^|]+)"
| table Device_ID timestamp
| streamstats count as s_no by Device_ID
| sort 0 - s_no
| table Device_ID s_no timestamp
| streamstats current=f last(timestamp) as last_timestamp by Device_ID
| eval last_timestamp_h=last_timestamp, timestamp_h=timestamp
| eval last_timestamp=strptime(last_timestamp,"%Y-%m-%d %H:%M:%S.%4N")
| eval timestamp=strptime(timestamp,"%Y-%m-%d %H:%M:%S.%4N")
| eval diff=timestamp-last_timestamp
| eval ref=if(diff<0,last_timestamp,null)
| filldown ref
| eval ref_diff=timestamp-ref
| fillnull ref_diff value=0
| search ref_diff>=0
| fields Device_ID s_no timestamp_h
But when i try to run for all devices, some values are missing and got messed.
How can i run for each device_id separately and store the result
