"_time is _the_ most important field " is precisely why we don't want to use the DATETIME_CONFIG=current solution. We are still using the _time, would be nice to use it together with _indextime. We are operating at a scale too large to be fixing clocks.  When a misconfiguration is intended, we first have to catch it. We have to "account for lagging sources with our searches", which means very large time windows. Plus missing data in case of outages, so have to replay those searches to cover the outage timeframes. In any case, we are used to Splunk products being restrictive and making a lot of assumptions on how the customers should use it. We are working around exactly as you described it, just would be nice to have more options. ... View more

I can't, but at least I can catch that event with index time, correlate it with other security events and analyze to see a bigger picture. Things like that are expected in security world, and it's better to catch them with unreliable time than miss them. Being able to tell when something happened is not as critical as being able to tell it happened.  Missing such events may mean a lot of damage to the company. We are not asking for ES to be a time synchronization tool, but simply allowing to search on _indextime and _time would be incredibly useful.  ... View more

You are going to miss data if you are using event time for security alerting. Event time stamps are unreliable. We have seen event times 2 years in the future due to system clocks misconfigurations. Event delays and outages are common. Our average delay is 20 minutes, SLA for delivery is 24 hours.  If we want to run security alerting every hour to reduce the dwell time, we have to look back 24 hours instead of 1 hour. If we are running over 1K security searches, that adds up. On top of that, always a chance of missing a misconfigured clock unless we check AllTime. Using the _indextime for alerting, and event time for analyzing the events would work perfect for our use case. Unfortunately, it seems to be not feasible with all the constraints in ES, so we have to run our searches for a very large time span to make sure we account for the event delays, we have to check future times, and we have to have an outage replay protocols. Very inconvenient, I wish we could just run searches on _indextime (every hour) with a  broader _time (24 hours) (not AllTime).    ... View more