How to get last timestamp parsed from host?

[gfriedmann]

I'm working in an environment where the light forwarders watching windows eventlog inputs are configured for many different timezones.

As i found in another question, this is a bit of a problem because the windows eventlog inputs don't include timezone info with their timezones.

Is there a quick query that can show me the last timestamp received from a host?

I am thinking that this might not be in metrics log because that might only contain info about how much the server parsed at that time. I'm looking for "Last timestamp from all hosts" in a way that doesn't have to sort through the raw results of every single event.

[gfriedmann]

|metadata is only showing the index times for windows lightforwarders, even though the timestamps sent end up getting parsed in the past.

Best method i have found to cope so far is this search which makes it visually apparent with a source is living in the past.

earliest=-12h source="WinEventLog:Security" | fields + host | timechart limit=100 span=1h count by host

[gfriedmann]

The best i can come up with is just reviewing the metadata lastTime on hosts i /think/ should have send data recently but might have incorrect timezone extrapolation. This is difficult because i have >1000 hosts that send correct time, but may be low volume, and maybe 100 that do not.

| metadata type=hosts | convert timeformat="%y-%m-%d %H:%M:%S" ctime(lastTime) as mytime | table host,lastTime,mytime

Weird thing is that metadata's lastTime is being set to the time provided by the light forwarder... not the time of indexing on my indexer. Perhaps this is a bug. Documentation describes lastTime as the last time the indexer saw an event from the host. I read this as "the last time data was sent from this host", but maybe that can't apply for light or heavy forwarders.

[gfriedmann]

Heh. A bit of a circular problem, that. The point of this search is to help identify hosts sending with the incorrect time, as happens when you have windows lightForwarders in with multiple timezone settings.

[Genti]

then why do you not add a props.conf to tell the indexer to format time correctly: TZ=...

[gfriedmann]

It is trusting the lightForwarder and setting the index time "lastTime" as the timestamp set by the lightForwarder, which is the oldness.

[gfriedmann]

Yah, but i know that is incorrect. I have live data coming in from a windows forwarded that gets timestamped 8 hours in the past.

[Genti]

drift=0 means your data is coming in live and you do not have archived data coming in. that, is a good thing, since its kind of saying your data is being timestamped correctly..

[Genti]

this should do:

| metadata type=hosts | table host, lastTime  

Also, you might want to refer to this in case you want recentTime instead of lastTime...

.gz

[gfriedmann]

I tried evaling the difference between recentTime and lastTime, but i think it is trusting the timestamp send by the light forwarder. which is the whole problem because that timestamp is incorrect due to not including timezone.

[gfriedmann]

I was hoping that i could do this by calculating a large difference in metadata restults.

|metadata type=hosts | eval drift=lastTime - recentTime

But in my environment, it seems this difference if always 0. -_-