Writing a search that will catch sourcetypes that have logged in x amount of time

[brent_weaver]

We are wokring on coming up with a methd to detect data that stops coming in based on sourcetype. I believe I will want to look at the metadata like:

|metadata index=index_name type=sourcetypes 

This yields a nice list (based on index of course). How do I structire this metadata query to pick out sourcetypes that have not reported in in say 1 day? I would assume I woudl leverage the epoch timestamp and do some simple math?

Any help is MUCH apprecaited!

[dineshraj9 · Accepted solution]

You can take a difference of current time and lastTime field and check if its greater than a day -

 | metadata index=index_name type=sourcetypes | eval diff=now() - lastTime | where diff > 86400 | convert ctime(lastTime)

Similar query be created to identify hosts which have stopped reporting (even if no non-internal logs are generated internal logs are continuously read from hosts) -

 | metadata type=hosts | eval diff=now() - lastTime | where diff > 86400 | convert ctime(lastTime)

[brent_weaver]

Thank you for the great info, and they both worked perfectly. One last question, is tstats any more efficient? We have also been playing with:

| tstats latest(_time) as latest where index=index* by sourcetype | where latest < relative_time(now(), "-60m") | eval "last seen"=strftime(latest,"%m/%d/%y %H:%M:%S") | fields – latest

Which is faster? I would suspect meta, but if I knew I woudl not bother you with this question! 🙂

Thanks!

[dineshraj9]

Metadata query are not time dependent, but tstats queries are. Also if you want to find a sourcetype that had stopped reporting last week, then the time range for tstats command has to be greater than 7days.
Metadata queries are faster in this respect, but they can also be slow if you have huge number of results(sourcetypes or hosts).