Splunk Search

Strategy to search entire estate for unique servers writing to audit.log

adamnealis
Observer

Is there a way to get the last time a host touched a file, within a certain period, e.g. earliest=-24h?

We got a request to do a "spot check" of 20 - 30 servers from a list of 720 servers, which according to someone's report run against splunk, have not written to /var/log/audit/audit.log in 24 hours.

I think it's boring to manually ssh to servers and collect ls -l ouput, and so I thought it might be nice to ask splunk which servers it has entries for in the audit.log file over the last 24h, then compare that list with the provided list in order to check how good their report is.

I have to use the GUI. First attempt (in fast mode). Tested with earliest=-1m

index=X OR index=Y earliest=-24h source=/var/log/audit/audit.log | table host | dedup host

As I watch the -24h query, I thought I'd ask if there are some more sane strategies to reduce the load.

In *nix terms I would simply connect to a server and check the ctime of the file.

The above query is just looking for all events in a 24h period, then depuping the list of servers. This seems a case of almost pure BFI. Good thing I am using index= and source=...

Labels (1)
Tags (3)
0 Karma

tscroggins
Influencer

@adamnealis 

To see the last timestamp extracted from audit.log by host over a given time range:

| tstats max(_time) as _time where index IN (X Y) source=/var/log/audit/audit.log by host

You can add earliest and latest constraints directly to the where clause if desired.

To see that last time a host forwarded audit.log data to Splunk, summarize the _indextime field:

| tstats max(_indextime) as _time where index IN (X Y) source=/var/log/audit/audit.log by host

As with earliest and latest, you can add _index_earliest and _index_latest constraints to limit results by index time. Be mindful of the interplay between _index_earliest/_index_latest and any time value you've selected in the search UI and double check your results.

0 Karma
Get Updates on the Splunk Community!

App Building 101 - Build Your First App!

WATCH RECORDING NOW   Tech Talk: App Dev Edition Splunk has tons of out-of-the-box functionality, and you’ve ...

Introducing support for Amazon Data Firehose in Splunk Edge Processor

We’re excited to announce a powerful update to Splunk Data Management with added support for Amazon Data ...

The Observability Round-Up: September 2024

What’s up Splunk Community! Welcome to the latest edition of the Observability Round-Up, a monthly series in ...