Getting Data In

Splunk and AIDE -- How do I ignore the first line of an AIDE log file?

matthewssa
Path Finder

Right now AIDE runs a check every 5 minutes and comes back with the same results each time of files Added, Removed, or Changed. The issue is the timestamp changes and the same results are being indexed over and over even though there has been no change. I would like to prevent indexing the same log file, but Splunk sees the log as a different file because the timestamp is changing on the first line. Is there a way to prevent Splunk from indexing the AIDE logs and only index them when there is a change in the rest of the AIDE log below the timestamp?

Example AIDE log.

Start timestamp: 2016-06-11 01:53:00
Summary:
Total number of files: 1116
Added files: 0
Removed files: 1
Changed files: 3


Removed files:


removed: /var/log/aide/aideCIM.log


Changed files:


changed: /var/log/aide
changed: /var/log/aide/aide.log
changed: /var/log/aide/aide_files.log


0 Karma
1 Solution

nickhills
Ultra Champion

Are you using the AIDE-TA?
https://splunkbase.splunk.com/app/2864/#/details

From a compliance point of view (if thats something you are working towards) its probably 'correct' to re-read the alert each time it is updated. AIDE is modifying the log file, so whilst you could drop the timestamp, its still a modified log.

Your compliance officer may take the view that the issue should be properly investigated (and resolved) to quiesce the alerts, rather than 'hiding' them - but thats a conversation to have with them.

With all of that said, your on a 'hiding to nothing' by trying to FSIM the FISM log 🙂
Its common to add an exception so that you don't monitor the log files that the monitor is writing, which triggers the monitor, which updates the file, and ..... and ....

If my comment helps, please give it a thumbs up!

View solution in original post

matthewssa
Path Finder

Thanks for the response.
I did actually try to use the AIDE-TA, but the issue was having to run Splunk as root. That of course was not an option.

I took into account your consideration about bringing in the log over and over which makes since. You would want to keep seeing the alerts until you fix them. What I ended up doing was creating a TA that would parse all the events and only bring in those that matched the words changed: removed: added: This actually worked well because now we don't have to deal with all the rest of the junk lines.

0 Karma

nickhills
Ultra Champion

Are you using the AIDE-TA?
https://splunkbase.splunk.com/app/2864/#/details

From a compliance point of view (if thats something you are working towards) its probably 'correct' to re-read the alert each time it is updated. AIDE is modifying the log file, so whilst you could drop the timestamp, its still a modified log.

Your compliance officer may take the view that the issue should be properly investigated (and resolved) to quiesce the alerts, rather than 'hiding' them - but thats a conversation to have with them.

With all of that said, your on a 'hiding to nothing' by trying to FSIM the FISM log 🙂
Its common to add an exception so that you don't monitor the log files that the monitor is writing, which triggers the monitor, which updates the file, and ..... and ....

If my comment helps, please give it a thumbs up!
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...