Hi,
We're using a simplified version of the Splunk_TA_nix app (basically just a subset of the bash scripts) to collect system metrics such as CPU memory disk usages;
It works well, but the problem is that these bash commands are constantly triggering our auditd rules on the end point (which is configured to monitor for execve syscalls), at an extremely high rate. E.g. every 30 seconds, one of these bash scripts executes, which triggers ~ 20 audit events, resulting in 100+ audit records, just from the sed, awk, tee, wc etc. commands within the script, which get fed back into splunk via the other auditd app we have (the amount of data generated from this is huge - larger than normal system activities in general).
One solution we're considering is to filter out audit records with auid=-1 from the audit rules side (with the tradeoff of losing some visibility);
Another approach is to rewrite all these bash scripts in python or some compiled languages so that it doesn't trigger that many execve syscalls;
Third option is to filter out the specific commands generated by these scripts;
Does anyone else have a similar problem? What's the best way to deal with this kind of issues? Is there another good app that serves a similar purpose but does not use bash (using python / C etc. instead)?
Any info is really appreciated!
