Deployment Architecture

Soucetype Linux_audit generating huge amounts of useless data

tbalouch
Path Finder

Hi Guys,

I'm troubleshooting excessive license usage for my Splunk cluster and it seems sourcetype = linux_audit is generating a huge amount of data and causing me to go over my license. How would I be able to disable these types of alerts on my splunk instance? Can I do this from the *Nix app or would it be better to specify in the /opt/splunk/etc/system/local/inputs.conf file of the server that is generating the huge amount of traffic? This server also happens to be the search head.

Tags (3)
0 Karma
1 Solution

doksu
SplunkTrust
SplunkTrust

To stop ingesting linux_audit events, remove (or disable) the rlog.sh scripted stanza in the Splunk_TA_nix app's local inputs.conf being deployed to your universal forwarders. If it's only a problem on one host, then yes you could duplicate Splunk_TA_nix's rlog.sh stanza with the argument 'disabled = true' in $SPLUNK_HOME/etc/system/local/inputs.conf on that particular host.

However, it's probably best to understand why so many linux_audit events are being generated because they're likely indicating a problem and so I wouldn't recommend simply removing them from Splunk. The rlog.sh script is used to resolve UIDs in auditd events to usernames, however as with any scripted input, it introduces more moving parts. I would recommend ingesting the auditd events with a simple monitor stanza on /var/log/audit/audit.log, then use the Linux Auditd app in Splunk (https://splunkbase.splunk.com/app/2642/) to figure out what's going on (it takes care of the UID resolution at search time - which is best practice).

View solution in original post

doksu
SplunkTrust
SplunkTrust

To stop ingesting linux_audit events, remove (or disable) the rlog.sh scripted stanza in the Splunk_TA_nix app's local inputs.conf being deployed to your universal forwarders. If it's only a problem on one host, then yes you could duplicate Splunk_TA_nix's rlog.sh stanza with the argument 'disabled = true' in $SPLUNK_HOME/etc/system/local/inputs.conf on that particular host.

However, it's probably best to understand why so many linux_audit events are being generated because they're likely indicating a problem and so I wouldn't recommend simply removing them from Splunk. The rlog.sh script is used to resolve UIDs in auditd events to usernames, however as with any scripted input, it introduces more moving parts. I would recommend ingesting the auditd events with a simple monitor stanza on /var/log/audit/audit.log, then use the Linux Auditd app in Splunk (https://splunkbase.splunk.com/app/2642/) to figure out what's going on (it takes care of the UID resolution at search time - which is best practice).

View solution in original post

tbalouch
Path Finder

Thanks yes I think i disabled it, however I can't stop the linux_audit attempts unless i disable the auditd service. This is very strange.

0 Karma

lukejadamec
Super Champion

I had similar trouble with the app. Out of the box it collects gobs of data, and in my case mostly duplicate data. You should check to see if it is duplicate data, and edit you inputs.conf log file monitoring section to exclude logs you don't want. I also turned of the rslog script because it creates a duplicate of the audit log.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.