Hello
I have some directories that I need to monitor. Using updated inputs for the TA_nix app I am adding syslog/linux:audit data is specific paths. It mostly works as expected BUT I had a few outliers.
Heres the basic directory structure:
/var/log is standard BUT the messages coming from other hosts goes to a path
/var/log/remote
in this path is the 2 types of logs: syslog and linux:audit as well as .bz2 which we never want indexed from any path.
/var/log/remote/202/02/<environment_name>/messages/<files>
/var/log/remote/202/02/<environment_name>/audisp/<files>
within each one of these is an archive directory as well, it contains files being written to and .bz2 which we never want indexed from any path.
/var/log/remote/202/02/<environment_name>/messages/archive/<files>
/var/log/remote/202/02/<environment_name>/audisp/archive/<files>
So the inputs I created looks like this:
[monitor:///var/log]
whitelist=(\.log|log$|messages|secure|auth|mesg$|cron$|acpid$|\.out)
blacklist=(lastlog|anaconda\.syslog|\.bz2$|audisp|\_audisp.log|\audisp.log\-)
index=nix_os
disabled = 0
[monitor:///var/log/remote/*]
whitelist=(messages|\_messages\.log|_messages\.log\-)
blacklist=(\.bz2$|audisp|\_audisp.log|\audisp.log\-)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/*]
whitelist=(audisp|\_audisp.log|\audisp.log\-)
blacklist=(\.bz2$|\_messages\.log|_messages\.log\-)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
What I have found is that there are files with the sourcetype set as the filename, which it should be either syslog or linux:audit since the path is:
/var/log/remote/2020/02/corp/messages/archive/hostname.domain.com_messages.log-20200206
got the sourcetype set to the file name:
hostname.domain.com_messages.log-20200206
Also these did not index:
/var/log/remote/2020/02/corp2/audisp/archive/:
<ip-hidden>_messages_audisp.log-20200204
<ip-hidden>_messages_audisp.log-20200205
<ip-hidden>_messages_audisp.log-20200206
Can anyone tell me:
1.Why did the messages file
hostname1234.domain.com_messages.log-20200206
get the sourcetype set to the file name (some are set to "too-small" as well)
sourcetype=hostname1234.domain.com_messages or sourcetype=hostname1234.domain.com_messages-too_small
Why didnt the /audisp directory and the corresponding files index? For example:
/var/log/remote/2020/02/corp2/audisp/archive/<ip-hidden>_messages_audisp.log-20200204
Thanks for you assistance
You can't have (as per your original post) two monitors on the same path:
[monitor:///var/log/remote/*]
whitelist=(messages|\_messages\.log|_messages\.log\-)
blacklist=(\.bz2$)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/*]
whitelist=(audisp|\_audisp.log|\audisp.log\-)
blacklist=(\.bz2$)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
Only one of these will ever take effect - the first one I think, which is why your audit logs were not correctly picked up.
Your latest comment has the correct approach:
[monitor:///var/log/remote/.../messages]
whitelist=(archive|_messages.log|_messages.log-)
blacklist=(.bz2$)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/.../audisp]
whitelist=(archive|_audisp.log|\audisp.log-)
blacklist=(.bz2$)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
You probably want to associate a hostname to these logs too - is corp/corp2 the hostname?
in which case you can add host_segment = 6
to each monitor to associate them with the hostname from the 6th segment in the filename
You can't have (as per your original post) two monitors on the same path:
[monitor:///var/log/remote/*]
whitelist=(messages|\_messages\.log|_messages\.log\-)
blacklist=(\.bz2$)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/*]
whitelist=(audisp|\_audisp.log|\audisp.log\-)
blacklist=(\.bz2$)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
Only one of these will ever take effect - the first one I think, which is why your audit logs were not correctly picked up.
Your latest comment has the correct approach:
[monitor:///var/log/remote/.../messages]
whitelist=(archive|_messages.log|_messages.log-)
blacklist=(.bz2$)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/.../audisp]
whitelist=(archive|_audisp.log|\audisp.log-)
blacklist=(.bz2$)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
You probably want to associate a hostname to these logs too - is corp/corp2 the hostname?
in which case you can add host_segment = 6
to each monitor to associate them with the hostname from the 6th segment in the filename
Thanks for the response! My issues now are that:
small files in the /messages path get either the filename as the sourcetype OR they get "too_small" added to the sourcetype.
All of the linux:audit logs got indexed with the vmstat source/sourcetype.
Any ideas on how to fix/make correct for future data?
Updated my inputs to this, just in acse something was blacklisting for some reason, still not getting the audisp files and still getting too_small sourcetype and sourcetype=filename on the syslog files:
[monitor:///var/log]
whitelist=(\.log|log$|messages|secure|auth|mesg$|cron$|acpid$|\.out)
blacklist=(lastlog|anaconda\.syslog|\.bz2$)
index=nix_os
disabled = 0
[monitor:///var/log/remote/*]
whitelist=(messages|\_messages\.log|_messages\.log\-)
blacklist=(\.bz2$)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/*]
whitelist=(audisp|\_audisp.log|\audisp.log\-)
blacklist=(\.bz2$)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
I update the inputs again:
[monitor:///var/log]
whitelist=(.log|log$|messages|secure|auth|mesg$|cron$|acpid$|.out)
blacklist=(lastlog|anaconda.syslog|.bz2$)
index=nix_os
disabled = 0
[monitor:///var/log/remote/.../messages]
whitelist=(archive|_messages.log|_messages.log-)
blacklist=(.bz2$)
index=nix_os
sourcetype = syslog
disabled = 0
recursive=true
[monitor:///var/log/remote/.../audisp]
whitelist=(archive|_audisp.log|\audisp.log-)
blacklist=(.bz2$)
index=nix_os
sourcetype = linux:audit
disabled = 0
recursive=true
Found a BUNCH of my audit data in the vmstat sourcetype. How would this happen?