I have seen that Splunk App for Unix extracts the user field from /var/log/secure logs. This appears to be working fine on my desktop Linux workstation. If you look below, you will see a list of the settings>data inputs>local inputs>files and directories.
If you look at the last line, it indicates it is monitoring /var/log/secure, has source type linux_secure, and app is SA-nix. I believe this is what is extracting the user field out of the /var/log/secure logs.
I tried doing the same thing on another server with the Splunk App for Linux and do not see this line in the inputs, only the line with /var/log. I assumed it was configured by the Splunk App for *nix. Can you tell me what I am doing wrong? I tried manually adding it, but it does not provide SA-nix as an application option.
Also, would like to be able to do this processing from forwarded logs from other servers. We have the logs forwarded via the rsyslog.conf file on port 514 from multiple servers, and are not using universal forwarders. Can you tell me how to configure the servers so that the user field will be extracted?
Full path to your data
Set host
Source type
Set the destination index
Number of files
App
Status
Actions
$SPLUNK_HOME/etc/splunk.version
Constant Value splunk_version _internal 1 system Enabled | Disable
$SPLUNK_HOME/var/log/introspection
Constant Value Automatic _introspection 15 introspection_generator_addon Enabled | Disable
$SPLUNK_HOME/var/log/splunk
Constant Value Automatic _internal 43 system Enabled | Disable
$SPLUNK_HOME/var/spool/splunk
Constant Value Automatic default system Enabled | Disable
$SPLUNK_HOME/var/spool/splunk/...stash_new
Constant Value stash_new default 1 system Enabled | Disable
/Library/Logs
Constant Value Automatic os Splunk_TA_nix Disabled | Enable
/etc
Constant Value Automatic os Splunk_TA_nix Disabled | Enable
/home/.../.bash_history
Constant Value bash_history os Splunk_TA_nix Disabled | Enable
/root/.bash_history
Constant Value bash_history os Splunk_TA_nix Disabled | Enable
/var/adm
Constant Value Automatic os Splunk_TA_nix Disabled | Enable
/var/log
Constant Value Automatic os Splunk_TA_nix Disabled | Enable
/var/log/secure
Constant Value linux_secure default SA-nix Disabled | Enable
Delete
First, you should make sure that your "secure" logs have the sourcetype of "linux_secure". This will give you some of the field extractions that you want automatically. For the additional fields that you want, you could try the Splunk Field Extractor. You will find it in the GUI under Settings » Fields » Field extractions. If you want, you can even paste the regular expression from the EXTRACT statement below into the Field Extractor.
If you want to do this manually: For the user field, on your search heads (or indexers if you don't have search heads), you could add the following to props.conf
[source::/var/log/secure]
EXTRACT-secureuser=for(?:\sinvalid user)?\s(?<user>\S+)
This is just a starting point.
Finally, you could look the in the SA-nix app or the Splunk_TA_nix app for the field extractions that you want - enable them and make them global. Or copy them and tweak them if you need to.
First, you should make sure that your "secure" logs have the sourcetype of "linux_secure". This will give you some of the field extractions that you want automatically. For the additional fields that you want, you could try the Splunk Field Extractor. You will find it in the GUI under Settings » Fields » Field extractions. If you want, you can even paste the regular expression from the EXTRACT statement below into the Field Extractor.
If you want to do this manually: For the user field, on your search heads (or indexers if you don't have search heads), you could add the following to props.conf
[source::/var/log/secure]
EXTRACT-secureuser=for(?:\sinvalid user)?\s(?<user>\S+)
This is just a starting point.
Finally, you could look the in the SA-nix app or the Splunk_TA_nix app for the field extractions that you want - enable them and make them global. Or copy them and tweak them if you need to.
turned out to be fast mode. when I changed to smart mode the fields were extracted properly. Thanks!
Thanks. However I do have one configuration that extracts the fields. On the second one it does not. It seems to be built into the *nix app, but not clear why it is not working. I would rather not create field extractors if they already exist.
The one that works has this configuration for file and directory local data inputs. I tried to set a new input that looks like this on the system that was not working but SA-nix was not one of the app choices.
path set host source type dest index app status
/var/log/secure constant value linux_secure default SA-nix enabled
Look at the permissions for the field extractions - are they the same in both configurations?
Is SA-nix a downloadable app from splunkbase? If doesn't appear on one of the configurations, it probably isn't installed on that box...