All Apps and Add-ons

Splunk App for Unix and Linux: How to extract user field from /var/log/secure?

Path Finder

I have seen that Splunk App for Unix extracts the user field from /var/log/secure logs. This appears to be working fine on my desktop Linux workstation. If you look below, you will see a list of the settings>data inputs>local inputs>files and directories.

If you look at the last line, it indicates it is monitoring /var/log/secure, has source type linux_secure, and app is SA-nix. I believe this is what is extracting the user field out of the /var/log/secure logs.

I tried doing the same thing on another server with the Splunk App for Linux and do not see this line in the inputs, only the line with /var/log. I assumed it was configured by the Splunk App for *nix. Can you tell me what I am doing wrong? I tried manually adding it, but it does not provide SA-nix as an application option.

Also, would like to be able to do this processing from forwarded logs from other servers. We have the logs forwarded via the rsyslog.conf file on port 514 from multiple servers, and are not using universal forwarders. Can you tell me how to configure the servers so that the user field will be extracted?

Full path to your data 
Set host 
Source type 
Set the destination index 
Number of files 
App 
Status 
Actions
$SPLUNK_HOME/etc/splunk.version 
Constant Value  splunk_version  _internal   1   system  Enabled | Disable 

$SPLUNK_HOME/var/log/introspection 
Constant Value  Automatic   _introspection  15  introspection_generator_addon   Enabled | Disable 

$SPLUNK_HOME/var/log/splunk 
Constant Value  Automatic   _internal   43  system  Enabled | Disable 

$SPLUNK_HOME/var/spool/splunk 
Constant Value  Automatic   default         system  Enabled | Disable 

$SPLUNK_HOME/var/spool/splunk/...stash_new 
Constant Value  stash_new   default     1   system  Enabled | Disable 

/Library/Logs 
Constant Value  Automatic   os      Splunk_TA_nix   Disabled | Enable 

/etc 
Constant Value  Automatic   os      Splunk_TA_nix   Disabled | Enable 

/home/.../.bash_history 
Constant Value  bash_history    os      Splunk_TA_nix   Disabled | Enable 

/root/.bash_history 
Constant Value  bash_history    os      Splunk_TA_nix   Disabled | Enable 

/var/adm 
Constant Value  Automatic   os      Splunk_TA_nix   Disabled | Enable 

/var/log 
Constant Value  Automatic   os      Splunk_TA_nix   Disabled | Enable 

/var/log/secure 
Constant Value  linux_secure    default         SA-nix  Disabled | Enable 
Delete
0 Karma
1 Solution

Legend

First, you should make sure that your "secure" logs have the sourcetype of "linux_secure". This will give you some of the field extractions that you want automatically. For the additional fields that you want, you could try the Splunk Field Extractor. You will find it in the GUI under Settings » Fields » Field extractions. If you want, you can even paste the regular expression from the EXTRACT statement below into the Field Extractor.

If you want to do this manually: For the user field, on your search heads (or indexers if you don't have search heads), you could add the following to props.conf

[source::/var/log/secure]
EXTRACT-secureuser=for(?:\sinvalid user)?\s(?<user>\S+)

This is just a starting point.
Finally, you could look the in the SA-nix app or the Splunk_TA_nix app for the field extractions that you want - enable them and make them global. Or copy them and tweak them if you need to.

View solution in original post

Legend

First, you should make sure that your "secure" logs have the sourcetype of "linux_secure". This will give you some of the field extractions that you want automatically. For the additional fields that you want, you could try the Splunk Field Extractor. You will find it in the GUI under Settings » Fields » Field extractions. If you want, you can even paste the regular expression from the EXTRACT statement below into the Field Extractor.

If you want to do this manually: For the user field, on your search heads (or indexers if you don't have search heads), you could add the following to props.conf

[source::/var/log/secure]
EXTRACT-secureuser=for(?:\sinvalid user)?\s(?<user>\S+)

This is just a starting point.
Finally, you could look the in the SA-nix app or the Splunk_TA_nix app for the field extractions that you want - enable them and make them global. Or copy them and tweak them if you need to.

View solution in original post

Path Finder

turned out to be fast mode. when I changed to smart mode the fields were extracted properly. Thanks!

0 Karma

Path Finder

Thanks. However I do have one configuration that extracts the fields. On the second one it does not. It seems to be built into the *nix app, but not clear why it is not working. I would rather not create field extractors if they already exist.

The one that works has this configuration for file and directory local data inputs. I tried to set a new input that looks like this on the system that was not working but SA-nix was not one of the app choices.

path set host source type dest index app status
/var/log/secure constant value linux_secure default SA-nix enabled

0 Karma

Legend

Look at the permissions for the field extractions - are they the same in both configurations?

Is SA-nix a downloadable app from splunkbase? If doesn't appear on one of the configurations, it probably isn't installed on that box...

0 Karma