All Apps and Add-ons

How to edit my configurations to get search time extractions to apply?

mdsnmss
SplunkTrust
SplunkTrust

I've been working through trying to get some search time extractions to apply using prop.conf and transforms.conf with REPORTS. The source is parsed on a Heavy Forwarder and I have verified the index time extractions as being correct. I have created an add-on (TA) to do search time parsing on the indexer and when searching in Splunk cannot get the extractions to apply. Here is what is in my configs:

props.conf

[tool:ssh]
REPORT-authentication = ssh-login-events, ssh-session-close, ssh-disconnect, sshd_authentication_kerberos_success, sshd_authentication_refused, sshd_authentication_tried, sshd_login_restricted, pam_unix_authentication_success, pam_unix_authentication_failure, sudo_cannot_identify, ksu_authentication, ksu_authorization, su_simple, su_authentication, su_successful, wksh_authentication, login_authentication

transforms.conf

[ssh-login-events]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?.*?(Accepted|Failed|failure|(?:Invalid user)).*?(\S+)\s+from.*?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\s+port\s+(\S+)\s+\w?\s*(ssh\d))?
FORMAT = app::$1 vendor_action::$3 user::$4 src_ip::$5 src_port::$7 sshd_protocol::$8

[ssh-session-close]
REGEX = .* ((?:session|Connection) (?:opened|closed))(?: for user (\w+))?(?: by \(uid=(\d+)\))?(?: by (\d+\.\d+\.\d+\.\d+))
FORMAT = name::$1 user::$2 user_id::$3 src_ip::$4

[ssh-disconnect]
REGEX = .* (Received disconnect) from (\w+):
FORMAT = name::$1 src_ip::$2

[sshd_authentication_kerberos_success]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authorized\s+to)\s+([^,]+)\,\s+krb5\s+principal\s+([^@]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4" src_user::"$5"

[sshd_authentication_refused]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+refused)\:.*?directory\s+\/home\/([^\/]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[sshd_authentication_tried]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+tried)\s+for\s+([^\s]+)(.*?host\=([^,]+),\s+ip=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))?
FORMAT = app::$1 vendor_action::$3 user::"$4" src_dns::"$6" src_ip::"$7"

[sshd_login_restricted]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Login\s+restricted)\s+for\s+([^:]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[pam_unix_authentication_failure]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+authentication\s+(failure)\;\s+logname\=([^\s]+)?\s+uid\=([^\s]+)?\s+euid=([^\s]+)?\s+tty=([^\s]+)?\s+ruser=([^\s]+)?\s+rhost=([^\s]+)?\s+user=([^\s]+)?
FORMAT = app::"$1" action::$2 src_user::$7 src_dns::$8 user::$9

[pam_unix_authentication_success]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+(session\s+opened)\s+for\s+user\s+([^\s]+)\s+by\s+(.*?)\(uid=\d+\)
FORMAT = app::"$1" vendor_action::"$2" user::$3 src_user::$4

[sudo_cannot_identify]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+auth\s+(could\s+not\s+identify\s+password)\s+for\s+\[([^]]+)
FORMAT = app::"$1" vendor_action::"$2" user::"$3"

[ksu_authentication]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?\'ksu\s+([^']+)\'\s+(authentication\s+failed|authenticated).*?for\s+(\w+)
FORMAT = app::$1 user::"$3" vendor_action::"$4" src_user::$5

[ksu_authorization]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?Account\s+([^:]+)\:\s+authorization\s+for\s+([^@]+).*?(failed|successful)
FORMAT = app::$1 user::"$3" src_user::"$4" vendor_action::$5

[login_authentication]
REGEX = (login)\:.*(failure).*from\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\,\s+(\S+))?
FORMAT = app::$1 action::$2 src_ip::$3 user::$5

default.meta

[]
access = read : [ * ], write : [ admin ]
export=system

Is there something I am missing or a reason these extractions are not appearing in search? Permissions issue? The props and transforms are pulled from the Splunk Add-on for Unix and Linux (Splunk_TA_nix), which is also installed on our indexers. I'm guessing the transforms could be removed in this custom TA since the REPORT will use the same names referenced in the other app, right? Any help is greatly appreciated!

0 Karma
1 Solution

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

View solution in original post

0 Karma

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...