All Apps and Add-ons

How to edit my configurations to get search time extractions to apply?

mdsnmss
SplunkTrust
SplunkTrust

I've been working through trying to get some search time extractions to apply using prop.conf and transforms.conf with REPORTS. The source is parsed on a Heavy Forwarder and I have verified the index time extractions as being correct. I have created an add-on (TA) to do search time parsing on the indexer and when searching in Splunk cannot get the extractions to apply. Here is what is in my configs:

props.conf

[tool:ssh]
REPORT-authentication = ssh-login-events, ssh-session-close, ssh-disconnect, sshd_authentication_kerberos_success, sshd_authentication_refused, sshd_authentication_tried, sshd_login_restricted, pam_unix_authentication_success, pam_unix_authentication_failure, sudo_cannot_identify, ksu_authentication, ksu_authorization, su_simple, su_authentication, su_successful, wksh_authentication, login_authentication

transforms.conf

[ssh-login-events]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?.*?(Accepted|Failed|failure|(?:Invalid user)).*?(\S+)\s+from.*?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\s+port\s+(\S+)\s+\w?\s*(ssh\d))?
FORMAT = app::$1 vendor_action::$3 user::$4 src_ip::$5 src_port::$7 sshd_protocol::$8

[ssh-session-close]
REGEX = .* ((?:session|Connection) (?:opened|closed))(?: for user (\w+))?(?: by \(uid=(\d+)\))?(?: by (\d+\.\d+\.\d+\.\d+))
FORMAT = name::$1 user::$2 user_id::$3 src_ip::$4

[ssh-disconnect]
REGEX = .* (Received disconnect) from (\w+):
FORMAT = name::$1 src_ip::$2

[sshd_authentication_kerberos_success]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authorized\s+to)\s+([^,]+)\,\s+krb5\s+principal\s+([^@]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4" src_user::"$5"

[sshd_authentication_refused]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+refused)\:.*?directory\s+\/home\/([^\/]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[sshd_authentication_tried]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+tried)\s+for\s+([^\s]+)(.*?host\=([^,]+),\s+ip=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))?
FORMAT = app::$1 vendor_action::$3 user::"$4" src_dns::"$6" src_ip::"$7"

[sshd_login_restricted]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Login\s+restricted)\s+for\s+([^:]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[pam_unix_authentication_failure]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+authentication\s+(failure)\;\s+logname\=([^\s]+)?\s+uid\=([^\s]+)?\s+euid=([^\s]+)?\s+tty=([^\s]+)?\s+ruser=([^\s]+)?\s+rhost=([^\s]+)?\s+user=([^\s]+)?
FORMAT = app::"$1" action::$2 src_user::$7 src_dns::$8 user::$9

[pam_unix_authentication_success]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+(session\s+opened)\s+for\s+user\s+([^\s]+)\s+by\s+(.*?)\(uid=\d+\)
FORMAT = app::"$1" vendor_action::"$2" user::$3 src_user::$4

[sudo_cannot_identify]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+auth\s+(could\s+not\s+identify\s+password)\s+for\s+\[([^]]+)
FORMAT = app::"$1" vendor_action::"$2" user::"$3"

[ksu_authentication]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?\'ksu\s+([^']+)\'\s+(authentication\s+failed|authenticated).*?for\s+(\w+)
FORMAT = app::$1 user::"$3" vendor_action::"$4" src_user::$5

[ksu_authorization]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?Account\s+([^:]+)\:\s+authorization\s+for\s+([^@]+).*?(failed|successful)
FORMAT = app::$1 user::"$3" src_user::"$4" vendor_action::$5

[login_authentication]
REGEX = (login)\:.*(failure).*from\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\,\s+(\S+))?
FORMAT = app::$1 action::$2 src_ip::$3 user::$5

default.meta

[]
access = read : [ * ], write : [ admin ]
export=system

Is there something I am missing or a reason these extractions are not appearing in search? Permissions issue? The props and transforms are pulled from the Splunk Add-on for Unix and Linux (Splunk_TA_nix), which is also installed on our indexers. I'm guessing the transforms could be removed in this custom TA since the REPORT will use the same names referenced in the other app, right? Any help is greatly appreciated!

0 Karma
1 Solution

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

View solution in original post

0 Karma

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...