All Apps and Add-ons

How to edit my configurations to get search time extractions to apply?

mdsnmss
SplunkTrust
SplunkTrust

I've been working through trying to get some search time extractions to apply using prop.conf and transforms.conf with REPORTS. The source is parsed on a Heavy Forwarder and I have verified the index time extractions as being correct. I have created an add-on (TA) to do search time parsing on the indexer and when searching in Splunk cannot get the extractions to apply. Here is what is in my configs:

props.conf

[tool:ssh]
REPORT-authentication = ssh-login-events, ssh-session-close, ssh-disconnect, sshd_authentication_kerberos_success, sshd_authentication_refused, sshd_authentication_tried, sshd_login_restricted, pam_unix_authentication_success, pam_unix_authentication_failure, sudo_cannot_identify, ksu_authentication, ksu_authorization, su_simple, su_authentication, su_successful, wksh_authentication, login_authentication

transforms.conf

[ssh-login-events]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?.*?(Accepted|Failed|failure|(?:Invalid user)).*?(\S+)\s+from.*?(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\s+port\s+(\S+)\s+\w?\s*(ssh\d))?
FORMAT = app::$1 vendor_action::$3 user::$4 src_ip::$5 src_port::$7 sshd_protocol::$8

[ssh-session-close]
REGEX = .* ((?:session|Connection) (?:opened|closed))(?: for user (\w+))?(?: by \(uid=(\d+)\))?(?: by (\d+\.\d+\.\d+\.\d+))
FORMAT = name::$1 user::$2 user_id::$3 src_ip::$4

[ssh-disconnect]
REGEX = .* (Received disconnect) from (\w+):
FORMAT = name::$1 src_ip::$2

[sshd_authentication_kerberos_success]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authorized\s+to)\s+([^,]+)\,\s+krb5\s+principal\s+([^@]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4" src_user::"$5"

[sshd_authentication_refused]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+refused)\:.*?directory\s+\/home\/([^\/]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[sshd_authentication_tried]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Authentication\s+tried)\s+for\s+([^\s]+)(.*?host\=([^,]+),\s+ip=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}))?
FORMAT = app::$1 vendor_action::$3 user::"$4" src_dns::"$6" src_ip::"$7"

[sshd_login_restricted]
REGEX = (sshd)\[\d+\]\:\s+(\[[^]]+]\s+)?(Login\s+restricted)\s+for\s+([^:]+)
FORMAT = app::$1 vendor_action::"$3" user::"$4"

[pam_unix_authentication_failure]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+authentication\s+(failure)\;\s+logname\=([^\s]+)?\s+uid\=([^\s]+)?\s+euid=([^\s]+)?\s+tty=([^\s]+)?\s+ruser=([^\s]+)?\s+rhost=([^\s]+)?\s+user=([^\s]+)?
FORMAT = app::"$1" action::$2 src_user::$7 src_dns::$8 user::$9

[pam_unix_authentication_success]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+(session\s+opened)\s+for\s+user\s+([^\s]+)\s+by\s+(.*?)\(uid=\d+\)
FORMAT = app::"$1" vendor_action::"$2" user::$3 src_user::$4

[sudo_cannot_identify]
REGEX = pam_unix\(([^:]+):\w+\)\:\s+auth\s+(could\s+not\s+identify\s+password)\s+for\s+\[([^]]+)
FORMAT = app::"$1" vendor_action::"$2" user::"$3"

[ksu_authentication]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?\'ksu\s+([^']+)\'\s+(authentication\s+failed|authenticated).*?for\s+(\w+)
FORMAT = app::$1 user::"$3" vendor_action::"$4" src_user::$5

[ksu_authorization]
REGEX = (ksu)\[\d+\]\:\s+(\[[^]]+]\s+)?Account\s+([^:]+)\:\s+authorization\s+for\s+([^@]+).*?(failed|successful)
FORMAT = app::$1 user::"$3" src_user::"$4" vendor_action::$5

[login_authentication]
REGEX = (login)\:.*(failure).*from\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(\,\s+(\S+))?
FORMAT = app::$1 action::$2 src_ip::$3 user::$5

default.meta

[]
access = read : [ * ], write : [ admin ]
export=system

Is there something I am missing or a reason these extractions are not appearing in search? Permissions issue? The props and transforms are pulled from the Splunk Add-on for Unix and Linux (Splunk_TA_nix), which is also installed on our indexers. I'm guessing the transforms could be removed in this custom TA since the REPORT will use the same names referenced in the other app, right? Any help is greatly appreciated!

0 Karma
1 Solution

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

View solution in original post

0 Karma

mdsnmss
SplunkTrust
SplunkTrust

Looks like the settings eventually took. One thing I did notice in the field extractor in the UI was the sourcetype did not appear in the dropdown selector initially. I did not make any changes to the sourcetype and it eventually appeared and the extractions above began working. Not sure why it took so long as I did a "reload deploy-server" to handle the HF configs and a "apply cluster-bundle" to push out the configs to the indexers.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...