topic Re: Linux extraction woes using Splunk Linux TA in Getting Data In

Issues using Linux extraction using Splunk Linux TA

mburgess97 — Fri, 17 Mar 2023 16:09:21 GMT

Can someone guide me in the right direction. I have an issue with src_ip extraction using the nix splunk TA.
I see that the [syslog] stanza in props.conf contains the config below, but I'm unsure how src_ip is actually being extracted using the props and transforms code blocks below.

Futhermore, I'm not 100% certain what transforms is actually doing. I was trying to narrow down where the issue might be with the extraction, but having some difficultly figuring that out. The regex seems very basic.

search: `index=ap_os_nix sourcetype=syslog`
sourcetype = `syslog`
source = `/var/log/auth`

This payload below parses incorrectly and also included the port number.
Mar 16 11:36:43 apnmls02 sshd[21198]: Received disconnect from 172.16.5.49 port 51798:11: Session closed [preauth]
`src_ip="172.16.5.49 port 51798:11"`

The payload below has parses the source IP correctly.
Mar 16 11:42:23 apcribl02 sshd[200646]: Connection closed by 172.16.5.49 port 56452
`src_ip = 172.16.5.49`

### Props for syslog sourcetype

```
###### Syslog ######
[source::....syslog]
sourcetype = syslog

[syslog]
EVENT_BREAKER_ENABLE = true

## Event extractions by type
REPORT-0authentication_for_syslog = remote_login_failure, bad-su2, passwd-auth-failure, failed_login1, bad-su, failed-su, ssh-login-failed, ssh-invalid-user, ssh-login-accepted, ssh-session-close, ssh-disconnect, sshd_authentication_kerberos_success, sshd_authentication_refused, sshd_authentication_tried, sshd_login_restricted, pam_unix_authentication_success, pam_unix_authentication_failure, sudo_cannot_identify, ksu_authentication, ksu_authorization, su_simple, su_authentication, su_successful, wksh_authentication, login_authentication
EVAL-action = if(app="su" AND isnull(action),"success",action)
REPORT-account_management_for_syslog = useradd, userdel, userdel-grp, groupdel, groupadd, groupadd-suse
REPORT-password_change_for_syslog = pam-passwd-ok, passwd-change-fail
REPORT-firewall = ipfw, ipfw-stealth, ipfw-icmp, pf
REPORT-routing = iptables
EVAL-signature = if(isnotnull(inbound_interface),"firewall",null())
REPORT-signature_for_syslog_timesync = signature_for_nix_timesync

REPORT-dest_for_syslog = host_as_dest
LOOKUP-action_for_syslog = nix_action_lookup vendor_action OUTPUTNEW action
REPORT-src_for_syslog = src_dns_as_src, src_ip_as_src
FIELDALIAS-dvc = dest as dvc

EVAL-vendor_product = if(isnull(vendor_product), "nix", vendor_product)
```

### Transforms line referenced in Props

```
[src_ip_as_src]
SOURCE_KEY = src_ip
REGEX = (.+)
FORMAT = src::"$1"
```

Re: Linux extraction woes using Splunk Linux TA

richgalloway — Thu, 16 Mar 2023 20:35:57 GMT

The src_ip_as_src transform is not the only one referenced in props (but probably the most relevant). All it is doing is copying all characters from the src_ip field to the src field. There are other ways to do it, too.

The next thing to figure out is where the src_ip field is extracted from the raw data. That seems to be the cause of the problem.

Re: Linux extraction woes using Splunk Linux TA

PickleRick — Thu, 16 Mar 2023 22:33:18 GMT

I know it will not be of much help at this point but yes, sometimes the TA_nix is not up to speed with changes of log formats (and different kernel versions and different services versions can report events in different formats). So it's not unusual to have to adjust your extractions.

Re: Linux extraction woes using Splunk Linux TA

mburgess97 — Fri, 17 Mar 2023 10:56:42 GMT

What would be the best practice method for dealing with this? I assume changing the app isn't preferred due to updates overwriting changes.

Create another app/props file that looks at this specific sourcetype?

Re: Linux extraction woes using Splunk Linux TA

mburgess97 — Fri, 17 Mar 2023 11:09:15 GMT

That's what is confusing me... I can't find the extraction method. I would assume it would regex, but I don't think it is.

Apologies for the elementary questions... I read the props spec - not sure what I am missing. The only area referencing [syslog] in props is what I posted earlier. The only other (non-sourcetype specific) area is below.

[config_file]
LINE_BREAKER = ^((?!))$
TRUNCATE = 1000000
SHOULD_LINEMERGE = false
DATETIME_CONFIG = NONE
CHECK_METHOD = modtime
KV_MODE = none
pulldown_type = true
SEGMENTATION-all = whitespace-only
SEGMENTATION-inner = whitespace-only
SEGMENTATION-outer = whitespace-only
SEGMENTATION-standard = whitespace-only
LEARN_MODEL = false
LEARN_SOURCETYPE = false

Re: Linux extraction woes using Splunk Linux TA

PickleRick — Fri, 17 Mar 2023 13:41:04 GMT

Either create your own props/transforms in application's local directory (this way if you upgrade the app your changes won't get overwritten) or create a new app (mind the conf file precedence).