Getting Data In

Issues using Linux extraction using Splunk Linux TA

mburgess97
Path Finder

 

Can someone guide me in the right direction. I have an issue with src_ip extraction using the nix splunk TA.
I see that the [syslog] stanza in props.conf contains the config below, but I'm unsure how src_ip is actually being extracted using the props and transforms code blocks below.

 

Futhermore, I'm not 100% certain what transforms is actually doing. I was trying to narrow down where the issue might be with the extraction, but having some difficultly figuring that out.  The regex seems very basic.


search: `index=ap_os_nix sourcetype=syslog`
sourcetype = `syslog`
source = `/var/log/auth`

This payload below parses incorrectly and also included the port number.
Mar 16 11:36:43 apnmls02 sshd[21198]: Received disconnect from 172.16.5.49 port 51798:11: Session closed [preauth]
`src_ip="172.16.5.49 port 51798:11"`

The payload below has parses the source IP correctly.
Mar 16 11:42:23 apcribl02 sshd[200646]: Connection closed by 172.16.5.49 port 56452
`src_ip = 172.16.5.49`


### Props for syslog sourcetype

```
###### Syslog ######
[source::....syslog]
sourcetype = syslog

[syslog]
EVENT_BREAKER_ENABLE = true

## Event extractions by type
REPORT-0authentication_for_syslog = remote_login_failure, bad-su2, passwd-auth-failure, failed_login1, bad-su, failed-su, ssh-login-failed, ssh-invalid-user, ssh-login-accepted, ssh-session-close, ssh-disconnect, sshd_authentication_kerberos_success, sshd_authentication_refused, sshd_authentication_tried, sshd_login_restricted, pam_unix_authentication_success, pam_unix_authentication_failure, sudo_cannot_identify, ksu_authentication, ksu_authorization, su_simple, su_authentication, su_successful, wksh_authentication, login_authentication
EVAL-action = if(app="su" AND isnull(action),"success",action)
REPORT-account_management_for_syslog = useradd, userdel, userdel-grp, groupdel, groupadd, groupadd-suse
REPORT-password_change_for_syslog = pam-passwd-ok, passwd-change-fail
REPORT-firewall = ipfw, ipfw-stealth, ipfw-icmp, pf
REPORT-routing = iptables
EVAL-signature = if(isnotnull(inbound_interface),"firewall",null())
REPORT-signature_for_syslog_timesync = signature_for_nix_timesync

REPORT-dest_for_syslog = host_as_dest
LOOKUP-action_for_syslog = nix_action_lookup vendor_action OUTPUTNEW action
REPORT-src_for_syslog = src_dns_as_src, src_ip_as_src
FIELDALIAS-dvc = dest as dvc

EVAL-vendor_product = if(isnull(vendor_product), "nix", vendor_product)
```

### Transforms line referenced in Props

```
[src_ip_as_src]
SOURCE_KEY = src_ip
REGEX = (.+)
FORMAT = src::"$1"
```

 

 

 

Labels (3)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

Either create your own props/transforms in application's local directory (this way if you upgrade the app your changes won't get overwritten) or create a new app (mind the conf file precedence).

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

I know it will not be of much help at this point but yes, sometimes the TA_nix is not up to speed with changes of log formats (and different kernel versions and different services versions can report events in different formats). So it's not unusual to have to adjust your extractions.

0 Karma

mburgess97
Path Finder

What would be the best practice method for dealing with this?  I assume changing the app isn't preferred due to updates overwriting changes.

Create another app/props file that looks at this specific sourcetype?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Either create your own props/transforms in application's local directory (this way if you upgrade the app your changes won't get overwritten) or create a new app (mind the conf file precedence).

richgalloway
SplunkTrust
SplunkTrust

The src_ip_as_src transform is not the only one referenced in props (but probably the most relevant).  All it is doing is copying all characters from the src_ip field to the src field.  There are other ways to do it, too.

The next thing to figure out is where the src_ip field is extracted from the raw data.  That seems to be the cause of the problem.

---
If this reply helps you, Karma would be appreciated.

mburgess97
Path Finder

That's what is confusing me... I can't find the extraction method.  I would assume it would regex, but I don't think it is.

Apologies for the elementary questions... I read the props spec - not sure what I am missing.  The only area referencing [syslog] in props is what I posted earlier.  The only other (non-sourcetype specific) area is below.

 

 

#####################
## Configuration Logs
#####################
[source::(....(config|conf|cfg|inii|cfg|emacs|ini|license|lng|plist|presets|properties|props|vim|wsdl))]
sourcetype = config_file
CHECK_METHOD = modtime

[config_file]
LINE_BREAKER = ^((?!))$
TRUNCATE = 1000000
SHOULD_LINEMERGE = false
DATETIME_CONFIG = NONE
CHECK_METHOD = modtime
KV_MODE = none
pulldown_type = true
SEGMENTATION-all = whitespace-only
SEGMENTATION-inner = whitespace-only
SEGMENTATION-outer = whitespace-only
SEGMENTATION-standard = whitespace-only
LEARN_MODEL = false
LEARN_SOURCETYPE = false

 

 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...