All Apps and Add-ons

Truncation of some sourcetypes

gauthig
New Member

I am getting some strange behaviors for some of the sourcetype transforms. 70% of the events are still showing sourcetype=pfsense. The only 2 that get transformed correctly is pfsense:filterlog, pfsense:dhcpclient.

Also, there seems to be a truncation that occurs that strips out what log the event came from. For example, for all unbound events, here is what happens:

Sent from PFSense as:
Oct 17 19:23:09 unbound: [36942:1] info: validator operate: query semanticlocation-pa.googleapis.com. A IN

Indexed into splunk as (sourcetype if pfsense, not pfsense:unbound):
[36942:1] info: validator operate: query semanticlocation-pa.googleapis.com. A IN

So none of the fields for these are extracted for any sourcetype=pfsense, while sourcetype=filterlog or dhcpclient have all fields extracted properly.

I tested the sourcetyper regex from the transforms.prop on the above raw event and it pulls the correct sourectype "unbound". Does this for nginx and openvpn also.

I verified all files are there including the lookups which must be manually installed now.

PFSense 2.4.4-RELEASE-p3
Splunk Enterprise 7.3.2

0 Karma

gauthig
New Member

Hi Rich, I am using the stock files and configuration from the publishes TA-pfsense app. I can't attach any files as I do not have enough karma points yet 🙂 . But, the file can be downloaded from the apps or from this github https://github.com/datapunctum/TA-pfsense

This morning I did remove the sed commands (SEDCMD) in the props .conf and it seems to work now . Just not sure what the published app was doing with the two sed lines. Are they needed for something else I am missing?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The SEDCMDs appear to be cleaning up some unneeded text in the events. Perhaps your version of PFSense doesn't have that same text. At least you have it working now.

Please accept the answer to help future readers.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share the inputs.conf settings for that source, including the sourcetype, as well as the props.conf settings for the sourcetype.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!