I am getting some strange behaviors for some of the sourcetype transforms. 70% of the events are still showing sourcetype=pfsense. The only 2 that get transformed correctly is pfsense:filterlog, pfsense:dhcpclient.
Also, there seems to be a truncation that occurs that strips out what log the event came from. For example, for all unbound events, here is what happens:
Sent from PFSense as:
Oct 17 19:23:09 unbound: [36942:1] info: validator operate: query semanticlocation-pa.googleapis.com. A IN
Indexed into splunk as (sourcetype if pfsense, not pfsense:unbound):
[36942:1] info: validator operate: query semanticlocation-pa.googleapis.com. A IN
So none of the fields for these are extracted for any sourcetype=pfsense, while sourcetype=filterlog or dhcpclient have all fields extracted properly.
I tested the sourcetyper regex from the transforms.prop on the above raw event and it pulls the correct sourectype "unbound". Does this for nginx and openvpn also.
I verified all files are there including the lookups which must be manually installed now.
Hi Rich, I am using the stock files and configuration from the publishes TA-pfsense app. I can't attach any files as I do not have enough karma points yet 🙂 . But, the file can be downloaded from the apps or from this github https://github.com/datapunctum/TA-pfsense
This morning I did remove the sed commands (SEDCMD) in the props .conf and it seems to work now . Just not sure what the published app was doing with the two sed lines. Are they needed for something else I am missing?