Hi Splunk folks,
My team is seeing a pesky issue with Palo Alto logs where a small subset are not being sourcetyped into pan:traffic/threat, etc. As the pan:log is the default, we have a few logs that keep this sourcetype. We have attempted to regex and nullQueue out the remainder of the pan:log logs, but no success. When we implement this TRANSFORMS/Props.conf entry, we place it at the end as we understood the order followed a left to right priority.
Example of a log that is being sourcetyped as "pan:log", and we would want to drop. It seems as if this is fragmented from Syslog, but nonetheless, junk to us.
000-1823048e98,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2021-11-10T10:04:03.905+00:00,,,infrastructure,networking,network-protocol,3,"used-by-malware,has-known-vulnerability,pervasive-use",,dns,no,no,0
Palo Alto props.conf:
[pan_log]
pulldown_type = false
SHOULD_LINEMERGE = false
TIME_PREFIX = ^(?:[^,]*,){5}
MAX_TIMESTAMP_LOOKAHEAD = 100
TRANSFORMS-sourcetype = pan_threat, pan_traffic, pan_system, pan_config, pan_hipmatch, pan_correlation, pan_userid, pan_globalprotect, pan_decryption
We added a new nullQueue entry into the transforms.conf and then inserted the pan_discard after the last entry in the transforms above.
[pan_discard]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
Any suggestions?
Hi @danielfurtaw ,
Nullqueue entry in transforms.conf looks fine.
As far as I know, entry for TRANSFORMS- in the props.conf only matters. It executes from Right to Left.
Order does not matter in the transforms.conf file.
i.e. From the below entries, first pan_decryption is executed, then pan_globalprotect and so on.
TRANSFORMS-sourcetype = pan_threat, pan_traffic, pan_system, pan_config, pan_hipmatch, pan_correlation, pan_userid, pan_globalprotect, pan_decryption
So if you have the regexes ready for all the below transform segments.. you need to place pan_discard in the first.
TRANSFORMS-sourcetype = pan_discard ,pan_threat, pan_traffic, pan_system, pan_config, pan_hipmatch, pan_correlation, pan_userid, pan_globalprotect, pan_decryption
So after all the regexes are executed, pan_discard comes into picture and the remaining logs will be sent to nullQueue.
Try the above and let me know . If its helps, please give an upvote 🙂
Happy Splunking 🙂