Getting Data In

Using props.conf and transforms.conf to exclude 'USERID' events in Palo Alto logs

km1986
Path Finder

Hello All, I'm trying to prevent the 'USERID' events from getting indexed by making the following changes on my Heavy Forwarder. However, after adding the TRANSFORMS-null statement and the [setnull] stanza in transforms.conf, I'm not seeing any logs getting indexed at all. Any guidance is appreciated

inputs.conf

[monitor:///var/log/palo]
disabled = false
sourcetype = pan:traffic

 

props.conf

[pan:traffic]
TRANSFORMS-null = setnull
TZ = America/New_York
TRANSFORMS-host = paloalto-host
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
disabled = false
pulldown_type = true

 

transforms.conf

[paloalto-host]
SOURCE_KEY = _raw
FORMAT = host::$1
DEST_KEY = MetaData:Host

[setnull]
REGEX = ^(?:[^,\n]*,){3}USERID
DEST_KEY = queue
FORMAT = nullQueue

Labels (3)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @km1986,

it's difficoult to see you regex: please use the "Insert/Edit Code Sample" button when you have code:

Anyway, the problem is that all the logs are filtered, is it correct?

you could try using a simpler regex

REGEX = USERID

and

a more complete props and transforms:

pros.conf:

[pan:traffic]
TRANSFORMS-set= setparsing,setnull

transforms.conf

[setnull]
REGEX = USERID
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

Then I see that the host override is missing of a command:

[paloalto-host]
REGEX = .
FORMAT = host::$1
DEST_KEY = MetaData:Host

Ciao.

Giuseppe

View solution in original post

0 Karma

km1986
Path Finder

Hey @gcusello 

The files are located on the Heavy Forwarder (/var/log/palo). I have double-checked the regex, it seems to be fine. None of the logs are getting indexed at all, even the ones not of 'USERID' type, which is why I was thinking if something is wrong in the props/transforms?

 

Below is a sample:

Sep 7 03:29:28 ttt-tt-ttt-9 1,2020/09/07 03:29:28,000000000000000,USERID,end,2304,2020/09/07 03:29:18,172.17.132.5,172.17.130.68,0.0.0.0,0.0.0.0,tttttt-tttttttt,,,dns,vsys1,trust,trust,ethernet1/2,ethernet1/2,default,2020/09/07 03:29:18,386215,1,50473,53,0,0,0x64,udp,allow,260,102,158,2,202 0/09/07 03:28:47,0,any,0,10906416,0x8000000000000000,ttt-tt-ttttt-ttttt,ttt-tt-ttttt-ttttt,0,1,1,aged-out,324,327,0,0,,tttttttttttt,from-policy,,,0,,0,,N/A,0,0,0,0,tttttttt-tttt-tttt-tttt-tttttttttttt,0

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @km1986,

it's difficoult to see you regex: please use the "Insert/Edit Code Sample" button when you have code:

Anyway, the problem is that all the logs are filtered, is it correct?

you could try using a simpler regex

REGEX = USERID

and

a more complete props and transforms:

pros.conf:

[pan:traffic]
TRANSFORMS-set= setparsing,setnull

transforms.conf

[setnull]
REGEX = USERID
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

Then I see that the host override is missing of a command:

[paloalto-host]
REGEX = .
FORMAT = host::$1
DEST_KEY = MetaData:Host

Ciao.

Giuseppe

0 Karma

sureshkumaar
Path Finder

Hi @gcusello - Will this work if we give some more values to be considered for indexing in transforms.conf?

[setparsing]
REGEX = systemd | auditd | CROND
DEST_KEY = queue
FORMAT = indexQueue

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sureshkumaar ,

it isn't a good idea to attach a new question to a closed question, even if on the same topic: it's always better to open a new one to have a surely faster and probably better answer to your question.

Anyway, if the regex that you're using matches all the events to filter, it's correct and you can use it.

Ciao.

Giuseppe

km1986
Path Finder

Thanks @gcusello, this worked. I think I had an issue with the splunk test instance which was not indexing logs properly since I was seeing issues with some other logs as well.

I spun up a fresh instance and tried it and it worked.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @km1986,

Good for you.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉

gcusello
SplunkTrust
SplunkTrust

HBi @km1986,

wher do you located the files?

thet must stay on the Indexers or (when present) on Heavy Forwarders.

Are you sure about the regex? if you share a sample of your logs  I could help you in this check.. 

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...