How to remove logs from specific sourcetype being indexed?

[newsplunker1]

Hi Everyone,

Im trying to stop the following index from being indexed into Splunk using the props/transforms confs  on HF but with no luck - What am i doing wrong here ? 

props.conf

[pan:userid]
TRANSFORMS-set-nullqueue=set_nullqueue

transforms.conf

[set_nullqueue]
REGEX=.
DEST_KEY=queue
FORMAT=nullQueue

Thank you!!

[scelikok · Accepted solution]

Hi @newsplunker1,

Palo Alto TA uses sourcetype renaming. You should apply your transform to original sourcetype which is pan:log or pan_log. But as @gcusellowarned, above setting will filter everything in that case. Your regex must be specific. Please try below;

props.conf

[pan:log]
TRANSFORMS-filter_pan_userid = filter_pan_user_id

transforms.conf

[filter_pan_user_id]
REGEX = ^[^,]+,[^,]+,[^,]+,USERID,
DEST_KEY = queue
FORMAT = nullQueue

[newsplunker1]

Thanks @scelikok  - I ll test it and report back 

[newsplunker1]

Sorry I meant sourcetype NOT index 

[gcusello]

Hi @newsplunker1,

using this configuration, you discard all logs with that sourcetype, so the question is: why don't you disable the related input instead log filtering after input?

the solution you describe is useful to discard selected logs not all logs.

To discard all logs from a predefined sourcetype it's easier to disable input for that sourcetype.

Ciao.

Giuseppe

[newsplunker1]

Hi @gcusello 

Thanks for taking the time to look into this. 

I could not find that sourcetype in the inputs.conf .  Splunk monitors the following file 

source="/var/log/splunk/network/paloalto/IP@/syslog.log"

my inputs.conf 

[monitor:///var/log/splunk/network/paloalto/IP@/syslog.log]
disabled = 0
host =
host_segment = 6
sourcetype = pan:log
index = test

I can only see the pan:userid in the props conf 

[gcusello]

Hi @newsplunker1,

if you want to discard all logs with the sourcetype=pan:userid and you have only this input with this sourcetype, you can simply disable this input using "disabled = 1".

Ciao.

Giuseppe

[newsplunker1]

This will disable all sourcetypes

pan:threat

pan:traffic

pan:system

I just want to disable or ignore the logs for pan:userid sourcetype 

[gcusello]

Hi @newsplunker1,

as @scelikok said, this Add-on makes a transformation changing the sourcetype.

In this case the solution is the original one to install on Indexers or (if present) on Heavy Forwarders.

on props.conf:

[pan:log]
TRANSFORMS-filter_pan_userid = filter_pan_user_id

transforms.conf:

[filter_pan_user_id]
REGEX = ^[^,]+,[^,]+,[^,]+,USERID,
DEST_KEY = queue
FORMAT = nullQueue

Ciao.

Giuseppe