Getting Data In

How to remove logs from specific sourcetype being indexed?

newsplunker1
Path Finder

Hi Everyone,

Im trying to stop the following index from being indexed into Splunk using the props/transforms confs  on HF but with no luck - What am i doing wrong here ? 

props.conf

[pan:userid]
TRANSFORMS-set-nullqueue=set_nullqueue

transforms.conf

[set_nullqueue]
REGEX=.
DEST_KEY=queue
FORMAT=nullQueue

 

Thank you!!

0 Karma
1 Solution

scelikok
SplunkTrust
SplunkTrust

Hi @newsplunker1,

Palo Alto TA uses sourcetype renaming. You should apply your transform to original sourcetype which is pan:log or pan_log. But as @gcusellowarned, above setting will filter everything in that case. Your regex must be specific. Please try below;

props.conf

[pan:log]
TRANSFORMS-filter_pan_userid = filter_pan_user_id

transforms.conf

[filter_pan_user_id]
REGEX = ^[^,]+,[^,]+,[^,]+,USERID,
DEST_KEY = queue
FORMAT = nullQueue

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

View solution in original post

scelikok
SplunkTrust
SplunkTrust

Hi @newsplunker1,

Palo Alto TA uses sourcetype renaming. You should apply your transform to original sourcetype which is pan:log or pan_log. But as @gcusellowarned, above setting will filter everything in that case. Your regex must be specific. Please try below;

props.conf

[pan:log]
TRANSFORMS-filter_pan_userid = filter_pan_user_id

transforms.conf

[filter_pan_user_id]
REGEX = ^[^,]+,[^,]+,[^,]+,USERID,
DEST_KEY = queue
FORMAT = nullQueue

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

newsplunker1
Path Finder

Thanks @scelikok  - I ll test it and report back 

0 Karma

newsplunker1
Path Finder

Sorry I meant sourcetype NOT index 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @newsplunker1,

using this configuration, you discard all logs with that sourcetype, so the question is: why don't you disable the related input instead log filtering after input?

the solution you describe is useful to discard selected logs not all logs.

To discard all logs from a predefined sourcetype it's easier to disable input for that sourcetype.

Ciao.

Giuseppe

0 Karma

newsplunker1
Path Finder

Hi @gcusello 

Thanks for taking the time to look into this. 

 

I could not find that sourcetype in the inputs.conf .  Splunk monitors the following file 

source="/var/log/splunk/network/paloalto/IP@/syslog.log"

my inputs.conf 

[monitor:///var/log/splunk/network/paloalto/IP@/syslog.log]
disabled = 0
host =
host_segment = 6
sourcetype = pan:log
index = test

 

I can only see the pan:userid in the props conf 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @newsplunker1,

if you want to discard all logs with the sourcetype=pan:userid and you have only this input with this sourcetype, you can simply disable this input using "disabled = 1".

Ciao.

Giuseppe

0 Karma

newsplunker1
Path Finder

This will disable all sourcetypes

pan:threat

pan:traffic

pan:system

I just want to disable or ignore the logs for pan:userid sourcetype 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @newsplunker1,

as @scelikok said, this Add-on makes a transformation changing the sourcetype.

In this case the solution is the original one to install on Indexers or (if present) on Heavy Forwarders.

on props.conf:

[pan:log]
TRANSFORMS-filter_pan_userid = filter_pan_user_id

transforms.conf:

[filter_pan_user_id]
REGEX = ^[^,]+,[^,]+,[^,]+,USERID,
DEST_KEY = queue
FORMAT = nullQueue

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...

Explore the Latest Educational Offerings from Splunk (November Releases)

At Splunk Education, we are committed to providing a robust learning experience for all users, regardless of ...

New This Month in Splunk Observability Cloud - Metrics Usage Analytics, Enhanced K8s ...

The latest enhancements across the Splunk Observability portfolio deliver greater flexibility, better data and ...