Building for the Splunk Platform

help with props.conf sourcetype stanza wildards

damucka
Builder

Hello,

I would like to use the wild cards in the props.conf, sourcetype pattern.
I have similar / same configuration for several sourcetypes and would like to make the props.conf entries more general, without listing it one by one for each sourcetype.

On the example below I try to use the wildcards to forward several sourcetypes, e.g. BWP_hanatraces, ICP_abaptraces, ISP_transports to the transforms.conf and anonymize data. This does not work though and I assume the reason is not the transforms.conf part (the regex worked fine when used in the SEDCMD), but the wildcard part for the sourcetype in the props.conf.

props.conf

# Hashing sensitive data 
[(?::){0}*_traces]
TRANSFORMS-anon = anonymize-ip, anonymize-user
[(?::){0}*_transports]
TRANSFORMS-anon = anonymize-ip, anonymize-user


transforms.conf

[anonymize-user]
REGEX = ([=,>'\\":;|\s])([ICDicd]\d{3,})([,<:;|'&\\"\s])
FORMAT = $1(D\/C\/I)###########$3
DEST_KEY = _raw
REPEAT_MATCH = true

[anonymize-ip]
REGEX = ([=,\s])(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})([,:\s])
FORMAT = $1##.##.##.##$3
DEST_KEY = _raw
REPEAT_MATCH = true

Could you please advise?

Kind Regards,
Kamil

Tags (1)
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @damucka,

I have tested below config with sourcetype test_abctraces in my lab environment with SEDCMD and it is working fine. Also SEDCMD is easy compare to props and transforms, are there any reason not to use SEDCMD ?

[(?::){0}*_*traces]
0 Karma

damucka
Builder

Hi @harsmarvania57,

Thank you.
I guess this is my mistake, it should be

[(?::){0}*_*traces]

and not 

[(?::){0}*_traces]

The reason for moving the anonymization from SEDCMD to transforms.conf is that I want also to clone some entries from one index (mlbso) to another (mlbso_changelog) based on the regex in transforms.conf. The entries however should not be anonymized before cloning. This however does not seem to be possible, because in props.conf the SEDCMD will always be executed before the transforms.conf, meaning before cloning I will get it hashed already.

Kind Regards,
Kamil

0 Karma

hkacar
Engager

Hello,

did you have a look at these examples in this answer? What version of splunk do you use?

https://answers.splunk.com/answers/397051/can-you-regex-or-wildcard-props-sourcetype-stanzas.html

0 Karma

damucka
Builder

Thank you.
I guess the answer:

"Sourcetypes stanza in props.conf do not support regexes matching."

is clear enough.

Kind Regards,
Kamil

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...