Developing for Splunk Enterprise

help with props.conf sourcetype stanza wildards

damucka
Builder

Hello,

I would like to use the wild cards in the props.conf, sourcetype pattern.
I have similar / same configuration for several sourcetypes and would like to make the props.conf entries more general, without listing it one by one for each sourcetype.

On the example below I try to use the wildcards to forward several sourcetypes, e.g. BWP_hanatraces, ICP_abaptraces, ISP_transports to the transforms.conf and anonymize data. This does not work though and I assume the reason is not the transforms.conf part (the regex worked fine when used in the SEDCMD), but the wildcard part for the sourcetype in the props.conf.

props.conf

# Hashing sensitive data 
[(?::){0}*_traces]
TRANSFORMS-anon = anonymize-ip, anonymize-user
[(?::){0}*_transports]
TRANSFORMS-anon = anonymize-ip, anonymize-user


transforms.conf

[anonymize-user]
REGEX = ([=,>'\\":;|\s])([ICDicd]\d{3,})([,<:;|'&\\"\s])
FORMAT = $1(D\/C\/I)###########$3
DEST_KEY = _raw
REPEAT_MATCH = true

[anonymize-ip]
REGEX = ([=,\s])(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})([,:\s])
FORMAT = $1##.##.##.##$3
DEST_KEY = _raw
REPEAT_MATCH = true

Could you please advise?

Kind Regards,
Kamil

Tags (1)
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @damucka,

I have tested below config with sourcetype test_abctraces in my lab environment with SEDCMD and it is working fine. Also SEDCMD is easy compare to props and transforms, are there any reason not to use SEDCMD ?

[(?::){0}*_*traces]
0 Karma

damucka
Builder

Hi @harsmarvania57,

Thank you.
I guess this is my mistake, it should be

[(?::){0}*_*traces]

and not 

[(?::){0}*_traces]

The reason for moving the anonymization from SEDCMD to transforms.conf is that I want also to clone some entries from one index (mlbso) to another (mlbso_changelog) based on the regex in transforms.conf. The entries however should not be anonymized before cloning. This however does not seem to be possible, because in props.conf the SEDCMD will always be executed before the transforms.conf, meaning before cloning I will get it hashed already.

Kind Regards,
Kamil

0 Karma

hkacar
Engager

Hello,

did you have a look at these examples in this answer? What version of splunk do you use?

https://answers.splunk.com/answers/397051/can-you-regex-or-wildcard-props-sourcetype-stanzas.html

0 Karma

damucka
Builder

Thank you.
I guess the answer:

"Sourcetypes stanza in props.conf do not support regexes matching."

is clear enough.

Kind Regards,
Kamil

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!