Using setnull and setparsing for two different sourcetypes

[inderjot_rasila]

Hello Everyone,

We have following props.conf

[sourcetypeA]
KV_MODE = json
SHOULD_LINEMERGE = false
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3Q%Z
TRUNCATE = 0
LINE_BREAKER = ([\n\r]+){
TIME_PREFIX = (\"timestamp\"[^\"]+\")
TRANSFORMS-set = setnull,setparsing

and transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = Regex1
DEST_KEY = queue
FORMAT = indexQueue

Using this configuration we are getting filtered data in splunk and it is working as expected.

No we have a requirement where we want to apply similar settings to another sourcetype say sourcetypeB with having different regex for [setparsing].

I have updated the props.conf as

[sourcetypeA]
KV_MODE = json
SHOULD_LINEMERGE = false
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3Q%Z
TRUNCATE = 0
LINE_BREAKER = ([\n\r]+){
TIME_PREFIX = (\"timestamp\"[^\"]+\")
TRANSFORMS-set = setnull,setparsing

[sourcetypeB]
KV_MODE = json
SHOULD_LINEMERGE = false
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3Q%Z
TRUNCATE = 0
LINE_BREAKER = ([\n\r]+){
TIME_PREFIX = (\"timestamp\"[^\"]+\")
TRANSFORMS-set = setnull,setparsing1

Transforms.conf has been modified as:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = Regex1
DEST_KEY = queue
FORMAT = indexQueue

[setparsing1]
REGEX = Regex2
DEST_KEY = queue
FORMAT = indexQueue

After applying these setting I see data only getting indexed for SourcetypeB and no data for SourcetypeA.

Could anyone please help what do i need to change to get data for both sourcetypes A and B. I have tried multiple combinations but only getting data for one sourcetype at one time

Regards,
Inderjot

[gurlest]

The only thing I can think to try would be to have two totally different, yet exactly the same 'setnull' stanzas in transforms.conf > setnullA for sourcetypeA and setnullB for sourcetypeB each with their own 'setparsing' as you have done already.

[woodcock]

The only thing that makes sense is that your Rregex1 is incorrect and never matches.

[inderjot_rasila]

Regex1 is just a sample here we have actual application name which matches but only data from one sourcetype gets ingested at once