Splunk Search

Why it changes the sourcetype for all sourcetype

ss026381
Communicator

I am trying to change the sourcetype of all events that are not from sourcetype starting with xyz. I am using following configuration

transforms.conf

[changeSourcetype]
SOURCE_KEY = MetaData:Sourcetype 
REGEX = ^(?!xyz).+
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::wrong:sourcetype

props.conf

  [default]
  TRANSFORMS-changesourcetype=changeSourcetype   

The above changes all the sourcetypes to wrong:sourcetype

Tags (2)
0 Karma

tiagofbmm
Influencer

Here is the test I made for your use case.

I just didn't use the sourcetype as SOURCE_KEY. Instead, I;m using _raw for simplicity in the mock up construction:

inputs.conf

[monitor::///home/tiago/Desktop/test.txt]
index=xyz
sourcetype=xyz

props.conf

[xyz]
TRANSFORMS-xyz = override_xyz
DATETIME_CONFIG =
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = true

transforms.conf

[override_xyz]
REGEX=^(?!xyz).+
SOURCE_KEY = _raw
DEST_KEY= MetaData:Sourcetype
FORMAT= sourcetype::overriden_sourcetype

Here is my testfile:

aqwdqwd
asdq
aaaxyz
xyz

And here are the results

3/19/18    1:01:17.000 PM   
xyz
host =  tiagoubuntu source =    /home/tiago/Desktop/test.txt sourcetype =   xyz
3/19/18   1:01:10.000 PM    
aaaxyz
host =  tiagoubuntu source =    /home/tiago/Desktop/test.txt sourcetype =   overriden_sourcetype
3/19/18  1:01:02.000 PM 
asdq
host =  tiagoubuntu source =    /home/tiago/Desktop/test.txt sourcetype =   overriden_sourcetype
3/19/18  1:00:51.000 PM 
aqwdqwd
host =  tiagoubuntu source =    /home/tiago/Desktop/test.txt sourcetype =   overriden_sourcetype

So my diagnose is your problem is not the regex itself, but probably the value you are having in your sourcetype at first.

Can you make a similar test to this one and give me feedback?

tiagofbmm
Influencer

Please let me know if the answer was useful for you. If it was, accept it and upvote. If not, give us more input so we can help you with that

0 Karma

ss026381
Communicator

Thanks @tiagofbmm it's helpful but still didn't solve my problem.

0 Karma

tiagofbmm
Influencer

Check if your regex is PCRE compliant to match want you want

0 Karma

ss026381
Communicator

Thanks for the reply, looks like it is PCRE compliant
https://regex101.com/r/eMo07d/1

The weird part is when I change it to filter sourcetype starts with xyz ^(xyz).+, it works.

0 Karma

tiagofbmm
Influencer

I agree it should work according to regex101. Another point here is why the negative lookup? Wouldn't a straightforward regex match what you desire?

0 Karma

ss026381
Communicator

I need to change the sourcetype of an event if it's not starting with xyz. If there any other way I can achieve this, I'll be happy to do this?

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...