Getting Data In

Applying changes to logs whose sourcetype has been changed on a per-event basis

ehowardl3
Path Finder

I have the need to change the sourcetype of certain logs on a per-event basis, then apply further changes on the new sourcetype in props.conf after the sourcetype change. For example:

transforms.conf:

 [set_new_sourcetype]
 DEST_KEY = MetaData:Sourcetype
 REGEX = some_regex
 FORMAT = sourcetype::my:new:sourcetype

props.conf:

[old:sourcetype]
TRANSFORMS-sourcetype_change = set_new_sourcetype

[my:new:sourcetype]
SEDCMD-whatever = s/change/that/g

However, the SEDCMD does not work on the new sourcetype. If I move the SEDCMD to the original sourcetype it works. Is there a workaround for this? If not, what's the point of being able to change the sourcetype on a per-event basis if you can't do anything with it afterwards?

Thanks

0 Karma

koshyk
Super Champion

Please have a try like

props.conf

[old:sourcetype]
TRANSFORMS-sourcetype_change = set_new_sourcetype,substitute_whatever

transforms.conf

[set_new_sourcetype]
 DEST_KEY = MetaData:Sourcetype
 REGEX = some_regex
 FORMAT = sourcetype::my:new:sourcetype

  [substitute_whatever]
  REGEX = (findSomeKey)
  FORMAT = substitutedValue
  DEST_KEY = _raw
0 Karma

ehowardl3
Path Finder

Hi @koshyk, thanks for the reply. I'm not sure this will work in my case. I'm setting the sourcetype based on one string that is only common to those logs, however the string I need removed from the new sourcetype is common to the logs in both the new and old sourcetypes. The way this is written, I believe it will affect both of my sourcetypes, correct?

0 Karma

koshyk
Super Champion

That is correct. So you essentially need "almost duplicate" data in two sourcetypes.
If that's the case, in the above logic, do as two lines. Ensure that the TRANSFORM names are increasing order of ASCII . Something like below

 [old:sourcetype]
 TRANSFORMS-1sourcetype_change = set_new_sourcetype
 TRANSFORMS-2substitute_whatever = substitute_whatever
0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...