Reassigning / replacing a sourcetype based on source

[beataficek]

I am looking to replace a sourcetype using props.conf / transforms.conf so far with no luck.

props.conf

[original_sourcetype]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TIME_PREFIX = oldtimeprefix

TIME_FORMAT=oldtimeformat
pulldown_type = 1
TRANSFORMS-set_new=set_new_sourcetype

[new_sourcetype_with_new_timeformat]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_PREFIX=newtimeprefix
TIME_FORMAT=newtimeformat
pulldown_type = 1
#rename=original_sourcetype



transforms.conf

[set_new_sourcetype]
SOURCE_KEY = MetaData:Source
REGEX = ^source::var/log/path/tofile.log
FORMAT = sourcetype::new_sourcetype_with_new_timeformat
DEST_KEY = MetaData:Sourcetype

tried different REGEX's, including 

REGEX = var/log/path/tofile.log

Also tried setting it like this in props.conf

[source::var/log/path/tofile.log]
TRANSFORMS-set_new=set_new_sourcetype

I am also looking at inputs.conf, which has monitoring stanzas for all syslog traffic, perhaps some blacklisting/ whitelisting based on source can be done there.

But I am curious as to what is not working with my props/transforms.

Thanks

[PickleRick]

OK. Sourcetype rewriting is one thing but I have a hunch that you're trying to not only get a particular metadata field rewritten but would also like to reprocess whole event with the new sourcetype. And that's (depending on how much "back" you would want to go) is either much more difficult or plain impossible.

See here - https://community.splunk.com/t5/Getting-Data-In/Diagrams-of-how-indexing-works-in-the-Splunk-platfor...

The event comes through all stages of the ingestion pipeline in sequence and there is no way for an event to "go back". Even if you rewrite the sourcetype, it will still get processed further "down the street" according to the props/transforms from the original sourcetype. The only difference will be that it will be written into the index with the new sourcetype and when you search your data the search-time props for the new sourcetype will be applied.

There is one "exception" - if you use CLONE_SOURCETYPE, the event will be "forked" and a new copy with the new sourcetype will be ingested again into the pipeline but it will still be after linebreaking and - as far as I remember - after timestamp parsing.

[richgalloway]

The value of the REGEX attribute must be a valid regular expression that contains at least one capturing group.  The expression is used to match the data in the incoming event (_raw) and cannot use key specifiers such as 'source::'.  Try these transforms

[set_new_sourcetype]
SOURCE_KEY = MetaData:Source
REGEX = (/var/log/path/tofile.log)
FORMAT = sourcetype::new_sourcetype_with_new_timeformat
DEST_KEY = MetaData:Sourcetype