How to override source from raw data while indexing in splunk?

[mitali]

For example below is my raw data in sample.log file.

This is a |AWS| test log testing. 

The source of this file is opt/sample.log but I want to change my source from source= /opt/sample.log to source=AWS which will be extracted from raw data  while indexing in splunk.

props.conf

[log]
TRANSFORMS-sourcechange=replacedefaultsource

Transforms.conf

[replacedefaultsource]
WRITE_META = true
SOURCE_KEY = _raw
REGEX = \|(.*)\|
DEST_KEY = MetaData:Source
FORMAT= source::$1

Thank you in advance please help me.

[PickleRick]

At first glance everything seems relatively OK.

Quick sanity check:

1) Are you ingesting this file with the "log" sourcetype?

2) On which component do you have those settings?

[mitali]

1).we are using random sourcetype and not a specific sourcetype does it matter?

2) We have to forward data from UF and props.conf and transforms.conf are on indexer.

[PickleRick]

1. Yes. Your excerpt from props.conf had stanza named [log] which means it would only be applied to the events with sourcetype of "log". See the documentation for the props.conf file.

2. OK. That's good. UF doesn't have any advanced processing capabilities so all the funky stuff happens on the first "heavy" component in event's path - indexer or heavy forwarder, whichever is first.

[mitali]

We have lot of logs coming from different sources which are going to a specific directory which is than getting monitored by splunk. These log source changes everyday (everyday a new log file is created).Out of these log some of them contain AWS logs so I  need to change the source for those particular logs which has AWS logs in it.

The only way to differentiate AWS logs from other logs is that AWS contains field which has "AWS" and that was my trying to do using props and transforms.

[PickleRick]

Well... fiddling with metadata is tricky and it might cause results unexpected at this point. Especially if you're ingesting well-known data format. Especially overwriting the source field globally with a static value does mean that you're losing some level of information (in case of file-based ingestion you're forgetting which file the data was read from). So it might not be the best possible idea. You could use another means of selecting the data from AWS by - for example - defining an eventtype. But if you really want to overwrite the source, you have to point your props.conf to apply the transform defined in transforms.conf to the appropriate sourcetypes/sources/hosts. Read  the https://docs.splunk.com/Documentation/Splunk/9.0.0/Admin/Propsconf#GLOBAL_SETTINGS thoroughly.