Props.conf and Transforms.conf Locations in a Clustered Environment with dedicated Search Head

[kdougherty_e1b]

I have a Clustered Environment (Cluster Master) with a dedicated Search Head. I am having trouble determining where props.conf and transforms.conf are supposed to be placed.

The goal of the below .conf files is to regex and replace a string located in events for a specific Source Type. This cannot be done at search time (best practice) as it is sensitive information. The index that contains the applicable Source Type uses a Universal Forwarder (not Heavy Forwarder).

My files are below (changed for posting). I believe the content may matter for proper placement:

Transforms.conf:

[mask_string]
Dest_Key = _raw
Regex = regex
Format = replacement

Props.conf:

[source::splunkSourcetype]
Transforms = mask_string

We are actively using the following directories on the Cluster Master to push cluster bundles to the indexes:
- /splunk/etc/master-apps
- /splunk/etc/deployment-apps

New indexes are declared with hot/cold paths and retention in the following conf file:
- /master-apps/all_indexes/local/indexes.conf

And the monitor stanzas with source paths are declared in the following conf file:
- /deployment-apps/app_name/local/inputs.conf

I have heard suggestions in other Answers to place these .conf files in /splunk/etc/master-apps/_cluster/local on the Cluster Master and /splunk/etc/master-apps/_cluster/local on the Search Head, but I have yet to try it.

Please advise. Hopefully I have provided enough background to help solve the issue.

Thank you in advance!

[nbarbato]

Did you every get clarification on this? It seems the answer given had nothing to do with your question. I would also like to know where these .conf files are located in a distributed environment.

[isoutamo]

Hi

here is link to document where you should put those parameters in distributed environment. 
In this case just create app / TA for that log integration an deploy it with CM to the indexers just like @maciep proposed  

r. Ismo

[maciep]

A few things...

First, if you're using an indexer cluster, plan to just use master-apps on the cluster master (cm) to push config to your indexers. If that same server is also configured as a deployment server, use that to manage other components of infrastructure. But note that it is recommended (as you scale at least) that the cm is just a cm and nothing else.

Second, we typically create apps/add-ons for our configuration. For example, if I work for the ACME corporation and we onboard Application ABC, then we may create an add-on called: TA-acme_abc. In that app would we would have a local folder where we put our configuration. And we would have metadata folder where we would create a simple local.meta file. And then we would put that TA in the master-apps folder and push it out. If you also have a search-time settings in there, you could also put it in deployment-apps and push it to your search head.

Third, for your actual conf files. I'm not sure if that's a copy/paste or something you just typed up when created the post. But case matters. All of the settings on the left side of the equal sign should be in all caps. Also, I'd suggest adding the description part to your transforms settings just to be sure it's unique. Also, if you specify source:: in the stanza name, it expects the source not the sourcetype. If you want to use sourcetype, then don't specify a prefix...it's the default behavior for a stanza.

So maybe like this:

props.conf

[splunkSourcetype]
TRANSFORMS-mask_sensitive_data = mask_string

transforms.conf

[mask_string]
 DEST_KEY = _raw
 REGEX = regex
 FORMAT = replacement

Also if possible, it's helpful to have a test/dev environment....even if it's just a standalone splunk instance. That way, you can create your add-ons there, manually upload sample data and verify that it works the way you want before pushing it your infrastructure. We don't want to keep pushing to production while trying to get something working.

And one final side-note, you could also use SEDCMD in props at parse time to mask data....if you wanted.