Hey Splunk developers, Quick question for you guys, I'm curious if it would be considered against best practices, or an anti-pattern to add your own code to the props.conf / transforms.conf file ...
See more...
Hey Splunk developers, Quick question for you guys, I'm curious if it would be considered against best practices, or an anti-pattern to add your own code to the props.conf / transforms.conf file of a Splunk supported technical addon in order to add support for your own custom sourcetypes? Specifically, The Splunk Addon for Microsoft Cloud Services. My use-case is this, I'm utilizing a "custom" event delivery mechanism to send Azure Diagnostic Logs generated by my resources in my Azure tenant to Splunk over HTTP Event Collection (HEC). The mechanism consists of an Azure Event Hub that receives diagnostic events from my resources in Azure. Downstream an Azure Function App monitors the upstream event-hub namespace for incoming events, parses those events, constructs a source-type based on the data in the parsed event, and then wraps the event payload in a HTTP request and sends it to Splunk. The code for the Azure Function App can be found here. The general source-typing logic for the function application is below. Functions that collect diagnostic log data attempt to construct a sourcetype based on the resourceId of the event. The logic for this sourcetype construction can be found in the getSourceType function in the ./helpers/splunk.js file. The following steps are used to construct the sourcetype:
A regular expression is used to extract two groups after the text /PROVIDERS
Example /PROVIDERS/MICROSOFT.RESOURCES/DEPLOYMENTS/
Periods (.) and forward slashes (/) are replaced with colons (:)
The event category is appended
Example
An event with a resourceId of /SUBSCRIPTIONS/subscription ID/RESOURCEGROUPS/group/PROVIDERS/MICROSOFT.RESOURCES/DEPLOYMENTS/FAILURE-ANOMALIES-ALERT-RULE-DEPLOYMENT-12345678 will have a sourcetype of azure:resources:deployments:administrative
If a sourcetype cannot be constructed from the event, the specified default sourcetype entered at setup will be used. Here's my dilemma, the team that administers our corporate Splunk instance would like to utilize as many TA's as possible to ensure CIM compliance of on-boarded data with as little work as possible, completely understandable. Therefore, for certain event sources I've hard-coded the source-type logic in my function app to match several of the pre-defined source-types found in the TA I linked above ("azure:monitor:aad" & "mscs:azure:security:alert" as examples). However, we're at the point where I'm trying to on-board data that does not have a pre-defined sourcetype within the TA's we're utilizing. As an example, Azure Front Door Firewall Logs which will sourcetype to "azure:network:frontdoors:frontdoorwebapplicationfirewalllog". To work around this, I've suggested to our content development team that we begin adding logic to the props.conf and transforms.conf of the TA listed above that will match the sourcetypes my function-app is sending over to Splunk and I've essentially been told no, as this isn't best practice. Is my suggestion truly against best practice, or an anti-pattern? In my mind the only extra overhead to take into consideration is merging our custom props.conf / transforms.conf with the base TA whenever an update for the TA is released. What's my best path forward here?