Sourcetype naming convention, predefined / learned sourcetypes, props / transforms share between 2 Search Head clusters

[imv327]

Hi guys,

I have several topics on the table.

1) I would like to know if you would have any advice, process or even document defining the principles of creating a naming convention when it comes to sourcetypes.
We have faced the classic issue where some of our sourcetypes had the same name and we would like to find ways to avoid that from now on.

2) Is it pertinent to add predefined / learned sourcetypes with a naming convention based on the format.? (then we could solve the point 1 with a naming convention like app_format for example). How do you technically add new predefined sourcetypes and how do you solve both the management of sourcetypes (point 1) and the management of predefined sourcetypes  ?

3) How do you share Knowledge Objects, props and transforms between 2 Search Head clusters, how do you implement a continuously synced mechanism that would keep these objects synced between both clusters ? Do we have to use Ansible deployments to perform the changes on both clusters or is there any Splunk way to achieve this synchronization in an easier way inside Splunk (via a script using REST API, command line, configuration etc) ?

[deepakc]

Point 1. This will help in sourcetype naming conventions (People still use my_app_soucetype, meaning they use underscores, which isn't a problem, but this is the recommended naming convention, see link below. 

https://docs.splunk.com/Documentation/AddOns/released/Overview/Sourcetypes

Point 2.  As there are a plethora of data sources, many are common and at some point, you will have a custom log source and will need to create your own sourcetype.

These are some of the pretrained ones

https://docs.splunk.com/Documentation/SplunkCloud/9.1.2312/Data/Listofpretrainedsourcetypes 

 Many common sourcetypes come out of the box with the Splunk TA's so this is your starting point, these should be used and do not need to change them as they categorise the data which is important for parsing.

 For any custom data sources, you need to analyse your log source, check the format, timestamp, and other settings, and use props.conf to set the sourcetype with a naming convention standards, this makes admin life easier.  See the below link

https://lantern.splunk.com/Splunk_Platform/Product_Tips/Data_Management/Configuring_new_source_types

 Point 3.  Syncing two different SH Clusters' mean you have one Deployer for each, that’s the Splunk setup. So, you need some kind of Repo like git where your KO's / Apps are located and keep that under code control. You can then Ansible to deploy the apps to the deployers for them to push the Apps. You could also use the linux rsync command to have a repo and sync with the deployers. So you should have a strategy for this type of app management based on the tools you use.   

[imv327]

Thank you for your answer.

All these points have already been kind of analyzed and taken into account.
I was expecting more insights / inputs from experience and challenges that splunkers may have faced.
However I do know as well that our setup is a bit typical and does not reflect most of the enterprise setups that others may work with.

[deepakc]

 I highly recommend that you look at the training Splunk offers, this will get you into the deeper aspects and how to administrate Splunk and build up knowledge. 

The Splunk Admin courses should get your started, the various modules should cover what you are looking for at a deeper level. 

https://www.splunk.com/en_us/training/course-catalog.html?sort=Newest&filters=filterGroup3SplunkEnte...  

[imv327]

I don´t need this course. It will absolutely not help me for what I have to do which is pretty advanced in terms of classic Splunk architecture.