I would like to hear from other admins on how they are keeping up with high demand of data onboarding requests into their Splunk instance in large organizations.
We are battling with more than 300 requests per month to onboard data into Splunk as every application in the organization wants to utilize Splunk for monitoring and the demand only keeps increasing. Most of these are custom application logs. The biggest bottleneck is defining props (LINE BREAKER, TIME STAMP etc..,) for the source types by having to manually analyze each individual log. Other parts of the onboarding (inputs.conf, indexes.conf etc..,) can be easily automated for seamless onboarding but not props.
Not defining props for source types to leave to Splunk defaults is not an option as we have seen some serious performance issues on indexers.
I would like to hear from Splunk if there is a strategic direction in this regard to make admins life easier with respect to onboarding and other admins who might have dealt with similar situation and overcome in creative ways.
Regards,
Pradeep
Even I have same problem like we get security logs from different applications on monthly basis.
The applications are customer specific so they don’t have proper logging mechanism. Sometimes we had to get them from debug logs where each event would further split into many events.
what I do is I onboard all the logs required for security monitoring, identify the required event types by just simulating few scenarios where high risk is involved and apply props|transforms, the rest of events which are not matching with my event types will be discarded.
I know this is not a solution for your question. Kind of feedback you can consider.
There is no such option to automate props and transforms on unknown data/events.
we can can automate field-aliases if the event is in key-value fair, not tagging of events. tagging of events is most important to make use of DATA MODELS.
Hi
In our experience there are something what you can do and request from you "clients/customers" which want to onboard data.
A good place to learn more is https://lantern.splunk.com/hc/en-us/sections/360007451513-Data-Management and whole Splunk Success Framework at lantern.splunk.com.
r. Ismo
@isoutamo thanks for your response. We follow most of these guidelines if not all. The guidelines is a generic framework and does not address my problem. We encourage our SMEs to provide props.conf during the on-boarding process but I can't enforce it. We try to reward by fast tracking the on-boarding process for them. There are only handful of them who provide us props upfront. But this is not sufficient to keep up with the demand. I am looking for a more self-service on-boarding by SMEs approach and we as admins to do governance on the data and keep a check on the license and hardware.