topic How to get sourcetypes to TA/apps mapping via Splunk API? in Getting Data In

How to get sourcetypes to TA/apps mapping via Splunk API?

koshyk — Tue, 11 Apr 2017 11:54:41 GMT

As part of performance analysis, we are asked to do sourcetypes and regex analysis. The first step I wanted to see is how I can map a sourcetype to an add-on/app/TA?
We have got around 2000 sourcetypes as part of a literal search coming from various TA's, so manually it is impossible.

eg of what I'm looking for is something like below

Sourcetype,App
cisco:ise:syslog,Splunk_TA_cisco-ise
vmware:esxlog:vmkwarning,Splunk_TA_esxilogs

I know I can write scripts/grep on btool and do. but trying to think a way to do directly within Splunk

Re: How to get sourcetypes to TA/apps mapping via Splunk API?

jkat54 — Tue, 11 Apr 2017 12:25:19 GMT

This will only work if you have the inputs on the splunk enterprise instance itself:

| rest /services/configs/inputs | table eai:acl.app sourcetype | where sourcetype!=""

If you're using forwarders to collect your data, you will need some sort of for loop and bat/powershell/bash scripts:

bash example to be run on deployment server:

for input in `find /opt/splunk/etc/deployment-apps -type f -name inputs.conf`; do echo $input; grep -i sourcetype $input; done

powershell example to be run on deployment server:

get-childitem 'C:\Program Files\Splunk\etc\deployment-apps' -Recurse -Include "inputs.conf" -WarningAction SilentlyContinue -ErrorAction SilentlyContinue|
foreach { 
    write-host $_.FullName
    select-string $_ -pattern "sourcetype"
}

# remove Warning & Error Actions above if you want to see permission issues, etc... shouldnt happen if your running powershell as administrator though.

You might also want to remove deployment-apps from the paths... which would then show everything under $splunk_home/etc and would work on any splunk server... if you want to run these on forwarders, again you will have to change the paths to match. YMMV!

Re: How to get sourcetypes to TA/apps mapping via Splunk API?

koshyk — Tue, 29 Sep 2020 13:36:30 GMT

Thanks jkat54
but only 60% of sourcetypes are present in inputs.conf ;For example if you go into Splunk_TA_vmware, the sourcetypes are assigned within props.conf

Is there similar search for "inputs", "props" , "transforms" etc?

Re: How to get sourcetypes to TA/apps mapping via Splunk API?

koshyk — Tue, 11 Apr 2017 15:26:20 GMT

Ah. Found a way. Combining conf-props, and inputs. Thanks jkat54 for the idea

| rest /services/configs/conf-props| stats count by eai:acl.app,title| search title!="source*"| rename title as sourcetype|eval endPoint="props"| append [| rest /services/configs/inputs| stats count by eai:acl.app,sourcetype| where sourcetype!=""| eval endPoint="inputs"]| stats count by eai:acl.app,sourcetype,endPoint

Re: How to get sourcetypes to TA/apps mapping via Splunk API?

jkat54 — Tue, 11 Apr 2017 22:06:27 GMT

You can change sourcetypes with props and transforms sure, but then they are dynamic based on regex, etc... So I don't know how to help you find those.

Probably some combination of the data source and the sourcetype and then the app will work.