topic How to resolve Splunk from missing some automatically recognized source types? in Splunk Search

How to resolve Splunk from missing some automatically recognized source types?

quantumburnz — Tue, 29 Sep 2020 14:46:46 GMT

I have transforms like access-extractions and access-request, which map to the automatically recognized source types like access_combined, access_combined_wcookie, and access_common in the Field Extractions. Fields from events with these sourcetypes have fields extracted at search time as they should. However, the automatically recognized source types list also mentions: websphere_trlog_syserr and websphere_trlog_sysout. I have a websphere_trlog but not the other two. These should be built-in and not require any other specific app, right? Why would I be missing these two?

Ref: http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Listofpretrainedsourcetypes

Splunk version 6.4.3

Re: How to resolve Splunk from missing some automatically recognized source types?

woodcock — Fri, 07 Jul 2017 21:40:20 GMT

You should never, Never, NEVER rely on Splunk's automatic sourcetyping feature (IMHO this should be deprecated and removed). You, as the admin, should ALWAYS manually sourcetype everything that you forward into splunk and you should consult and conform to this document:

http://docs.splunk.com/Documentation/Splunk/6.6.2/Data/Listofpretrainedsourcetypes
YOU should be the one to do this mapping with a sourcetype=<foo> inside of EVERY stanza in EVERY inputs.conf.

Re: How to resolve Splunk from missing some automatically recognized source types?

quantumburnz — Fri, 07 Jul 2017 23:33:56 GMT

Thanks woodcock - that's some pretty strong wording. 🙂

The sourcetypes don't currently conform to the document so they'll need to be renamed. Regardless, once this is done, I still want Splunk to properly extract fields at search time but there's no definition for these sourcetypes in props.conf.

What would cause this? The extractions should be available by default, correct? Hence "pretrained sourcetypes."

Re: How to resolve Splunk from missing some automatically recognized source types?

woodcock — Sat, 08 Jul 2017 14:29:47 GMT

This product is not a magic 8-ball or genie in a bottle; it can't know everything. Check splunkbase for an app related to your source/sourcetype; if you are using enterprise software, it is probably there. If not, you will have to build your own field extractions or force the product to output it's stuff in something Splunk automatically groks like JSON, CEP, CSV, or KVPs.

Re: How to resolve Splunk from missing some automatically recognized source types?

quantumburnz — Sat, 08 Jul 2017 15:21:27 GMT

Agree - there's no such thing as a "silver bullet." However, I guess my question comes down to: Is there a built-in extraction for all of the pretrained sourcetypes? I assumed there was; otherwise, what's the point of the "pretrained?"

Re: How to resolve Splunk from missing some automatically recognized source types?

woodcock — Sat, 08 Jul 2017 15:29:22 GMT

I suppose that is a good point. Start with the CIM app:
https://splunkbase.splunk.com/app/1621/
It changes frequently. Build out from there.