topic Where is the logtype source type defined? in Getting Data In

Where is the logtype source type defined?

DUThibault — Fri, 24 Nov 2017 15:20:53 GMT

I've added a (universal) forwarder's local /var/log as a data input, specifying sourcetype = automatic. For audit.log, the indexed data are all marked with sourcetype=logtype, but logtype is not found in Settings: (Data) Source types. Where is logtype defined?

Re: Where is the logtype source type defined?

gcusello — Fri, 24 Nov 2017 15:23:10 GMT

Hi DUThibault,
see in Indexer.
Bye.
Giuseppe

Re: Where is the logtype source type defined?

DUThibault — Fri, 24 Nov 2017 15:34:26 GMT

Where, exactly? In Splunk Web, no Indexer to be found. Searching docs.splunk.com for "indexer source type" yields 90 hits but no obvious answer.

Re: Where is the logtype source type defined?

gcusello — Fri, 24 Nov 2017 15:48:00 GMT

Hi DUThibault,
what is your architecture? do you have an all-in-one server or do you have search Heads and Indexers?

Anyway, you can find in Splunk server (not Forwarders) sourcetypes in [Settings -- Source types].
I searched logtype in my installation and I didn't find it!

Pretrained sourcetypes are described in https://docs.splunk.com/Documentation/Splunk/7.0.0/Data/Listofpretrainedsourcetypes and http://docs.splunk.com/Documentation/Splunk/7.0.0/Data/Whysourcetypesmatter

Bye.
Giuseppe

Re: Where is the logtype source type defined?

DUThibault — Fri, 24 Nov 2017 16:02:35 GMT

My architecture is minimalistic, with a single instance indexer and search head, fed by one or more universal forwarders.

logtype is not in the list of pre-trained source types, all of which are listed in Settings: (Data) Source types except in three cases: sugarcrm_log4php is absent, while websphere_trlog_syserr and websphere_trlog_sysout seem to have been merged into a single websphere_trlog source type. Could the 7.0.0 documentation pages be incorrect or out of date?

Note that splunk btool props list logtype returns nothing.

Re: Where is the logtype source type defined?

DUThibault — Fri, 24 Nov 2017 17:20:34 GMT

I've now scoured the instance's props.conf and inputs.conf as well as the forwarder's, and I found sourcetype = logtype in /opt/splunkforwarder/etc/apps/search/local/inputs.conf. This seems to be an artefact of how I first set up the forwarder (you can do splunk add monitor on the forwarder's system or you can configure the monitor from the Splunk instance, using Splunk Web; I should have done just the latter). Sure enough, stopping the forwarder, commenting out the sourcetype assignation and restarting the forwarder resulted in the sourcetype becoming linux_audit. Mystery solved!

Re: Where is the logtype source type defined?

woodcock — Sun, 26 Nov 2017 01:16:23 GMT

OK, to set your sourcetype to automatic, you don't actually set your sourcetype at all, just leave completely out of your inputs.conf stanza definition. If you truly set sourcetype=automatic, then I would have expected that your sourcetype value would literally be the literal string automatic. I am unsure of how it could have come to be the literal string.

But let's back up. It is a generally poor practice to allow Splunk to decide what your sourcetypes are (should be). If you are going to start there, then turn it on, dump everything to a disposable index (like main) and then double-check everything. In all likelihood, it isn't going to tell you anything that you either didn't already know or wouldn't have immediately realized by glancing at your data. In any case, for *NIX files under /var/log/, splunk should do a find job of sourcetyping, if you set nothing at all.

Re: Where is the logtype source type defined?

DUThibault — Tue, 28 Nov 2017 19:04:16 GMT

To sum up, any offending sourcetype will probably be caused by a sourcetype = <offending_type> stanza in [forwarding_system]/opt/splunkforwarder/etc/apps/search/local/inputs.conf. Comment out or delete the line (taking care to stop the forwarder before doing the edit, and restarting it afterwards), and the sourcetype will revert to automatic.