topic Re: Can you override sourcetype at inputs.conf without touching other config files ? in Getting Data In

Can you override sourcetype at inputs.conf without touching other config files ?

stwong — Thu, 21 Jun 2018 06:52:46 GMT

Hi all,

Seems we have to override the sourcetype to sourcetype other than 'recognized' ones (e.g. syslog) in order to make host_segment work. e.g.

[monitor:///ssd/*/*/cyrus.log]
disabled = false
host_segment = 3
index = test
sourcetype = cyrus

However, for some cases, we hope to make use of sourcetype from Splunk (e.g. syslog) but use host_segment instead of host parsed in the log content for that particular sourcetype. Possible to do that at inputs.conf without touching other config files like props.conf or transform.conf?

Sorry for the newbie question.

Thanks and regards

Re: Can you override sourcetype at inputs.conf without touching other config files ?

FrankVl — Thu, 21 Jun 2018 07:07:37 GMT

Syslog sourcetype comes with a TRANSFORMS stanza in etc/system/default/props.conf that pulls the hostname from the event. If you don't want that to happen, add the following empty transforms setting in your own local props.conf, to disable that feature for this particular source.

[source::/ssd/*/*/cyrus.log]
TRANSFORMS =

Re: Can you override sourcetype at inputs.conf without touching other config files ?

stwong — Thu, 21 Jun 2018 11:24:08 GMT

Got it. Thanks a lot.

Re: Can you override sourcetype at inputs.conf without touching other config files ?

stwong — Thu, 21 Jun 2018 11:52:05 GMT

tried but seems not working - still gets hostname from log content. My inputs.conf looks like following:

[monitor:///ssd///cyrus.log]
ignoreOlderThan=1d
disabled = false
TRANSFORMS =
TRANSFORMS-host =
host_segment = 3
index = test

sourcetype = cyrus

Did I miss anything?

Thanks and rgds

Re: Can you override sourcetype at inputs.conf without touching other config files ?

FrankVl — Thu, 21 Jun 2018 11:55:49 GMT

The source based stanza incl. the empty transforms as I put it in my answer should go into props.conf, not inputs.conf.

Re: Can you override sourcetype at inputs.conf without touching other config files ?

stwong — Thu, 21 Jun 2018 12:53:28 GMT

stupid me. sorry for the newbie mistake...

Just put following in props.conf but seems no effect.

[source::/ssd///cyrus.log]
TRANSFORMS =
TRANSFORMS-host =

splunk btools props list shows the setting is correct. Would you help? Thanks again.

Re: Can you override sourcetype at inputs.conf without touching other config files ?

FrankVl — Thu, 21 Jun 2018 13:27:30 GMT

Hmm, I would think that should overwrite the setting in default/props.conf. Can you try setting it under [syslog] in props.conf, rather than that source based stanza?

PS: where are you deploying this props.conf? It should be on heavy forwarder or indexer. If you are collecting this data with a universal forwarder, it doesn't work putting this props.conf on the UF.

Re: Can you override sourcetype at inputs.conf without touching other config files ?

stwong — Fri, 22 Jun 2018 02:34:46 GMT

It should be on heavy forwarder or indexer

Got it. Will this overrides host part for all data with sourcetype=syslog?
Chances are we hope to use host_segments for syslog data on some UF, while keep using host in log content for some other UF.

Sorry to bother again.
Thanks a lot.

Re: Can you override sourcetype at inputs.conf without touching other config files ?

FrankVl — Fri, 22 Jun 2018 04:14:36 GMT

When you set it for [syslog] it applies to all feeds with that sourcetype. So better set it using a source based stanza as I originally suggested (assuming that indeed works).

Re: Can you override sourcetype at inputs.conf without touching other config files ?

stwong — Fri, 22 Jun 2018 08:35:13 GMT

Noted and thanks. Will try it out.