topic Several small log files - sourcetype = local-too_small in Getting Data In

Several small log files - sourcetype = local-too_small

ikulcsar — Wed, 29 Nov 2017 13:47:31 GMT

Hi,

I've got a problem with monitoring several log files generated by syslog-ng. There are 50+ switches. I am collecting their logs with a syslog-ng server, generating separate log files for every switch, every day. Some of them send only a few lines so that logs file is small.
I can collect all the logs, but I have got an issue with the sourcetype. All (most?) of the small log file has a local-too_small sourcetype instead of syslog, which I configured explicitly. Based on my research and testing, the auto sourcetype can cause this, but I already add the sourcetype. So what I am doing wrong, why the Splunk ignore it?

inputs.conf:
[monitor:///var/log/remotelogs/*/log/]
host_segment = 8
index = default
sourcetype=syslog

Regards,
István

Re: Several small log files - sourcetype = local-too_small

harsmarvania57 — Wed, 29 Nov 2017 14:03:49 GMT

Hi @ikulcsar,

Can you please check your inputs.conf configuration using btool $SPLUNK_HOME/bin/splunk cmd btool inputs --debug list and check whether sourcetype=syslog is assigned to your monitor stanza or not? If it is assigned then can you please try to restart splunkforwarder ?

Re: Several small log files - sourcetype = local-too_small

ikulcsar — Tue, 29 Sep 2020 16:58:42 GMT

Hi,
Thank you for your comment. Here is the output. I modified the monitor definition to be more specific, restart the full server, too. But no change.

/opt/splunk/etc/system/local/inputs.conf [monitor:///var/log/remotelogs//log///]
/opt/splunk/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunk/etc/system/local/inputs.conf host = shadow
/opt/splunk/etc/system/local/inputs.conf host_segment = 8
/opt/splunk/etc/system/local/inputs.conf index = default
/opt/splunk/etc/system/local/inputs.conf sourcetype = syslog

Any other idea?

Regards,
István

Re: Several small log files - sourcetype = local-too_small

ikulcsar — Sat, 02 Dec 2017 09:21:50 GMT

Hi,

Finally, I reinstall it from the scratch with Splunk Ent. 7.0, reconfigure the inputs and it works... I can not explain and unfortunately cannot reproduce that behavior...

Thank you for your kind help.
Regards,
István

Re: Several small log files - sourcetype = local-too_small

ikulcsar — Sat, 02 Dec 2017 09:21:50 GMT

Hi,

Finally, I reinstall it from the scratch with Splunk Ent. 7.0, reconfigure the inputs and it works... I can not explain and unfortunately cannot reproduce that behavior...

Thank you for your kind help.
Regards,
István