topic Re: Sourcetypes not configured in Linux Server are shown in Splunk in Splunk Enterprise

Sourcetypes not configured in Linux Server are shown in Splunk

Splunk_Ryan — Mon, 22 Mar 2021 03:54:28 GMT

This is confusing me.

On my Linux server the universal forwarder is installed, and the following sourcetypes are specified in inputs.conf. Nothing more is added.

[monitor:///var/log/httpd/access_log]
sourcetype=access_combined
index = apache

[monitor:///var/log/httpd/error_log]
sourcetype=apache:error
index = apache

When I search for this Linux server on Splunk. there are way many sourcetypes coming up. Top 10 values are as follows. It is good to see access_combined and apache:error coming up, but why are the others coming up too? I did not specify them in inputs.conf!

access_combined 69,824 74.23%
ps 18,353 19.511%
bash_history 1,999 2.125%
Unix:UserAccounts 936 0.995%
cpu 870 0.925%
df 580 0.617%
usersWithLoginPrivs 360 0.383%
protocol 290 0.308%
Unix:Update 204 0.217%
apache:error 188 0.2%

Btw, I installed Splunk App for Unix and Splunk Add-on for Unix and Linux on my Splunk. But this shall not attribute to the additional sourcetypes coming up on Splunk, because as far as I know I have to first specify the additional sourcetypes (e.g. [monitor:///xxxx], sourcetyp=cpu) in inputs.conf which I have not done so.

Could anyone advise? much appreciated.

Re: Sourcetypes not configured in Linux Server are shown in Splunk

Splunk_Ryan — Mon, 22 Mar 2021 04:05:16 GMT

I just discovered something interesting. There are multiple inputs.conf files in the Linux Servers.

In one Linux server, there are:
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf
/opt/splunkforwarder/etc/apps/introspection_generator_addon/default/inputs.conf
/opt/splunkforwarder/etc/apps/splunk_httpinput/default/inputs.conf
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/local/inputs.conf
/opt/splunkforwarder/etc/system/default/inputs.conf
/opt/splunkforwarder/etc/system/local/inputs.conf

In the other one Linux server, there are:
/opt/splunkforwarder/etc/apps/search/local/inputs.conf
/opt/splunkforwarder/etc/apps/splunk_httpinput/default/inputs.conf
/opt/splunkforwarder/etc/apps/introspection_generator_addon/default/inputs.conf
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf
/opt/splunkforwarder/etc/system/local/inputs.conf
/opt/splunkforwarder/etc/system/default/inputs.conf

How come the following two files exist in some servers, but not in other servers?
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/default/inputs.conf
/opt/splunkforwarder/etc/apps/Splunk_TA_nix/local/inputs.conf

Re: Sourcetypes not configured in Linux Server are shown in Splunk

isoutamo — Mon, 22 Mar 2021 07:13:44 GMT

When you are installing those apps they could have some default inputs already configured.

Have you vanilla Splunk_TA_nix for splunkbase or have you used your company own version, which could have some defaults? Have you used same package where you are installing this app for all servers?

Basically those configurations under default have come from package and you should never modify those. Those which are under local are usually modified in those individual servers. That can do directly with editor + file or used cli commands.

Re: Sourcetypes not configured in Linux Server are shown in Splunk

Splunk_Ryan — Mon, 22 Mar 2021 08:17:47 GMT

Hi soutamo,

Thanks to let me realize the default inputs coming with those apps.

So I just installed / copied this directory /opt/splunkforwarder/etc/apps/Splunk_TA_nix/ to those Linux clients and now every client is sending logs to Splunk instance.

Thanks again.