Why are logs not getting in from Linux machine?

[an]

Logs are not getting in from Linux machine

I am using Splunk cloud trial and in a Linux machine and installed universal forwarder and added 

monitor path as well. But no luck.

[an · Accepted solution]

Problem solved, by reinstalling the UF. Thanks

[danielcj]

Hello @an ,

You could do the following to troubleshoot this issue:

1 - Download the Universal Forwarder app from your Splunk Cloud trial instance and install it on /opt/splunkforwarder/etc/apps (If you not already installed this app).

2 - Verify the permissions of the files that you added on the monitor path (if the user that is running Splunk can read/execute these files).

3 - Verify if the monitor configs are correct and an index is being defined on the monitor stanza.

4 - Run splunk btool check command to check for any typos on .conf files.

5 - Restart the Universal Forwarder (The changes on the Universal Forwarder are only applied after a restart).

6 - Search for index=_internal host=<your_uf_host> on Splunk Cloud trial to verify if internal logs are being indexed. If the internal logs are being indexed, search for the index that you configured your monitor on the step 3.
6.1 - If there are internal logs from your instance but there are no logs on your index, there is an error on the configs of this input.
6.2 - If there are no internal logs, the error could be related to a firewall issue that you could try to troubleshoot on the UF log files (search for an error that shows that you cannot connect to the Splunk Cloud): cat /opt/splunkforwarder/var/log/splunk/splunkd.log | grep -i ERROR

Thanks.

[an]

this is getting if i am running 

cat /opt/splunkforwarder/var/log/splunk/splunkd.log | grep -i ERROR

and copied input and output config

[isoutamo]

Hi

When you are using Splunk Cloud then outputs.conf must contain TLS certs. It seems that you haven't those on your outputs.conf. Should be something like:

[tcpout]
defaultGroup = SC_STACK
useACK = true

[tcpout:SC_STACK]
server = inputs1.SC_STACK.splunkcloud.com:9997, inputs2.SC_STACK.splunkcloud.com:9997, ..... inputs15.SC_STACK.splunkcloud.com:9997
compressed = false

clientCert = $SPLUNK_HOME/etc/apps/100_SC_STACK_splunkcloud/default/SC_STACK_server.pem

sslCommonNameToCheck = *.SC_STACK.splunkcloud.com
sslVerifyServerCert = true
useClientSSLCompression = true

You could download a new valid UF connection package from your SC instance. Just select App "Universal Forwarder" on left side panel and follow it's instructions.

r. Ismo 

[richgalloway]

Double-check your firewalls to make sure they allow connections from the UF to Splunk Cloud.

The outputs.conf file doesn't look right.  There should be a certificate specified, but I don't see it.  Are you sure this is the outputs.conf that came with the 100_splunkclouduf app?

It's not necessary to specify source in a monitor clause.  The source field is populated automatically from the monitored file name.

[richgalloway]

Did you install the Universal Forwarder app from your cloud SH on the UF?  Did you then restart the UF?

Have you checked the UF's logs to see if any messages might explain why it's unable to send to Splunk Cloud?

[an]

yes installed Universal Forwarder and restarted 

[sidnakvee]

Hi !

I am facing the same issue getting windows logs and sysmon logs but not getting any Linux and zeek logs . Using this inputs.conf file and all settings followed per documentation credneial package installed sucessfully as well. Also installed Zeek Apps as well. Sorry forgot to mention that seeing hosts when do index=_internal search last 24 hours. 

Any help please ?

default]
host = zeek-VirtualBox

[monitor:///var/log/messages]
disabled = 0
index = unix

[monitor:///var/log/syslog]
disabled = 0
index = unix

[monitor:///var/log/faillog]
disabled = 0
index = unix

[monitor:///var/log/auth.log]
disabled = 0
index = unix

[monitor:///opt/zeek/log/current]
disabled = 0
_TCP_ROUTING = *
index = zeek
sourcetype = bro:jason
whitelist = \.log$

[richgalloway]

This thread is more than 2 years old.  For better chances at having more people see it, please post a new question.