topic Logs not received into splunk in Getting Data In

Logs not received into splunk

VijaySrrie — Wed, 03 Jun 2020 07:37:29 GMT

Hi Team,

HF has been installed in a server, connectivity has been created to splunk, but we are not able to see any logs in splunk.
We have two different hosts.
For one of the hosts we are able to see the logs, but not able to see the logs for another host.

Note:
1) Host2 is using the same index name and log files are placed in same path as of host 1

Re: Logs not received into splunk

gcusello — Wed, 03 Jun 2020 08:00:10 GMT

Hi @vijaysri,
your architecture isn't so clear for me:

you have two servers that send logs to an Heavy Forwarder,
Heavy Forwarder sends logs to a Splunk Enterprise,
is it correct?

If this is your architecture, where you're not able to see logs on Splunk Enterprise or on Heavy Forwarder?

On HF you can see logs only if you hace a local copy of the logs (with duplicated license consuption), otherwise you can see logs only on Splunk Enterprise.

At first, did you enabled receiving on HF and Splunk?
if not, do this in [Settings -- Forwardrding and Receiving -- Receiving] in both the servers.

if yes, If you don't see logs on Splunk Enterprise, you should check the connection between hosts and HF and between HF and Splunk Enterprise.
To check this at first you should run this search on Splunk Enterprise:

index=_internal | stats count BY host

and see if the hostnames of host1, host2 and HF are prosent or not:

if you haven't none of them there's a problem between HF and Splunk,
if you have HF's logs but not hos1 and host2 logs there's a problem between hosts and HF.

in both the cases, check connections using telnet on port 9997 from the source system to the target (e.g. HF to Splunk or host1 to HF).

Ciao.
Giuseppe

Re: Logs not received into splunk

VijaySrrie — Wed, 30 Sep 2020 05:35:47 GMT

@gcusello

index=_internal "host1" --> able to see the logs
index=_internal "host2" --> able to see the logs

for host1 ---> I am able to see the logs into the particular index assigned.
Issue is only with host2

I am not able to see the logs for host 2 into the particular index. May I know what troubleshooting can be done?

Re: Logs not received into splunk

gcusello — Wed, 03 Jun 2020 08:50:02 GMT

Hi @vijaysri,
if you see internal logs from host2 but not other logs, check the differences with host1 in inputs.conf.

Then try the monitor paths if there are results, e.g. if you have in inputs.conf

[monitor:///app/log/*log]

you could try in Linux

ls -la /app/log/*log

Ciao.
Giuseppe