Installation

Splunk Heavy Forwarder Problem

Darsh1561
Explorer

Hello Community,

I would like to inquire about some issues I am facing while setting up a heavy forwarder in splunk. Please take a look at the below issues :- 

1) Hosts are visible in splunk but all of them are not forwarding their logs to the indexer.

2) Linux server are not able to forward logs to the indexer.

3) Some host are able to forward their logs to indexer post a modification in their universal forwarder file manually, but it takes an hour or so before they forward their logs.

4) The most recently added do not show their logs in real time i.e. when a time frame recently added devices should logs in last 4 hours or 60 minutes but they do show only post 24 hours time filter.

 

Thanks in advance.

0 Karma
1 Solution

gcusello
Esteemed Legend

Hi @Darsh1561m,

please detail your questions:

1) Hosts are visible in splunk but all of them are not forwarding their logs to the indexer.

you mean that you see some hosts in the Deployment Server but that you don't see their logs or what else?

2) Linux server are not able to forward logs to the indexer.

are you meaning that all your Linux servers don't sed logs?

I suppose that you already configured:

  • your indexers and your Heavy Forwarders to receive logs,
  • your Forwarders to send logs to the Indexers or to Heavy Forwarders, 

how did you do this?

did you checked that the firewall routes between Forwarders and Indexers and Heavy Forwarders are open?

What's you architecture?

3) Some host are able to forward their logs to indexer post a modification in their universal forwarder file manually, but it takes an hour or so before they forward their logs.

Which local configuration did you do?

are you using a Deployment server?

have you followed the instructions at https://www.splunk.com/en_us/resources/videos/getting-data-in-with-forwarders.html or https://docs.splunk.com/Documentation/Splunk/9.0.3/Data/Usingforwardingagents or https://docs.splunk.com/Documentation/Splunk/9.0.3/Forwarding/Aboutforwardingandreceivingdata ?

4) The most recently added do not show their logs in real time i.e. when a time frame recently added devices should logs in last 4 hours or 60 minutes but they do show only post 24 hours time filter.

did you checked the timestamp of these events, is it correct?

Ciao.

Giuseppe

View solution in original post

PickleRick
Ultra Champion

You seem to have multiple separate problems here. So isolate them and try to troubleshoot one by one.

First question is what architecture do you have. Second - what _is_ working. Third - what change did you introduce lately. What was the expected behaviour after this change and what is the actual observed behaviour.

Don't try to do multiple things at once and then try to pinpoint why something is not working as expected because this way you can't track cause-effect relationships.

0 Karma

Darsh1561
Explorer

Thanks for your input.

0 Karma

aad
Loves-to-Learn

Thank you!  that make sense

0 Karma

gcusello
Esteemed Legend

Hi at all,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

0 Karma

gcusello
Esteemed Legend

Hi @Darsh1561m,

please detail your questions:

1) Hosts are visible in splunk but all of them are not forwarding their logs to the indexer.

you mean that you see some hosts in the Deployment Server but that you don't see their logs or what else?

2) Linux server are not able to forward logs to the indexer.

are you meaning that all your Linux servers don't sed logs?

I suppose that you already configured:

  • your indexers and your Heavy Forwarders to receive logs,
  • your Forwarders to send logs to the Indexers or to Heavy Forwarders, 

how did you do this?

did you checked that the firewall routes between Forwarders and Indexers and Heavy Forwarders are open?

What's you architecture?

3) Some host are able to forward their logs to indexer post a modification in their universal forwarder file manually, but it takes an hour or so before they forward their logs.

Which local configuration did you do?

are you using a Deployment server?

have you followed the instructions at https://www.splunk.com/en_us/resources/videos/getting-data-in-with-forwarders.html or https://docs.splunk.com/Documentation/Splunk/9.0.3/Data/Usingforwardingagents or https://docs.splunk.com/Documentation/Splunk/9.0.3/Forwarding/Aboutforwardingandreceivingdata ?

4) The most recently added do not show their logs in real time i.e. when a time frame recently added devices should logs in last 4 hours or 60 minutes but they do show only post 24 hours time filter.

did you checked the timestamp of these events, is it correct?

Ciao.

Giuseppe

Get Updates on the Splunk Community!

New Splunk Observability innovations: Deeper visibility and smarter alerting to ...

You asked, we delivered. Splunk Observability Cloud has several new innovations giving you deeper visibility ...

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...