TCPDUMP Command

[anandhalagarasa]

Hi Team,

We have configured HTTP Event Collector in our Indexer server (Linux machine) and shared the same across to the requester. But actually there is an connection issue hence they couldn't able to ingest the logs into splunk.

So we want to know how to run the tcpdump from our indexer and see if there is any traffic issues.

What is the command to fetch the tcpdump in a linux machine of an indexer server. and from where should i need to run the command as well.

[renjith_nair]

@anandhalagarasan,

Probably you should start from looking at the client side, i.e. whats the error getting at the client side. Here is a list of error codes and its description for your reference

https://docs.splunk.com/Documentation/Splunk/7.2.4/Data/TroubleshootHTTPEventCollector#Possible_erro...

Also make sure that HEC is enabled correctly and port configured (default 8088) is listening (netstat).

Reference : http://dev.splunk.com/view/event-collector/SP-CAAAE7F

If you have a firewall in between your client and indexer, you might want to check there as well to make sure that the traffic is allowed (ip & port)

Simple tcpdump can be run from indexer as

 tcpdump -i eth0 -nn -s0 -v port 8088

[nickhills]

Ok, a couple of things:
1.) It is not recommended to run an HEC on an indexer (its fine for a Lab/test environment, but if this is Production you should really install a dedicated HEC receiver, and use it to forward events to your indexers)
2.) You need to know the port that your HEC is running on - the default port is 8088, but you may have set this to something else.
3.) Before you start running tcpdump (which on a busy indexer could be very intensive) start by checking some other things first:
a.) is the HEC running - run netstat -lpen and look for processes running on the relevant port (8088 the process should be splunkd)
b.) check your firewall rules sudo iptables -L -v -n | more or firewall-cmd --list-all [--zone=your_zone] Make sure there are rules for 8088
c.) check that the certificate presented by your HEC is valid openssl s_client -connect your.splunk.hec.server:8088 (Run this on the server to check the cert is presented, and also run this from the client to confirm end-to-end communications)
4.) if all of the above checks out, then briefly run tcp dump to see what traffic arrives tcpdump -i eth0 'port 8088'