How to find the IP address of the AWS(f5) data coming through port 9997 to a heavy forwarder?

[Rocky31]

The port 9997 is enabled, data hitting the Heavy Forwarder. How to validate specific data and IP address?

[yannK]

If the port 9997 is used to listen to splunk forwarder,
the metrics.conf on the heavy forwarder will only tell you the IP of the previous forwarder sending the data, not the type of data per source.

try index=_internal host=myheavyforwarder fwdType, it will show you the orignal forwarders connecting to 9997. (But not the nature of the data.)

If you really want to go down to the forwarder level, you can look in the metrics.log of the forwarders themselves.
But by default the metrics may not be forwarded (check with inputs.conf and outputs.conf whitelists on the forwarder settings to enable it)

[lguinn2 · Accepted solution]

On the heavy forwarder, the metrics.log will contain information about the volume of data received from various inputs. In the metrics log, look for group=tcpin_connections and you will see that it shows the host name and ip address for all the times that a forwarder connected and sent data.

If your heavy forwarder is sending its internal logs to the indexer(s) - as it should be - then you can run the following search

index=_internal group=tcpin_connections host=nameofheavyforwarder 

[Rocky31]

I am not happy with your answer. i tried no but no match. exactly the same thing to type in my metrics.lo are it just a syntax.

[lguinn2]

Just try this then

index=_internal group=tcpin_connections

This would show contacts from all the fowarders, so you might need to drill down into the results.
If you want to look in the log files, I think you need to look in splunkd.log on both the indexer(s) and the heavy forwarder.