Ip resolving from host name in events

[Astorn]

I have some forwarders which are sending logs to indexers in another subnets and i have connected search head to these indexers.

Each event have the host field but not ip field. I'm looking forward to resolve ip address from these hosts. I have already found dnslookup, and metric (index=_internal component=Metric group=tcpin_connections) | stats values(hostname) as host by sourceIp).

But there are some problems, dnslookup can't resolve ip of hosts that there are sending to indexer in other subnet. In metric there are not all hosts from log. And in some case dnslookup resolve ip and metric not (what is very strange for me). I have spent several dozen of hours on this problem (which seems for me, should be realy first issue in splunk administration) and i'm realy confused about it. I'm looking for universal solution or next solution of resolving ip from host name from events. Thank you for any help

[tscroggins]

You can review the Splunk implementation of dnslookup in $SPLUNK_HOME/etc/system/bin/external_lookup.py. The configuration is documented at https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Configureexternallookups#External_look....

On the Splunk server, you can validate the functionality of external_lookup.py from e.g. a Bash shell:

$ cd $SPLUNK_HOME/etc/system/bin
$ ./etc/system/bin/external_lookup.py host src_ip <<EOF
> host,src_ip
> www.splunk.com ,
> ,8.8.8.8
> EOF
host,src_ip
www.splunk.com,23.12.144.246 
dns.google,8.8.8.8

In this example, I've provided three input lines:

1) the CSV header matching the fields provided on the command line;
2) the host www.splunk.com and no src_ip;
and 3) the src_ip 8.8.8.8 and no host.

On my test system, both the forward and reverse lookup were successful.

If the command fails or returns the wrong result, validate your host's DNS configuration, including your DNS servers, domain search list, and resolver cache if enabled.

If you're having name resolution problems with TCP or UDP inputs, i.e. you see IP addresses in the host field but expected to see host names, confirm the connection_host setting on the input. If connection_host is set to dns, Splunk uses FCrDNS to validate resolved names.

E.g.:

Source is 8.8.8.8.
Reverse lookup returns dns.google.
Forward lookup returns 8.8.8.8. => PASS, host=dns.google

Source is 10.0.0.1.
Reverse lookup returns foo.example.com.
Forward lookup returns 10.254.254.254. => FAIL, host=10.0.0.1