topic Re: Splunk Forwarder from public server to local server on a network in Getting Data In

Splunk Forwarder from public server to local server on a network

johnnykhoueiry — Thu, 11 Jan 2018 09:19:51 GMT

Hello,

I was able to set splunk forwarder from local server to local splunk server on our network.
How can i set it from a public server to send records to that same splunk hosted on our local server ?

This is the error i currently receive:
01-11-2018 03:14:26.364 -0500 ERROR TcpOutputProc - Processing server from outputs.conf: can't resolve a valid IP address for host="ourhost"

Thank you.

Re: Splunk Forwarder from public server to local server on a network

mayurr98 — Thu, 11 Jan 2018 12:13:06 GMT

you can try putting dns name instead of ip address

Re: Splunk Forwarder from public server to local server on a network

nickhills — Thu, 11 Jan 2018 13:14:35 GMT

There are several things to consider before doing this with real data.

1.) You will want to configure your 'receiving' server to have a TLS listener.
2.) Ensure your forwarder is configured to send TLS data only.
You want to do both of these things, because otherwise your events (which may contain all of your sensitive data) will be sent in clear text
3.) You will need to make sure your 'receiving' server has a rule on your firewall, as well as any NAT or IP configuration to make it accessible from the public internet on your TLS port.
4.) You might want to consider installing a heavy forwarder as your 'receiving' server. Generally speaking it would be considered bad practice to put indexers directly on the internet - although sometimes 'needs must'.
5.) You should make sure you keep on top of Splunk & OS updates on your receiving server. Its also a good idea to restrict other Splunk ports (web/app/kv/mgt) ports so that you can only access them from your LAN. - Perhaps use your firewall to restrict to the IP of your forwarder.
6.) Consider if you can use the HEC which is more suited to sending events across the internet.
7.) Confirm that your forwarder has a direct route to the internet - if your traffic is marshalled through a proxy server, you may encounter issues on the forwarding side.
8.) Finally, confirm the IP address is rout able (some ISPs do nasty things to prevent this) and that any DNS name you have configured, correctly resolves to your IP.

Re: Splunk Forwarder from public server to local server on a network

micahkemp — Thu, 11 Jan 2018 15:10:01 GMT

It sounds to me like your forwarder is unable to resolve the hostname. From that server try these to help troubleshoot:

ping ourhost

dig ourhost

If your forwarder can't resolve the hostname to an IP address, you should first determine why. If it's a reasonable answer (split horizon DNS, etc), perhaps you just need to hardcode the IP address into outputs.conf. Or perhaps work on getting the name resolvable by DNS.