Hi, we are trying to use Splunk Insights for Infrastructure in a large company where security policies are quite tight. By default, machines have open connections to port 9997 on Heavy Forwarders, so opening a new port on 8088 for each agent install is not practical.
Is it possible to make insights CollectD agent use the Universal Forwarder, that is also in the agent machine, to send its data instead of directly sending data to the Infrastructure server using HEC on port 8088?
Any other suggestions?
Many thanks in advance!
Luis Bontempo
Hey Luis!
Thanks for checking out insights for infra!!
While what your asking is totally doable, - to the best of my knowledge, at least - catching HEC with UF is not officially supported.
That being said, there are customers who do it and have no issues. You can chat with some of them on our slack chat, sign up here: splk.it/slack
I would still like to explore the option with you and help you try it out to see if you can achieve what makes deploying easier for you!
please hit me up at the email in my bio, or at @mattymo on Slack in our #insights-for-infra channel, where myself and the PM and ENG folks hang out!
Hi all,
Thanks to Matthew we were able to sort this one out so here are the steps to do it.
On the machine where you installed the agents do as sysadmin:
Edit the agent machine universal forwarder configuration "inputs.conf" and change it to
[http]
disabled=0
port=8088
enableSSL=0
dedicatedIoThreads=2
maxThreads = 0
maxSockets = 0
useDeploymentServer=0
# ssl settings are similar to mgmt server
sslVersions=*,-ssl2
allowSslCompression=true
allowSslRenegotiation=true
[http://uf_hec_local]
disabled = 0
token = 00000000-0000-0000-0000-000000000000
Restart the universal forwarder
sudo /opt/splunkforwarder/bin/splunk restart
Edit the CollectD agent
vi /etc/collectd/collectd.conf
server "localhost"
port "8088"
token "00000000-0000-0000-0000-000000000000"
ssl true
verifyssl false
Restart the CollectD service
service collectd restart
SECURITY INFO: for simplicity the token was all zeros but for security reasons best to change to another key
WARNING: This is not officially supported !
would also recommend setting hec to only listen to localhost, unless you are trying to receive remote calls to hec using acceptFrom
in inputs.conf
acceptFrom = ...
* Lists a set of networks or IP addresses from which to accept connections.
* Specify multiple rules with commas or spaces.
* Each rule can be in the following forms:
1. A single IPv4 or IPv6 address (examples: "10.1.2.3", "fe80::4a3")
2. A CIDR block of addresses (examples: "10/8", "fe80:1234/32")
3. A DNS name, possibly with a '*' used as a wildcard (examples:
"myhost.example.com", "*.splunk.com")
4. A single '*', which matches anything.
* You can also prefix an entry with '!' to cause the rule to reject the
connection. The input applies rules in order, and uses the first one that
matches. For example, "!10.1/16, *" allows connections from everywhere except
the 10.1.*.* network.
* Defaults to "*" (accept from anywhere)
Thanks for sharing!
Hey Luis!
Thanks for checking out insights for infra!!
While what your asking is totally doable, - to the best of my knowledge, at least - catching HEC with UF is not officially supported.
That being said, there are customers who do it and have no issues. You can chat with some of them on our slack chat, sign up here: splk.it/slack
I would still like to explore the option with you and help you try it out to see if you can achieve what makes deploying easier for you!
please hit me up at the email in my bio, or at @mattymo on Slack in our #insights-for-infra channel, where myself and the PM and ENG folks hang out!
We're also looking for a similar solution!
we got it working! Will Ask Luis to share his experience 🙂