Getting Data In

Universal Forwarder 6.4.0 to HEC

acamilo_2
New Member

Unfortunately, I have a few hundred hosts running 6.4 universal forwarder and I cannot upgrade them. I have a subset of hosts that need to send an application log to HEC on customer's splunk deployment but still send this and all the other logs to our splunk. We already have a few apps for the forwarder to handle different logs in a different way so I decided to create an app for those forwarders to send to the customer so that I can make sure that only the application log gets sent to them.
Hosts are Ubuntu 14.04
Splunk servers are on 16.04 and we run Splunk Enterprise 6.5.2
We don't use HEC in any of our hosts hence why I'm asking for help here.

I'm confident about all the config files except output.conf, I'm not clear on what defaultGroups does or if it's even necessary and the tcpout:app_upstart_logs stanza is also nebulous to me.

output.conf
[tcpout]
defaultGroup = ???
forwardedindex.filter.disable = true
indexAndForward = false
maxQueueSize = 250MB
maxConnectionsPerIndexer = 5

[tcpout:app_upstart_logs]
useACK = false
token = XXXXXXXXXXXXXXXXXXXXXXXXXX
server = customer.splunk-server.net:8080

This is my inputs.conf

[monitor:///var/log/upstart/]
disabled = false
sourcetype = app_upstart_logs
blacklist = (\.gz$|\.0$|.1$|\.2$|.3$|\.4$|.5$|\.6$|.7$|\.8$|\.9$|\.10$|\.report$|lost\+found)

PS: We are running some really old stuff. Feel free to roll your eyes like a teenage valley girl (is that still a thing?) I stand ashamed.

0 Karma

bandit
Motivator

An http event collector (hec) is essentially a heavy forwarder with the special hec listener. You would use the same sort of outputs.conf that you would use on any search head or heavy forwarder to route logs to your downstream indexers.

Example outputs.conf config:

# BASE SETTINGS

# TURN OFF INDEXING ON SEARCH HEAD OR HEAVY FORWARDER
[indexAndForward]
index = false

[tcpout]
# FORWARD ALL INTERNAL LOGS
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =

# round robin to all indexers
defaultGroup = all_indexers

[tcpout:all_indexers]
server = host500.acme.com:9997,host501.acme.com:9997,host502.acme.com:9997,host503.acme.com:9997,host504.acme.com:9997
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...