Heavy Forwarder vs. Reduced Splunk Enterprise & DB Connect App

[thomastaylor]

Hello everyone!

My team and I are attempting to create a service for our departments' applications that enable them to easily send logs to our Splunk Enterprise; however, we do not control the Splunk Enterprise since it's handled by another department. We are essentially an intermediary between the Splunk department and our department to create an easy-to-implement solution.

We are also restricted to only sending logs by either Universal Forwarder or Heavy Forwarder. We have seen the discouragement associated with the heavy forwarder, and we would like to get a few things cleared up. Please, correct us if we're wrong in any of these bulletin points:

1. Universal Forwarder is the way-to-go. Only has the ability to monitor files / directories / system logs. Does not index. It cannot view logs stored IN A DATABASE column.
2. The Heavy Forwarder can be implemented as a "slave" to prevent license usage so that it acts strictly as a forwarder. It can take HTTPs Event as an input and forward it onto the Splunk department WITHOUT impeding our usage. It can support DB Connect App for forwarding logs over to the Splunk environment. It does NOT have a web interface.
3. A Splunk Enterprise Instance can be configured to be a slave and NOT act as an indexer (how difficult is this?). We would potentially want to do this so that we have access to a website interface, have the capabilities of extraction, and have the availability to access db connect app from an interface view.

One thing to note here, we are creating libraries in Python and Java that can extend applications' loggers to add our easy-to-implement heavy forwarder or Splunk instance. It would essentially be through either HTTPs, UDP, or TCP.

One more question: if we had a db connect app on a heavy forwarder, could multiple applications hosted on different machines / servers connect to the database connect app?

Does Splunk Light come into this at all?

[FrankVl]

Not entirely sure what you mean by a Heavy Forwarder, if you expect it to not have a web interface and be different from a Splunk Enterprise instance that doesn't index locally. Because a Heavy Forwarder is just that: a full Splunk Enterprise instance which is configured to forward data rather than index it. And as such, it does have a web gui, for instance for managing apps like DBConnect.

DBConnect on a HF can be used to pull logs from multiple databases. You can configure multiple DB connections and corresponding data inputs. Note: DB Connect connects to the DB to read the logs, applications don't connect to DB Connect.

[richgalloway · Accepted solution]

Allow me to clarify a few things.

1) The universal forwarder is for monitoring files and direcftories. It cannot run Python scripts and does not have a UI.

2) Heavy Forwarders are just Splunk instances that don't index. They have the full power of Splunk, including the UI and the ability to run Python, HTTP Event Collector (HEC), DB Connect, and other apps.

3) A Splunk instance that doesn't index is called a Heavy Forwarder.

If you need to run Python scripts, HEC, or DB Connect then you should use a heavy forwarder. Otherwise, use a universal forwarder. The universal forwarder uses less resources.

Applications don't connect to DB Connect. DB Connect makes connections to databases and runs SQL queries to extract data.

[thomastaylor]

This was exactly what we were looking for. Thank you so much for providing your insight!