All Apps and Add-ons

How to forward data from DB Connect 3 to multiple indexers?

vincenteous
Communicator

Hello Experts,

Recently our client decided to ingest data from their database servers to the existing Splunk environment. The existing Splunk environment is like this:

  • 3 Search Heads under a load balancer (no cluster)
  • 4 Indexers (no cluster)
  • 3 Heavy Forwarders

We have already installed Splunk DB Connect 3 in one of the heavy forwarders and we can clearly see the query result from the target database using SQL Explorer. We have also made sure that the HTTP Event Collector port is not blocked or anything so no bind port error in splunkd.log. Several data inputs have also been created. Unfortunately, we still don't know how to forward the events to the indexers.

Here is the outputs.conf we created in $SPLUNK_HOME/etc/system/local:

[default]
defaultGroup = hf_load_balance

[tcpout:hf_load_balance]
compressed = true
server = <idx1>:9997, <idx2>:9997, <idx3>:9997, <idx4>:9997
sslCertPath = /apps/splunk/etc/auth/servercert.pem
sslPassword = $1$bqkDNmCaJfWrxZxBi5bW
sslRootCAPath = /apps/splunk/etc/auth/CoreCA.pem
sslVerifyServerCert = true

And here is one of the inputs created (configuration from db_inputs.conf):

[UXP_Track_Logs]
connection = UXP
description = UXP DB Track_Logs Table
disabled = 0
index = app_uxpdb
index_time_mode = dbColumn
input_timestamp_column_number = 2
interval = 0,15,30,45 * * * *
mode = rising
query = <query for UXP>
query_timeout = 300
sourcetype = uxptracklogs
tail_rising_column_number = 2

The index app_uxpdb exists in all target indexers.

From DB Connect 3 documentation, only indexes.conf configuration is mentioned but not the way to forward the data. Can somebody please guide me for this one?

Thank you.

0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

No, it will not index locally because you already have outputs.conf in place which will send data to all indexers but still blank index is require on heavy forwarder so please create app_uxpdb index on heavy forwarder and it will send data to all indexers because app_uxpdb is already present on all indexers based on your question.

View solution in original post

harsmarvania57
SplunkTrust
SplunkTrust

No, it will not index locally because you already have outputs.conf in place which will send data to all indexers but still blank index is require on heavy forwarder so please create app_uxpdb index on heavy forwarder and it will send data to all indexers because app_uxpdb is already present on all indexers based on your question.

vincenteous
Communicator

I see. It looks like I missed that one. Thank you.
Please convert your comment to answer so I can accept yours.

0 Karma

anjambha
Communicator

Hello vincenteous,

You can do this by sending data over default port 9997 with general configuration and without HEC .
Make sure connection between indexer and UF is proper and also you have created app_uxpdb on both UF and indexers.

0 Karma

vincenteous
Communicator

Hi,

Index app_uxpdb has already been created in all indexers. It looks like I missed creating a blank index in the HF to trigger the forwarding mechanism as harsmarvania57 pointed out. Thank you for answering.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Your mention of HTTP event collector is throwing me off... 🙂 Port 9997 is the default port for the Splunk-2-Splunk TCP listener and your regular steps to enable forwarding apply.

Can you please clarify your use of the HTTP event collector in this context? I suspect it's just a terminology issue and your problem is likely that you did not enable the SplunkForwarder app.

0 Karma

vincenteous
Communicator

Hi,

I was following the troubleshooting documentation about HEC in db connect. Here it is: http://docs.splunk.com/Documentation/DBX/latest/DeployDBX/Troubleshooting#Debug_HTTP_Event_Collector...

At first, because my splunkd port was set to 8088, HEC created by DB Connect cannot bind the default 8088 TCP port. In that case, I changed the port to something else and no error again.

I have checked the splunkd.log and found the entry "Connected to ...." so the connection seems to be fine.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

Have you created app_uxpdb index on heavy forwarder on which DB connect is installed?

0 Karma

vincenteous
Communicator

Hi,

No, I haven't. Won't it cause Splunk to index locally instead? I don't know, maybe I'm mistaken here.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...