Deployment Architecture

Splunk DB Connect: If two Splunk servers create a database input from TableA, will this cause duplicates to be indexed?

oraclebox
Explorer

If two Splunk server create a Database input from table A, will this cause any conflict? Will both Server1 and Server2 import to index if a new record is inserted into table A?

0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

Yes, both Splunk inputs will grab the same [full] set of data, though perhaps at different times. Inputs like these should be independent. See below for more explanation.

Assume you have a DB input on Splunk Server 1 that gets its input from, say, DBServerA, TableX. Then another DB input on Splunk Server 2 that gets its input from DBServerA, TableX as well.

Each Splunk server runs a SQL Query like select * from TableX {{ where $rising_column$ > ?}}

The first time through, the {{ ... }} is left off so the input can grab all the data. After that, each Splunk server will remember the last value for whatever has been declared as the $rising_column$ and use that for subsequent queries. Therefore, each server will get a copy of all rows that are newer than the last row that server had last requested, regardless of what other queries may have happened in the interim (applications, other Splunk servers, etc...)

View solution in original post

Richfez
SplunkTrust
SplunkTrust

Yes, both Splunk inputs will grab the same [full] set of data, though perhaps at different times. Inputs like these should be independent. See below for more explanation.

Assume you have a DB input on Splunk Server 1 that gets its input from, say, DBServerA, TableX. Then another DB input on Splunk Server 2 that gets its input from DBServerA, TableX as well.

Each Splunk server runs a SQL Query like select * from TableX {{ where $rising_column$ > ?}}

The first time through, the {{ ... }} is left off so the input can grab all the data. After that, each Splunk server will remember the last value for whatever has been declared as the $rising_column$ and use that for subsequent queries. Therefore, each server will get a copy of all rows that are newer than the last row that server had last requested, regardless of what other queries may have happened in the interim (applications, other Splunk servers, etc...)

jcoates_splunk
Splunk Employee
Splunk Employee

you'll get a duplicate copy of your table.
DB1.TableA -> SplunkX.Input1A -> Index1
DB1.TableA -> SplunkY.Input1A -> Index1

Index1 now contains:
DB1.TableA.Row1
DB1.TableA.Row1
DB1.TableA.Row2
DB1.TableA.Row2
et cetera et cetera

Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...