Deployment Architecture

Splunk DB Connect: If two Splunk servers create a database input from TableA, will this cause duplicates to be indexed?

oraclebox
Explorer

If two Splunk server create a Database input from table A, will this cause any conflict? Will both Server1 and Server2 import to index if a new record is inserted into table A?

0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

Yes, both Splunk inputs will grab the same [full] set of data, though perhaps at different times. Inputs like these should be independent. See below for more explanation.

Assume you have a DB input on Splunk Server 1 that gets its input from, say, DBServerA, TableX. Then another DB input on Splunk Server 2 that gets its input from DBServerA, TableX as well.

Each Splunk server runs a SQL Query like select * from TableX {{ where $rising_column$ > ?}}

The first time through, the {{ ... }} is left off so the input can grab all the data. After that, each Splunk server will remember the last value for whatever has been declared as the $rising_column$ and use that for subsequent queries. Therefore, each server will get a copy of all rows that are newer than the last row that server had last requested, regardless of what other queries may have happened in the interim (applications, other Splunk servers, etc...)

View solution in original post

Richfez
SplunkTrust
SplunkTrust

Yes, both Splunk inputs will grab the same [full] set of data, though perhaps at different times. Inputs like these should be independent. See below for more explanation.

Assume you have a DB input on Splunk Server 1 that gets its input from, say, DBServerA, TableX. Then another DB input on Splunk Server 2 that gets its input from DBServerA, TableX as well.

Each Splunk server runs a SQL Query like select * from TableX {{ where $rising_column$ > ?}}

The first time through, the {{ ... }} is left off so the input can grab all the data. After that, each Splunk server will remember the last value for whatever has been declared as the $rising_column$ and use that for subsequent queries. Therefore, each server will get a copy of all rows that are newer than the last row that server had last requested, regardless of what other queries may have happened in the interim (applications, other Splunk servers, etc...)

jcoates_splunk
Splunk Employee
Splunk Employee

you'll get a duplicate copy of your table.
DB1.TableA -> SplunkX.Input1A -> Index1
DB1.TableA -> SplunkY.Input1A -> Index1

Index1 now contains:
DB1.TableA.Row1
DB1.TableA.Row1
DB1.TableA.Row2
DB1.TableA.Row2
et cetera et cetera

Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...