Deployment Architecture

How to troubleshoot why Splunk DB Connect 1.1.7 suddenly stopped indexing data from our Oracle database?

nawneel
Communicator

I am currently experiencing issue in our production environment and wanted to check if any of you have encountered similar issue. I was getting data in index using Splunk DB Connect 1 since 2 days back, but suddenly it stopped indexing data. The following is our DB Connect configuration:

  1. We are dumping data from Oracle database to Splunk index.
  2. Splunk versions is 6.2.2
  3. DB connect version is 1.1.7
  4. DB input strategy is DB Dump
  5. Output format is Multi-Line key Format.

When I run a query in the DB Connect query browser, it returns results successfully.
There are NO errors in splunkd logs, dbx_debug shows that this dbinput triggered and completed gracefully.

Splunk DB Connect app gets data from the database and keeps in SPLUNK_HOME/var/spool/dbmon directory. From where it runs a batch job to write data to index using policy sinkhole, on successful indexing of data, it deletes the temporary file from in SPLUNK_HOME/var/spool/dbmon directory.

I have data in SPLUNK_HOME/var/spool/dbmon, but not in the index.

I also have some other indexes which are getting the data from same Oracle db using the same DB Connect 1.1.7 and they continue to work fine while only few indexes have stopped working.

Kindly suggest appropriate actions.

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

General principles: if it worked and then stops, something has changed. What changed? Assuming that all the applicable logs are in Splunk, set the search window to the time when it stops working and look for errors... or even just

(* OR index=_internal) | dedup punct

to see a rough idea of what types of things are in there.

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...