All Apps and Add-ons

Why is Splunk DB Connect 3 days behind indexing ePO data?

bluemarvel
Path Finder

We are seeing logs in Splunk but there are lagging; meaning they are always 3 days behind. I.E we can only see logs from the 14th and prior. Is there a some time format or other configuration that needs to be changed?

0 Karma
1 Solution

gjanders
SplunkTrust
SplunkTrust

As per the documentation https://docs.splunk.com/Documentation/DBX/3.0.1/DeployDBX/Createandmanagedatabaseinputs is it possible your max rows to retrieve is too small?
There is also a query timeout of 30 seconds within the db_inputs.conf you could possibly be hitting https://docs.splunk.com/Documentation/DBX/3.0.1/DeployDBX/dbinputs

View solution in original post

0 Karma

gjanders
SplunkTrust
SplunkTrust

As per the documentation https://docs.splunk.com/Documentation/DBX/3.0.1/DeployDBX/Createandmanagedatabaseinputs is it possible your max rows to retrieve is too small?
There is also a query timeout of 30 seconds within the db_inputs.conf you could possibly be hitting https://docs.splunk.com/Documentation/DBX/3.0.1/DeployDBX/dbinputs

0 Karma

bluemarvel
Path Finder

thank you for the advice, much appreciated. Still having and issue after I changed the max_rows value to all, (max_row = all) which is the default from what i have read. >>{/splunk/etc/apps/dbx/local/inputs.conf}

0 Karma

gjanders
SplunkTrust
SplunkTrust

Do the DB Connect logs in /opt/splunk/var/log/splunk/splunk_app_db_connect_* give you any hints?
You can also find them in the internal index, they might tell you waht is happening...

Furthermore, are you sure the records are old ? i.e. do you have an issue parsing the timestamps/using the wrong time column or similar ?

0 Karma

bluemarvel
Path Finder

hello again, no there are no error messages in the dbx.log

======================================================================================
2017-03-20 11:45:34.916 dbx4196:INFO:TailDatabaseMonitor - Database monitor=[dbmon-tail://epo_b-edc/epo_b-edc_5] finished with status=true resultCount=10000 in duration=886266 ms
2017-03-20 11:45:34.916 dbx4196:INFO:ExecutionContext - Execution finished in duration=886266 ms

2017-03-20 11:45:34.916 monsch1:INFO:Scheduler - Execution of input=[dbmon-tail://epo_b-edc/epo_b-edc_5] finished in duration=886266 ms with resultCount=10000 success=true continueMonitoring=true

what is happening is that the logs are coming, but they are three days behind, so today is the 20th, we can see logs from up to the 17th only. Tomorrow the 21st we will see logs up to the 18th.

0 Karma

gjanders
SplunkTrust
SplunkTrust

The logs indicate:
resultCount=10000

Did you try increasing this number to see if you just need more data per-request? Do you have more than 10K of rows per day? Or alternatively you can set the query to run more often/more times per day?

0 Karma

bluemarvel
Path Finder

So we decided to move on to DB Connect 2 for our EPO events, however I am seeing two different database drivers for this, whicn is the right one?

sqljdbc4.jar and postgresql-9.4.1212.jre6.jar

0 Karma

bluemarvel
Path Finder

is this also in the local.con file because I am not seeing it.
the other config change that I made proved not have worked.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...