Getting Data In

DBconnect cron schedule behavior

vin_ven27
Explorer

I used DBconnect to pull data from the database in every 1min *(cron: * /1 * * * *). I would like to ask if this schedule is simple as:

#1 - Run every e.g. 10:00, then 10:01 then 10:02 and so on... or?

#2 - Run every e.g. 10:00 (wait until the job done) example 10:00:35 so the job will run at 10:01:35. 

Please advise as we encountering missed of data when we tried the #1.  Thanks

Labels (1)
Tags (2)
0 Karma

vin_ven27
Explorer

thanks for a quick response @venkatasri 
Actually that is our first schedule setup which is every 5mins it's just so happen that the client want a data updated of what will be the possible minimum time. So we change to 1min cron schedule and the SQL set as batch and get only past 1min of the data using updated time column. For us to resolve the missing data our work around is to get 90secs past data instead of 1min in SQL and the cron still 1min. But not sure though if this work this out. Any idea?/suggestion?

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@vin_ven27  Yes that's every min you can verify same here - Crontab.guru - The cron schedule expression editor

Every min seems bit aggressive have you tried changing the schedule to every 5 mins at least, there could be other issues as well for missing logs.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

@vin_ven27  batch mode actually duplicates your data hence chances of missing your data is very slim. Did you check the DBConnect logs?

The time column that you are using sometimes transaction commits with delay in DB.  you have to check if there are delays to commit in DB, your DBA can help.

0 Karma
Get Updates on the Splunk Community!

Splunk Platform | Upgrading your Splunk Deployment to Python 3.9

Splunk initially announced the removal of Python 2 during the release of Splunk Enterprise 8.0.0, aiming to ...

From Product Design to User Insights: Boosting App Developer Identity on Splunkbase

co-authored by Yiyun Zhu & Dan Hosaka Engaging with the Community at .conf24 At .conf24, we revitalized the ...

Detect and Resolve Issues in a Kubernetes Environment

We’ve gone through common problems one can encounter in a Kubernetes environment, their impacts, and the ...