automated query results from run to run

[codedtech]

I'm creating a query that runs every day at 03:00 I need to use the field "INSERT_DATE" as my time entry. Its currently formatted like this 2019-08-12 00:10:33.0, I need to find a way to get all of the previous dates in and all dates going forward with out duplicating any data.

[niketn]

@codedtech for the community to assist you better, you need to add way more details than this. You need to explain with sample data that you have and Splunk query you are using right now. Where exactly is the issue? What do you mean by get all of the previous dates in and all dates going forward?

[codedtech]

I'm Using DBX to query a database that is updated every day at 00:10:00 and add these results to my index. due to logistical issues, I can not use DBX to do this automatically. I was told to use the collect command to do this or manipulate the DB.

This is the base query:
dbxquery query=myQUery |eval Used_GB= VDC_TOTAL_CAP_GB-VDC_FREE_CAP_GB|eval Pool_Perc = Used_GB/VDC_TOTAL_CAP_GB|eval _time=strptime(INSERT_DATE, "%Y-%m-%d %H:%M:%S")|bin span=1d _time

I need to find a way to only new events that occur between 00:00:00, and 03:00 (of today)into my index, with out adding duplicates of entries from the previous run(Yesterday), and being dynamic enough so that I never need to manually run the query.