dbconnect - data base input (dump) - double events every time it runs

[cramasta]

I have a db input setup to take a dump using a query once a day.

My settings are
-Dump
-I have a custom query
-Key-Value Format
-Include Timestamp

Everytime that the input runs i get duplicates of each event. My query returns results that dont contain a timestamp which is why I configured the input to create one. Each duplicate event will have the same splunk generated timestamp.

If i run the same query with the dbquery command i get the correct number of results

Any ideas why this is happening?

[ziegfried]

Have you upgraded from an older version of DB Connect/DBX? If so, which version?

[Dan]

I believe that is expected behavior for the dump command. The full results of the query will be indexed every time. If you don't have a suitable rising column in the table, you will not be able to get just the new events.

Can you please post your custom query, or better yet, the contents of inputs.conf?

Thanks!

[cramasta]

Hi Dan.

The thing is I have the dump setup as a cron job that runs once a day. If I clear the index and wait till the next time the job runs I find duplicate events for each row returned from the query. Im letting splunk generate the index time for each event and I am finding that each duplicate event has the same index time. I would expect to see different timestamps if it was from a previous dump. I found with troubleshooting that setting the input to use the table name instead of the query will only index the table data x1. Ill add my inputs.conf and query shortly.

Thanks,
Joe

[cramasta]

FYI - If i tell it to dump the table instead of a query it doesn't index events x2