Seems Splunk is ingesting same data after few minutes and creating duplicate events. No problem with the log files ingestions, however data coming in from DB connect V2 is being ingested again and again.
Anyone facing similar problem ?
Use the raising column option. Set that to the date field. This will make sure you will not get duplicates.
If you only need to do it once, then set it up, let it run, and then disable it. Otherwise, it will run every time execution frequency time passes.
execution frequency
Is your rising column actually rising?
I am doing batch input. These are stationary data I am collecting only once from postgresql using the DB Connect app.
What is the execution frequency of the DB Connect 2 input?
I put 31104000 which is the number of seconds (approx) in a year. But I noticed after few minutes it ingested the data again.
Makes me wonder if the query failed and DBX2 is retrying. Is there anything in dbx2.log?
I dont see any failing information in the dbx2.log.
I am using my search as index=X source=Y sourcetype=Z
As soon as I disable the index (X) from Menu->Settings->Indexes the query says "No results found"
Why disable the index? @woodcock suggested disabling the DBX2 input once it has read everything once.
Don't disable the index, disable the input.
