Splunk DB Connect: How do I avoid indexing duplicate data from an Oracle database table if some lines can be created with a delay up to 7 days?

[jbechchar]

Hi everybody,

Here is my problem:

I have a table in an Oracle database.
My Goal is to index each invoice once, and it's possible for some lines to be created with a 7 day delay.
Example : yesterday, a new line was added in my table with and older creation_invoice date.

So to be sure to get the line it, means that each day I have to get the data from the last 7 days. (I do not have a system creation date of each line) and use my creation_invoice date from timestamp.

The problem is when I get the data from the last 7 days, it will index some data which is already indexed.
I would like to index only the new line to avoid duplicate events.

Do you have any ideas ? I heard about kv stor,e but no idea if it can be help me.

Thanks in advance for yours answers.

Regards,
Jordan

[araitz]

Try using _index_time, which is the time that data was indexed, rather than _time:

 index=whatever_indextime>[|stats count | eval search=(now()-300)]

[jbechchar]

Thanks for your answer.

But the script help to get the data I want but the data are already indexed.

I really would like to have a unique event for each invoice in my index. And not to have 2 events for an invoice.

I do not know if it's possible with my problem.