hi guys
I am using the Splunk add-on builder to collect logs from a 3rd partu REST API. I am using the checkpoint attribute so the collection can be incremental every time.
My problem is that the last event of every collection is always the first one of the next collection, so I am indexing one duplicate event with every new data collection I do.
Example:
First data collection:
Event 1 with date: 1-1-2018
Event 2 with date: 2-1-2018
Second data collection (using the date of Event 2 as checkpoint):
Event 2 with date: 2-1-2018
Event 3 with date: 3-1-2018
Event 4 with date: 4-1-2018
Third data collection (using the date of Event 4 as checkpoint)
Event 4 with date: 4-1-2018
Event 5 with date: 5-1-2018
This is probably a bug in the REST API, but I would like to know if there is a way of getting the checkpoint value and formatting it by adding 1 hour, or day, or minute....
any ideas about how I could get this solved?? I don´t want to index duplicate events...
