All Apps and Add-ons

Splunk Addon data-inputs duplicate events

jawaharas
Motivator

Hello,

I have created an custom add-on to pull events from 'Teachworks' API. But, as per my config (refer screenshot), duplicate records are created whenever the API call try pull the new events.

Example:
Run 1: 5 records available. 5 records pulled into Splunk
Run 2: 5 records available. 0 records pulled into Splunk
Run 3: 6 records available. 6 records pulled into Splunk

I expect only 1 record (new entry) to be pulled into Splunk during 'Run 3', not all 6 records. Any assistance will be helpful.

alt text

0 Karma
1 Solution

jawaharas
Motivator

The issue was due to missing parameter in the REST API URL.

When I add the checkpoint parameter in the REST API call (as below), I don't see duplicate events being pulled.
https://api.teachworks.com/v1/lessons?from_date[gt]>01-01-2020

Reference:
https://docs.splunk.com/Documentation/AddonBuilder/3.0.1/UserGuide/ConfigureDataCollection#Use_check...

View solution in original post

0 Karma

jawaharas
Motivator

The issue was due to missing parameter in the REST API URL.

When I add the checkpoint parameter in the REST API call (as below), I don't see duplicate events being pulled.
https://api.teachworks.com/v1/lessons?from_date[gt]>01-01-2020

Reference:
https://docs.splunk.com/Documentation/AddonBuilder/3.0.1/UserGuide/ConfigureDataCollection#Use_check...

View solution in original post

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!