All Apps and Add-ons

Splunk Addon data-inputs duplicate events

jawaharas
Motivator

Hello,

I have created an custom add-on to pull events from 'Teachworks' API. But, as per my config (refer screenshot), duplicate records are created whenever the API call try pull the new events.

Example:
Run 1: 5 records available. 5 records pulled into Splunk
Run 2: 5 records available. 0 records pulled into Splunk
Run 3: 6 records available. 6 records pulled into Splunk

I expect only 1 record (new entry) to be pulled into Splunk during 'Run 3', not all 6 records. Any assistance will be helpful.

alt text

0 Karma
1 Solution

jawaharas
Motivator

The issue was due to missing parameter in the REST API URL.

When I add the checkpoint parameter in the REST API call (as below), I don't see duplicate events being pulled.
https://api.teachworks.com/v1/lessons?from_date[gt]>01-01-2020

Reference:
https://docs.splunk.com/Documentation/AddonBuilder/3.0.1/UserGuide/ConfigureDataCollection#Use_check...

View solution in original post

0 Karma

jawaharas
Motivator

The issue was due to missing parameter in the REST API URL.

When I add the checkpoint parameter in the REST API call (as below), I don't see duplicate events being pulled.
https://api.teachworks.com/v1/lessons?from_date[gt]>01-01-2020

Reference:
https://docs.splunk.com/Documentation/AddonBuilder/3.0.1/UserGuide/ConfigureDataCollection#Use_check...

View solution in original post

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!