Modular Input Checkpoint

[sanjay_shrestha]

I am writing a modular input and the script pulls list of the records in each interval when it runs.

e.g.

Name       Address
Joe         Ankeny
Bob        Clive

I do get duplicate events as I have not implemented Checkpoint yet. Since the script would bring all rows every time, do I need to save this every single row in checkpoint file and run through verification if row exists in the file or not?

Thanks,
Sanjay

[MuS · Accepted solution]

Hi sanjay.shrestha,

if your script pulls in all rows on each run, your approach sounds good.

It would be easier if you could use some timestamp to get the data in; in this case you can safe the last time the script ran in the checkpoint and use this last time stamp in the next script run.
I have some modular inputs doing exactly this.

Hope this helps ...

cheers, MuS

[sanjay_shrestha]

Thanks Michael. I will implement your advise and let you know.