Is there any way to do checkpoints or avoid duplicate events when make a REST API call?

[ansif]

I am building an addon for Networker using REST API input.

Not able to configure checkpoints because REST API not support greater than or lesser than symbols to get last 5 mins data or so.

Is there any way to do checkpoints or avoid duplicate events when make a REST API call?

[mydog8it]

The Dell EMC NetWorkers getting started with API documentation speaks to time filtering in chapter 4 page 35:
https://www.dellemc.com/pl-pl/collaterals/unauth/technical-guides-support-information/products/data-...

[ansif]

This works with time ranges. How can I give this as checkpoint?

[mydog8it]

If you want to truly make this a checkpoint of the message, you might consider writing a checkpoint module that you script runs after it downloads the data. I have done something similar in powershell, but not python. I took a list of two columns, concatenated each pair and then hashed them. I then searched the hashes for previous matches.
You could do the same thing with the last message read. Hash all the messages and save and save the last one in a batch as a checkpoint. When you pull the next batch of messages only forward the ones that come in after your checkpoint hash is matched.

[ansif]

Where to keep REST API response or which module I need to modify to call other script that takes the response JSON and do check ?

[dmarling]

If you could provide the rest call that is failing and the failure message that would help us answer your question. My first guess is if you are attempting to pass in a greater than or less than symbol in the body of the transmission, you most likely need to make it a utf-8 url encoded greater than: %3E or less than: %3C

[ansif]

The REST API itself not supports to query timestamp greater than or lesser than.

For this situation I need help to adjust in script to create a checkpoint of id's that is indexed already.

https://${nw_server_hostname}:9090/nwrestapi/v3/global/${endpoint}?q=completionStatus: "Failed"

This end point gives me backup failed events like below:

{
"count":6,
"jobs":[
{
"adhocJob":false,
"command":"/xxx/xxxx/xxxxxxx -C DefaultReportHomeTask",
"completionStatus":"Failed",
"dependentJobIds":[
0
],
"endTime":"2020-01-16T08:00:00+08:00",
"exitCode":1,
"id":221372,
"itemIdLong":221372,
"links":[
{
"href":"https://{nw_server_hostname}:9090/nwrestapi/v3/global/jobs/221372",
"rel":"item"
}
],
"name":"DefaultReportHomeTask",
"ndmp":false,
"parentJobId":0,
"previousJobId":0,
"rootParentJobId":0,
"runOnHost":"{nw_server_hostname}.xxxxxxx.com",
"siblingJobIds":[
],
"startTime":"2020-01-16T08:00:00+08:00",
"state":"Completed",
"stopped":true,
"tenant":"",
"type":"task job"
},
{
"adhocJob":false,
"command":"/xxx/xxxx/xxxxxxx -C DefaultNsrclientfixTask",
"completionStatus":"Failed",
"dependentJobIds":[
0
],
"endTime":"2020-01-16T07:00:00+08:00",
"exitCode":1,
"id":221371,
"itemIdLong":221371,
"links":[
{
"href":"https://{nw_server_hostname}:9090/nwrestapi/v3/global/jobs/221371",
"rel":"item"
}
],
"name":"DefaultNsrclientfixTask",
"ndmp":false,
"parentJobId":0,
"previousJobId":0,
"rootParentJobId":0,
"runOnHost":"{nw_server_hostname}.xxxxxxx.com",
"siblingJobIds":[
],
"startTime":"2020-01-16T07:00:00+08:00",
"state":"Completed",
"stopped":true,
"tenant":"",
"type":"task job"
},
{
"adhocJob":false,
"command":"/xxx/xxxx/xxxxxxx -C DefaultReportHomeTask",
"completionStatus":"Failed",
"dependentJobIds":[
0
],
"endTime":"2020-01-15T08:00:00+08:00",
"exitCode":1,
"id":221124,
"itemIdLong":221124,
"links":[
{
"href":"https://{nw_server_hostname}:9090/nwrestapi/v3/global/jobs/221124",
"rel":"item"
}
],
"name":"DefaultReportHomeTask",
"ndmp":false,
"parentJobId":0,
"previousJobId":0,
"rootParentJobId":0,
"runOnHost":"{nw_server_hostname}.xxxxxxx.com",
"siblingJobIds":[
],
"startTime":"2020-01-15T08:00:00+08:00",
"state":"Completed",
"stopped":true,
"tenant":"",
"type":"task job"
},
{
"adhocJob":false,
"command":"/xxx/xxxx/xxxxxxx -C DefaultNsrclientfixTask",
"completionStatus":"Failed",
"dependentJobIds":[
0
],
"endTime":"2020-01-15T07:00:00+08:00",
"exitCode":1,
"id":221123,
"itemIdLong":221123,
"links":[
{
"href":"https://{nw_server_hostname}:9090/nwrestapi/v3/global/jobs/221123",
"rel":"item"
}
],
"name":"DefaultNsrclientfixTask",
"ndmp":false,
"parentJobId":0,
"previousJobId":0,
"rootParentJobId":0,
"runOnHost":"{nw_server_hostname}.xxxxxxx.com",
"siblingJobIds":[
],
"startTime":"2020-01-15T07:00:00+08:00",
"state":"Completed",
"stopped":true,
"tenant":"",
"type":"task job"
},
{
"adhocJob":false,
"command":"/xxx/xxxx/xxxxxxx -C DefaultReportHomeTask",
"completionStatus":"Failed",
"dependentJobIds":[
0
],
"endTime":"2020-01-14T08:00:00+08:00",
"exitCode":1,
"id":220875,
"itemIdLong":220875,
"links":[
{
"href":"https://{nw_server_hostname}:9090/nwrestapi/v3/global/jobs/220875",
"rel":"item"
}
],
"name":"DefaultReportHomeTask",
"ndmp":false,
"parentJobId":0,
"previousJobId":0,
"rootParentJobId":0,
"runOnHost":"{nw_server_hostname}.xxxxxxx.com",
"siblingJobIds":[
],
"startTime":"2020-01-14T08:00:00+08:00",
"state":"Completed",
"stopped":true,
"tenant":"",
"type":"task job"
},
{
"adhocJob":false,
"command":"/xxx/xxxx/xxxxxxx -C DefaultNsrclientfixTask",
"completionStatus":"Failed",
"dependentJobIds":[
0
],
"endTime":"2020-01-14T07:00:00+08:00",
"exitCode":1,
"id":220874,
"itemIdLong":220874,
"links":[
{
"href":"https://{nw_server_hostname}:9090/nwrestapi/v3/global/jobs/220874",
"rel":"item"
}
],
"name":"DefaultNsrclientfixTask",
"ndmp":false,
"parentJobId":0,
"previousJobId":0,
"rootParentJobId":0,
"runOnHost":"{nw_server_hostname}.xxxxxxx.com",
"siblingJobIds":[
],
"startTime":"2020-01-14T07:00:00+08:00",
"state":"Completed",
"stopped":true,
"tenant":"",
"type":"task job"
}
]
}

I can only do a check point with endTime field.

But cant make a call with "endTime" greater than some timestamp.The vendor REST API is not supported with greater than or less than symbols(https://www.support.nec.co.jp/DownLoad.aspx?file=document%2Fdocu91962_NetWorker-18.2-REST-API-Gettin...)

Is there any way to do a checkpoint so that I can make sure that no duplicate events I am getting everytime make a REST API call.

NB:- id field is unique for each event. Is ther a way to do checkpoint using id field?

[jamsfreak]

Nice responde

but 

How to monitoring jobs with completionStatus = Failed and Abandoned

How to endpoint is made?

https://${nw_server_hostname}:9090/nwrestapi/v3/global/${endpoint}?q=completionStatus: "Failed" ????

Failed and status abandone

Try it in your lab and post the json response here