I have a python script which makes an API call and get the events . Number of events, its collecting are correct however its adding duplicate entries per field. Can you please assist, I am ? Here is my script response = helper.send_http_request(rest_url, 'GET' ,parameters=queryParam, payload=None,headers=headers, cookies=None,verify=False, cert=None, timeout=None, use_proxy=False) r_headers = response.headers r_json = response.json() r_status = response.status_code if r_status !=200: response.raise_for_status() final_result = [] for _file in r_json: responseStr='' fileid = str(_file["fileid"]) state = helper.get_check_point(str(fileid)) if state is None: final_result.append(_file) helper.save_check_point(str(str(fileid)), "Indexed") event=helper.new_event(json.dumps(final_result), time=None, host=None, index=None, source=None, sourcetype=None, done=True, unbroken=True) ew.write_event(event) response: [ { "fileid": "abc.txt", "source": "source1", "destination": "dest1", "servername": "server1", }, { "fileid": "xyz.txt", "source": "source2", "destination": "dest2", "servername": "server2", } ] Response after collecting data to Index looks as below: fileid source destination servername "abc.txt abc.txt source1 source1 dest1 dest1 server1 server1 xyz.txt xyz.txt source2 source2 dest2 dest2 server2 server2
... View more