All Apps and Add-ons

Splunk Add-on Builder Python script - duplicate entries per field in Index

AshChakor
Path Finder

I have a python script which makes an API call and get the events . Number of events, its collecting are correct however its adding duplicate entries per field. Can you please assist, I am ?

Here is my script

response = helper.send_http_request(rest_url, 'GET' ,parameters=queryParam, payload=None,headers=headers, cookies=None,verify=False, cert=None, timeout=None, use_proxy=False)

r_headers = response.headers

r_json = response.json()

r_status = response.status_code

if r_status !=200:
    response.raise_for_status()

final_result = []

for _file in r_json:
    responseStr=''
    fileid = str(_file["fileid"])
    state = helper.get_check_point(str(fileid))
    if state is None:
        final_result.append(_file)
        helper.save_check_point(str(str(fileid)), "Indexed")
event=helper.new_event(json.dumps(final_result), time=None, host=None, index=None, source=None, sourcetype=None, done=True, unbroken=True)
ew.write_event(event)

response:

[
{
"fileid": "abc.txt",
"source": "source1",
"destination": "dest1",
"servername": "server1",
},
{
"fileid": "xyz.txt",
"source": "source2",
"destination": "dest2",
"servername": "server2",
}
]

Response after collecting data to Index looks as below:

fileidsourcedestinationservername

"abc.txt

abc.txt

source1

source1

dest1

dest1

server1

server1

xyz.txt

xyz.txt

source2

source2

dest2

dest2

server2

server2

Labels (2)
0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!