All Apps and Add-ons

Splunk Add-on Builder: when exporting JSON for auto-extracting, the following error occurs.

OBsecurity
Explorer

Hello.

i'm posting to an HTTP endpoint and exporting results as JSON for auto extracting:

json_data = r.json()

while using 'addevent' and 'writeevents' to index results:

helper.addevent(json_data,sourcetype=eventSourceType)
helper.writeevents(index="main",host="localhost", source="sourcetype")

an error occurred:
Error: coercing to Unicode: need string or buffer, list found

only converting JSON to string resolves the issue, but it's preventing the results from being auto extracted.

is there a workaround other than convert JSON to string?

Thanks.

0 Karma

chli_splunk
Splunk Employee
Splunk Employee

Usually when we got a big json, especially it's a list, we should iterate each element and call addevent() in a loop. However, if you want to compose a big event with all the elements, it's fine, Splunk will extract fields automatically only if it's a valid JSON.

0 Karma

OBsecurity
Explorer

Hey!
thanks for your comment.
my issue is not big json, but passing dict\list object to 'addevent' function.
even if i iterate it (see snip below). its still passing it as dict\list object. so error is still happening - signature="Error: coercing to Unicode: need string or buffer, list found.

passing it as string resolve it. the cons is the the results are not extracted automaticly
its all indexed as one line.

thanks

helper.addevent("",
sourcetype="sample_sourcetype")
for line in r.iter_lines():
if line:
print(json.loads(line))
helper.addevent(json.loads(line),
sourcetype=eventSourceType)
helper.writeevents(index="main", host="localhost", source="abuseipdb")

0 Karma

chli_splunk
Splunk Employee
Splunk Employee

addevent() only accepts string as a param instead of JSON.
If it's a valid JSON, Splunk should be able to extract events automatically. If it's not, please try to add KV_MODE=json in your props.conf under the sourcetype stanza.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...