All Apps and Add-ons

Splunk Add-on Builder: when exporting JSON for auto-extracting, the following error occurs.

OBsecurity
Explorer

Hello.

i'm posting to an HTTP endpoint and exporting results as JSON for auto extracting:

json_data = r.json()

while using 'addevent' and 'writeevents' to index results:

helper.addevent(json_data,sourcetype=eventSourceType)
helper.writeevents(index="main",host="localhost", source="sourcetype")

an error occurred:
Error: coercing to Unicode: need string or buffer, list found

only converting JSON to string resolves the issue, but it's preventing the results from being auto extracted.

is there a workaround other than convert JSON to string?

Thanks.

0 Karma

chli_splunk
Splunk Employee
Splunk Employee

Usually when we got a big json, especially it's a list, we should iterate each element and call addevent() in a loop. However, if you want to compose a big event with all the elements, it's fine, Splunk will extract fields automatically only if it's a valid JSON.

0 Karma

OBsecurity
Explorer

Hey!
thanks for your comment.
my issue is not big json, but passing dict\list object to 'addevent' function.
even if i iterate it (see snip below). its still passing it as dict\list object. so error is still happening - signature="Error: coercing to Unicode: need string or buffer, list found.

passing it as string resolve it. the cons is the the results are not extracted automaticly
its all indexed as one line.

thanks

helper.addevent("",
sourcetype="sample_sourcetype")
for line in r.iter_lines():
if line:
print(json.loads(line))
helper.addevent(json.loads(line),
sourcetype=eventSourceType)
helper.writeevents(index="main", host="localhost", source="abuseipdb")

0 Karma

chli_splunk
Splunk Employee
Splunk Employee

addevent() only accepts string as a param instead of JSON.
If it's a valid JSON, Splunk should be able to extract events automatically. If it's not, please try to add KV_MODE=json in your props.conf under the sourcetype stanza.

0 Karma
Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...