topic Python script json in Splunk Dev

Python script json

csimonnet — Tue, 29 Sep 2020 19:44:13 GMT

Hi,
I have a python script which return a json file like this:

{"address": "Pont Raymond Barre", "address2": "Parvis du Musée des Confluences", "availability": "Vert", "availabilitycode": "1", "available_bike_stands": "2", "available_bikes": "17", "banking": "False", "bike_stands": "20", "bonus": "Non", "code_insee": "69382", "commune": "Lyon 2 ème", "gid": "1107", "last_update": "2018-05-31 10:33:44", "last_update_fme": "2018-05-31 10:35:07", "lat": "45.7333902949857180", "lng": "4.8189455344322543", "name": "Musée des Confluences", "nmarrond": "20", "number": "2020", "pole": "Musée des Confluences", "status": "OPEN"}, {"address": "5 rue Félix Rollet", "address2": "None", "availability": "Gris", "availabilitycode": "0", "available_bike_stands": "0", "available_bikes": "0", "banking": "True", "bike_stands": "20", "bonus": "None", "code_insee": "69383", "commune": "Lyon 3 ème", "gid": "1108", "last_update": "2018-05-31 00:30:02", "last_update_fme": "2018-05-31 00:30:02", "lat": "45.7483920031766260", "lng": "4.8759903056684522", "name": "Friche RVI", "nmarrond": "50", "number": "3050", "pole": "ancienne friche RVI", "status": "OPEN"}

It works perfectly when I use the command sudo /opt/splunk/bin/splunk cmd python /opt/splunk/etc/apps/bike_sharing/bin/script_velo.py in my terminal but when I try to add it in Splunk, no results are found.
Why it doesn't work properly?

Thank you for your answer

Re: Python script json

kamlesh_vaghela — Thu, 31 May 2018 10:12:00 GMT

Hi @csimonnet,

Can you please share configurations file?
Meanwhile you can cross verify your configuration using following link.

https://docs.splunk.com/Documentation/Splunk/7.1.0/AdvancedDev/ScriptSetup

Re: Python script json

poete — Thu, 31 May 2018 11:28:53 GMT

What do you mean when you write 'it doesn't work properly?'
Is there an error of the script, or the info are not indexed by Splunk?
I have a python script building an input that is indexed by splunk.
The result indexed by Splunk is in fact the result of a json.dumps(somePythonDico). Please note that this result is different from yours, and would look like

 **[**{"address": "Pont Raymond Barre", "address2": "Parvis du Musée des Confluences", "availability": "Vert", "availabilitycode": "1", "available_bike_stands": "2", "available_bikes": "17", "banking": "False", "bike_stands": "20", "bonus": "Non", "code_insee": "69382", "commune": "Lyon 2 ème", "gid": "1107", "last_update": "2018-05-31 10:33:44", "last_update_fme": "2018-05-31 10:35:07", "lat": "45.7333902949857180", "lng": "4.8189455344322543", "name": "Musée des Confluences", "nmarrond": "20", "number": "2020", "pole": "Musée des Confluences", "status": "OPEN"}, {"address": "5 rue Félix Rollet", "address2": "None", "availability": "Gris", "availabilitycode": "0", "available_bike_stands": "0", "available_bikes": "0", "banking": "True", "bike_stands": "20", "bonus": "None", "code_insee": "69383", "commune": "Lyon 3 ème", "gid": "1108", "last_update": "2018-05-31 00:30:02", "last_update_fme": "2018-05-31 00:30:02", "lat": "45.7483920031766260", "lng": "4.8759903056684522", "name": "Friche RVI", "nmarrond": "50", "number": "3050", "pole": "ancienne friche RVI", "status": "OPEN"}**]**

In addition, check the content of the inputs.conf file, as described in https://docs.splunk.com/Documentation/Splunk/7.1.0/AdvancedDev/ScriptSetup

Re: Python script json

poete — Thu, 31 May 2018 11:29:51 GMT

[ is meant for a bold [

Re: Python script json

csimonnet — Tue, 29 Sep 2020 19:44:24 GMT

inputs.conf

[script://$SPLUNK_HOME/etc/apps/bike_sharing/bin/script_velo.py]
disabled = false
index = bike_sharing
interval = 60.0
sourcetype = _json_velo_rest

props.conf

[_json_velo_rest]
BREAK_ONLY_BEFORE =
DATETIME_CONFIG =
KV_MODE = json
LINE_BREAKER = }(,)\s{"status"
NO_BINARY_CHECK = true
SEDCMD-add_closing_bracket = s/\s$/ }/g
SEDCMD-correctly-close = s/}\s}\s}/\n/g
TIME_PREFIX = "last_update"
TRUNCATE = 0
category = Custom
crcSalt = <SOURCE>
description = json pour API rest velo
disabled = false
pulldown_type = 1
SHOULD_LINEMERGE = false
EVAL-taux_occupation = available_bikes/bike_stands

transform.conf is empty