Getting Data In

Index input script json results -> Issue The json is not broken correctly when indexed by splunk

lpolo
Motivator

I have a set of input scripts that are working as expected. The problem I am facing is that I need to index the results but the event is not broken correctly. This is an example of the result :

[
{
"a": "4620",
"b": "splunk",
"x": "0",
"d": "3.0",
"e": "50",
"f": "0",
"g": "41.0",
"_time": "2014-01-17T10:26:43.000-05:00",
"h": "abc",
"i": "4620",
"j": "0.00",
"k": "21.0",
"l": "6.00"
},
{
"a": "4620",
"b": "ABC",
"x": "0",
"d": "3.0",
"e": "50",
"f": "0",
"g": "41.0",
"_time": "2014-01-17T10:26:43.000-05:00",
"h": "abc",
"i": "4620",
"j": "0.00",
"k": "21.0",
"l": "6.00"
}
]

This is what I have in the inputs.conf:

[script:///opt/splunk/bin/scripts/splunk-sdk-python/examples/abc.py]
disabled = 0
index = main
interval = */5 * * * *
sourcetype = feed 
Tags (2)
0 Karma

dolivasoh
Contributor

try sourcetype = json instead of feed

0 Karma

lpolo
Motivator

it breaks at any position.
The work around I did to make it work was to write the results in a file. Then, I monitor file using a monitoring stanza in inputs.conf. I tried to apply the same config to the script stanza but no success. The config I used in monitoring the file is the following:

LINE_BREAKER = "(^)["
TRUNCATE = 0
SHOULD_LINEMERGE = false

I should be able to index the results correctly without doing the presented work around.

Any idea?

Thanks,
Lp

0 Karma

laserval
Communicator

I believe the JSON log parser expects logs of the format:

{"key":"value"}
{"key":"othervalue"}

I.E. JSON log files aren't JSON, they are files with one JSON object per line.

So, remove the square brackets and the comma. If that doesn't work, try putting each event on a separate line (so drop the pretty-printing).

0 Karma

lpolo
Motivator

I have been able to index any type of valid json structure without any problem using the monitoring file stanza in the inputs.conf

0 Karma

laserval
Communicator

Is that result one or two events?

Edit: Wait, what I meant is that Splunk, when parsing JSON, doesn't expect valid JSON but instead a single JSON object per line. That is at least what I've experienced.

0 Karma

lpolo
Motivator

The event I presented in the original question is a valid Json.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

In what way is your event not broken correctly?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Mastering Threat Intelligence in ES 8.5, Splunk AI Assistant v2, and More from Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

Break the Build: Inside the KubeDoom Lounge at .conf26

    You step up to the machine. The pixelated corridors of a certain 1993 FPS load in front of you, EMP Pulse ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...