Getting Data In

Ingesting JSON data via a python script, why are fields with numeric values indexed as multivalue fields with two identical values?

splunknewby
Path Finder

I have a json file with entries in the following form:
{ "ABC" : "XYZ" , "DEF" : 123 , "GHI" : "456" , ... }
There are about 15 or so variables defined in a single json formatted line with multiple lines for a given output.

Splunk picks up the output via a python script which essentially prints everything to stdout.

The issue I'm having is that, when Splunk ingests the data, some of the fields end up being multivalued where a field has two identical values. I can see this occurring when I click on the "show as raw text" in the splunk search results.

Somewhat interesting is that these fields are all fields with numerical values in them. So it's occurring for both "DEF": 123 and "GHI" : "456" types..

Any ideas as to what could be causing this issue?

woodcock
Esteemed Legend

Your problem is probably the same as this:

http://answers.splunk.com/answers/301165/splunk-app-for-aws-billing-why-is-a-single-entry-o.html#ans...

You are probably telling Splunk to extract JSON fields twice: once at index time ( INDEXED_EXTRACTIONS=json ) and once at search time ( KV_MODE=json ). Get rid of the KV_MODE setting.

See this Q&A for a more complete discussion:

http://answers.splunk.com/answers/174939/why-are-my-json-fields-extracted-twice.html

0 Karma

landen99
Motivator

That is a good possibility. Would we see a similar mechanic if sourcetype=json (auto-sourcetyping) or a transforms call from props on an indexer? What are your thoughts on index time extractions vs search time?

0 Karma

woodcock
Esteemed Legend

Yes. For JSON, the events are fairly useless without extracting them so you are way better off doing it once for everybody at Index time rather than for every search (unless you have HUGE numbers of events that are rarely searched).

0 Karma

landen99
Motivator

Are there multiple entries of "ABC" : "123" , for example? If so, that would explain it.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...