topic Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1? in Splunk Search

Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

somesoni2 — Wed, 18 Feb 2015 23:13:33 GMT

Hi All,

I am ingesting a json log file. The data contains a JSON array with multiple fields. Sample format

{
  "payload": {
    "rootfield1": "1234567890",
    "rootfield2": "SDFDFDF"
  },
  "event": [
    {
      "eventfield1": "1234567890",
      "eventprimarykey": "2377",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    },
     {
      "eventfield1": "1234567890",
      "eventprimarykey": "2378",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    },
....
...
 {
      "eventfield1": "1234567890",
      "eventprimarykey": "2377",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    }
  ]
}

The number of elements in "event{}" array could go up to 50.

The problem that I am facing is that when I check the no of values in the multivalued fields created for the array (event{}.eventfield1, event{}.eventprimarykey etc), the total count is never 50, even though the raw data has 50 unique elements. The count varies from 33 to 39 but never matches actual count in the raw data.

Is there any limit of no of values that a JSON multivalued field can hold?

Thanks in advanced.

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

sk314 — Wed, 18 Feb 2015 23:58:27 GMT

you have questions too? 😄

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

suarezry — Wed, 14 Oct 2015 17:57:27 GMT

I seem to be having the same problem. Although in my case, the max number of values for a multivalued field is 10. Did you sort this out?

link to problem

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

woodcock — Wed, 14 Oct 2015 19:21:50 GMT

According to this answer, there is a limit if you use INDEXED EXTRACTIONS = JSON but not if you use KV_MODE = json:

https://answers.splunk.com/answers/319059/indexed-extractions-json-limiting-multivalued-fiel.html

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

jonathon — Wed, 14 Oct 2015 19:44:43 GMT

Yes, I'm not using indexed field extractions, this was just for the UI/search extracting at search time.

What worked for us was making sure that we had the following in the props.conf definition:

KV_MODE = JSON

and also

maxcols = 2000

in limits.conf on our search heads.

This combination resolved the issue for us.

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

dstaulcu — Tue, 28 Aug 2018 02:38:40 GMT

Just came across this answers article for a similar problem today. Thank you for the maxcols bit!

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

badarsebard — Mon, 22 Jul 2019 22:05:47 GMT

Also a good one to keep in mind along with maxcols is maxchars:

maxchars = integer
* Truncate _raw to this size and then do auto KV.
* Default: 10240 characters