Splunk Search
Highlighted

Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

SplunkTrust
SplunkTrust

Hi All,

I am ingesting a json log file. The data contains a JSON array with multiple fields. Sample format

{
  "payload": {
    "rootfield1": "1234567890",
    "rootfield2": "SDFDFDF"
  },
  "event": [
    {
      "eventfield1": "1234567890",
      "eventprimarykey": "2377",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    },
     {
      "eventfield1": "1234567890",
      "eventprimarykey": "2378",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    },
....
...
 {
      "eventfield1": "1234567890",
      "eventprimarykey": "2377",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    }
  ]
}

The number of elements in "event{}" array could go up to 50.

The problem that I am facing is that when I check the no of values in the multivalued fields created for the array (event{}.eventfield1, event{}.eventprimarykey etc), the total count is never 50, even though the raw data has 50 unique elements. The count varies from 33 to 39 but never matches actual count in the raw data.

Is there any limit of no of values that a JSON multivalued field can hold?

Thanks in advanced.

Highlighted

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

Builder

you have questions too? 😄

Highlighted

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

Builder

I seem to be having the same problem. Although in my case, the max number of values for a multivalued field is 10. Did you sort this out?

link to problem

0 Karma
Highlighted

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

Esteemed Legend

According to this answer, there is a limit if you use INDEXED EXTRACTIONS = JSON but not if you use KV_MODE = json:

https://answers.splunk.com/answers/319059/indexed-extractions-json-limiting-multivalued-fiel.html

View solution in original post

Highlighted

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

Path Finder

Yes, I'm not using indexed field extractions, this was just for the UI/search extracting at search time.

What worked for us was making sure that we had the following in the props.conf definition:

KV_MODE = JSON

and also

maxcols = 2000

in limits.conf on our search heads.

This combination resolved the issue for us.

Highlighted

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

Builder

Just came across this answers article for a similar problem today. Thank you for the maxcols bit!

0 Karma
Highlighted

Re: Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

Communicator

Also a good one to keep in mind along with maxcols is maxchars:

maxchars = integer
* Truncate _raw to this size and then do auto KV.
* Default: 10240 characters