Splunk Search

Field extraction: Is there a limit on the number of values a JSON multivalued field can hold in Splunk 6.2.1?

somesoni2
Revered Legend

Hi All,

I am ingesting a json log file. The data contains a JSON array with multiple fields. Sample format

{
  "payload": {
    "rootfield1": "1234567890",
    "rootfield2": "SDFDFDF"
  },
  "event": [
    {
      "eventfield1": "1234567890",
      "eventprimarykey": "2377",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    },
     {
      "eventfield1": "1234567890",
      "eventprimarykey": "2378",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    },
....
...
 {
      "eventfield1": "1234567890",
      "eventprimarykey": "2377",
      "timestamp": "2015-02-18T10:48:14-0500",
      "data": "sdfdfdfdf"      
    }
  ]
}

The number of elements in "event{}" array could go up to 50.

The problem that I am facing is that when I check the no of values in the multivalued fields created for the array (event{}.eventfield1, event{}.eventprimarykey etc), the total count is never 50, even though the raw data has 50 unique elements. The count varies from 33 to 39 but never matches actual count in the raw data.

Is there any limit of no of values that a JSON multivalued field can hold?

Thanks in advanced.

1 Solution

woodcock
Esteemed Legend

According to this answer, there is a limit if you use INDEXED EXTRACTIONS = JSON but not if you use KV_MODE = json:

https://answers.splunk.com/answers/319059/indexed-extractions-json-limiting-multivalued-fiel.html

View solution in original post

woodcock
Esteemed Legend

According to this answer, there is a limit if you use INDEXED EXTRACTIONS = JSON but not if you use KV_MODE = json:

https://answers.splunk.com/answers/319059/indexed-extractions-json-limiting-multivalued-fiel.html

jonathon
Path Finder

Yes, I'm not using indexed field extractions, this was just for the UI/search extracting at search time.

What worked for us was making sure that we had the following in the props.conf definition:

KV_MODE = JSON

and also

maxcols = 2000

in limits.conf on our search heads.

This combination resolved the issue for us.

badarsebard
Communicator

Also a good one to keep in mind along with maxcols is maxchars:

maxchars = integer
* Truncate _raw to this size and then do auto KV.
* Default: 10240 characters

dstaulcu
Builder

Just came across this answers article for a similar problem today. Thank you for the maxcols bit!

0 Karma

suarezry
Builder

I seem to be having the same problem. Although in my case, the max number of values for a multivalued field is 10. Did you sort this out?

link to problem

0 Karma

sk314
Builder

you have questions too? 😄

Get Updates on the Splunk Community!

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...