Missing events? JSON payload and indexed_extractions

swangertyler — Mon, 23 Sep 2019 19:04:59 GMT

We have events where the JSON payload has 100s of fields. When I table a field, we can see entries for some events but not others. However, if I spath the field beforehand, I then can discover it. We are using indexed_extractions? Is there a limit with this?

Re: Missing events? JSON payload and indexed_extractions

kamlesh_vaghela — Tue, 24 Sep 2019 06:03:41 GMT

@swangertyler

As your single JSON event has 100+ fields, so extracting first 100 fields is the default behaviour of the Splunk. You can set this number by updating limits.conf.

 limit = <integer>
    * The maximum number of fields that an automatic key-value field extraction
      (auto kv) can generate at search time.
    * If search-time field extractions are disabled (KV_MODE=none in props.conf)
      then this setting determines the number of index-time fields that will be
      returned.
    * The summary fields 'host', 'index', 'source', 'sourcetype', 'eventtype',
      'linecount', 'splunk_server', and 'splunk_server_group' do not count against
      this limit and will always be returned.
    * Increase this setting if, for example, you have indexed data with a large
      number of columns and want to ensure that searches display all fields from
      the data.
    * Default: 100

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf#.5Bkv.5D

If your JSON length is 5000+ character than you can also try by configuring extraction_cutoff of spath stanza.

extraction_cutoff = <integer>
* For extract-all spath extraction mode, only apply extraction to the first
  <integer> number of bytes.
* Default: 5000

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf#.5Bspath.5D

topic Missing events? JSON payload and indexed_extractions in Getting Data In

Missing events? JSON payload and indexed_extractions

Re: Missing events? JSON payload and indexed_extractions