Getting Data In

Do I have a possible KV extraction issue on the universal forwarder?

ehowardl3
Path Finder

I have some json events that are fairly long (10K-20K characters). Most events come through fine, except for the fact that some events have an issue with some of the fields towards the end of the event not being automatically extracted. I ran a couple of searches and found that the maximum raw length of the events with no issues is 10,197, while the minimum raw length of the events with the extraction issues is 10,251.

This led me to believe that I needed to bump up maxchars in the kv stanza in limits.conf, since it has a default limit of 10,240. However, upon making the change, the extraction issues remain. Here are my configurations:

On the forwarder:

props.conf:

[test:json]
KV_MODE = json
TIMESTAMP_FIELDS = Date
TRUNCATE = 0
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false

On the search head:

limits.conf:

[kv]
limit = 200
maxchars = 25000
maxcols = 1024

Any ideas?

Thanks!
E

0 Karma
1 Solution

harsmarvania57
Ultra Champion

HI @ehowardl3 ,

I am assuming that forwarder means Universal forwarder, in that case can you please apply below configuration on Universal Forwarder ?

props.conf

[test:json]
INDEXED_EXTRACTIONS = json

View solution in original post

harsmarvania57
Ultra Champion

HI @ehowardl3 ,

I am assuming that forwarder means Universal forwarder, in that case can you please apply below configuration on Universal Forwarder ?

props.conf

[test:json]
INDEXED_EXTRACTIONS = json

ehowardl3
Path Finder

Thank you! This worked. I had tried that previously, however I had:

[test:json]
INDEXED_EXTRACTIONS = JSON

I'm assuming the uppercase JSON was causing an issue.

0 Karma

harsmarvania57
Ultra Champion

Great, it worked. I have converted my comment to answer so that you can accept/upvote it.

Thanks,
Harshil

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...