Getting Data In

Do I have a possible KV extraction issue on the universal forwarder?

ehowardl3
Path Finder

I have some json events that are fairly long (10K-20K characters). Most events come through fine, except for the fact that some events have an issue with some of the fields towards the end of the event not being automatically extracted. I ran a couple of searches and found that the maximum raw length of the events with no issues is 10,197, while the minimum raw length of the events with the extraction issues is 10,251.

This led me to believe that I needed to bump up maxchars in the kv stanza in limits.conf, since it has a default limit of 10,240. However, upon making the change, the extraction issues remain. Here are my configurations:

On the forwarder:

props.conf:

[test:json]
KV_MODE = json
TIMESTAMP_FIELDS = Date
TRUNCATE = 0
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false

On the search head:

limits.conf:

[kv]
limit = 200
maxchars = 25000
maxcols = 1024

Any ideas?

Thanks!
E

0 Karma
1 Solution

harsmarvania57
Ultra Champion

HI @ehowardl3 ,

I am assuming that forwarder means Universal forwarder, in that case can you please apply below configuration on Universal Forwarder ?

props.conf

[test:json]
INDEXED_EXTRACTIONS = json

View solution in original post

harsmarvania57
Ultra Champion

HI @ehowardl3 ,

I am assuming that forwarder means Universal forwarder, in that case can you please apply below configuration on Universal Forwarder ?

props.conf

[test:json]
INDEXED_EXTRACTIONS = json

ehowardl3
Path Finder

Thank you! This worked. I had tried that previously, however I had:

[test:json]
INDEXED_EXTRACTIONS = JSON

I'm assuming the uppercase JSON was causing an issue.

0 Karma

harsmarvania57
Ultra Champion

Great, it worked. I have converted my comment to answer so that you can accept/upvote it.

Thanks,
Harshil

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...