All Apps and Add-ons

Duplicative JSON events in all but one App

peterm30
Path Finder

We are having an issue with duplicative JSON fields coming from Duo (the 2FA provider) events. Our senario seems to be a bit different than what others are encountering though. When searching within the Duo app (https://splunkbase.splunk.com/app/3504/), there is no duplication. However, when searching within any other app (default Searching and Reporting, other custom apps, etc.), we are seeing the duplication of fields.

We are using Splunk Cloud, so we do not have backend access to our search head or indexers, but we can change anything that can be configured in the web interface. We do have backend access to the heavy forwarders.

The Duo app is installed on one such heavy forwarder that recieves the events via API. Since the app seems to be working, our initial thought was to apply the app's default props.conf to all other apps by copying it to /opt/splunk/etc/system/local. Playing with the settings here, we can see that it does have an effect, but the default settings didn't solve the issue.

We've also tried setting several other settings as detailed here: https://answers.splunk.com/answers/765005/how-do-i-stop-getting-duplicate-entries-of-json-da.html
Specifically, unsetting INDEXED_EXTRATIONS and KV_MODE, as well as adding AUTO_KV_JSON = false. These particular settings didn't work, nor has any other combination that we've tried.

At this point, I'm soliciting any other suggested settings, along with where to place them.

0 Karma

jeremyhagand61
Communicator

Did you resolve this? I have the exact same problem.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
You should do those unsetings on first full splunk enterprise instance from where the events are coming into splunk.
0 Karma

jeremyhagand61
Communicator

Heavy forwarder?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Yes, if you have heavy forwarder as intermediate/gateway before splunk indexer(s).

0 Karma
Get Updates on the Splunk Community!

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...

Splunk App Dev Community Updates – What’s New and What’s Next

Welcome to your go-to roundup of everything happening in the Splunk App Dev Community! Whether you're building ...

The Latest Cisco Integrations With Splunk Platform!

Join us for an exciting tech talk where we’ll explore the latest integrations in Cisco + Splunk! We’ve ...