topic Data from a particular source is extracted in duplicate when searched from "Searching and reporting"? in Getting Data In

Data from a particular source is extracted in duplicate when searched from "Searching and reporting"?

joemiller — Wed, 18 Jan 2023 18:56:31 GMT

This is a single server Splunk deployment. I am indexing Duo MFA logs using the official splunk app. In the "Searching and reporting" app, when I use the table command to view that data, each field is a multivalue field with the value duplicated. When I try the same search using the Duo app instead of "Searching and reporting", the fields are extracted only once as expected, not duplicated. For example...

When I use the table command on this data in "Searching and reporting":

email user@example.com user@example.com

When I use the table command on this data in the "Duo" app:

email user@example.com

So this problem appears to be limited to the "Searching and reporting" app. But I'm not finding any configuration specific to "Searching and reporting" related to this app/source. For example, there is nothing in SPLUNK/etc/apps/search/local/ props.conf or transforms.conf that would affect this source.

The current configuration according to the btool is coming from SPLUNK/etc/apps/duo_splunkapp/default/props.conf and is:

[source::duo] INDEXED_EXTRACTIONS = json KV_MODE = none

I also tried changing the config to this:

INDEXED_EXTRACTIONS = none AUTO_KV_JSON = true KV_MODE = json

but that just resulted in neither the "Searching and reporting" app nor the "Duo" app having extractions for this data. How do I fix this so the "Searching and reporting" app has a single set of extractions and not duplicates?

Re: Data from a particular source is extracted in duplicate when searched from "Searching and reporting"

joemiller — Wed, 18 Jan 2023 03:37:06 GMT

Update: despite the btool not showing anything relevant in SPLUNK/etc/apps/search/local/props.conf, I added the following lines to that file:

[source::duo] AUTO_KV_JSON = false KV_MODE = none

and this seems to have fixed my problem. So now my question is why? Why weren't the lines in SPLUNK/etc/apps/duo_splunkapp/default/props.conf and (duo_splunkapp/local/props.conf) taking effect even though there were no contradicting lines in a conf file with greater precedence?

Re: Data from a particular source is extracted in duplicate when searched from "Searching and reporting"

yeahnah — Wed, 18 Jan 2023 04:12:03 GMT

Hi @joemiller

The issue is using INDEXED_EXTRACTION = json (forwarder/heavy forwarder config) and having KV_MODE = json (search head configuration)

https://docs.splunk.com/Documentation/Splunk/9.0.3/Admin/Propsconf#Structured_Data_Header_Extraction_and_configuration

The INDEXED_EXTRACTION setting means the fields are indexed at ingestion time. If this is done then KV_MODE = none (default is auto) should be set on the search head. Also ensure this configuration is globally shared for the sourcetype/source.

In this case the KV_MODE = none had only been shared in the duo_splunkapp app space, i.e. the search and reporting app does not see this config. To share its config globally: Apps > Manage apps > duo_splunkapp > Permissions > All apps (system)

Personally, I would avoid using INDEXED_EXTRACTION and then just let the Splunk search head KV_MODE = auto do all the work at search time - this is considered best practise. Sadly, a lot of third Splunk party apps come with an INDEXED_EXTRACTION enabled and deployed in the default directory. The last time I tired, the only way to disable an INDEXED_EXTRACTION was to remove the line from props.conf in the default folder, i.e. it cannot be disabled in the local directory props.conf with an INDEXED_EXTRACTION = none entry (admittedly, I've not tried this for a while). Not being able to disable it in a local folder means any upgrade of this app in the future will enable it again - unless manually removed again.

Anyway, bit off topic, but you were on the right track with the settings you were playing with.

Hope this helps

Re: Data from a particular source is extracted in duplicate when searched from "Searching and reporting"

joemiller — Wed, 18 Jan 2023 19:28:49 GMT

Thank you! Argh, so the permissions setting on the app was the reason why my changes weren't taking effect. Thanks for clearing that up for me. And that's confusing about the INDEXED_EXTRACTION not being able to be modified from the local props.conf. I understand why search time extraction mightbe preferable, but I don't want to create a setup that will break when there's an app update either.