Events with duplicate field extractions

[cfernaca]

Good afternoon,

I have a monitoring architecture with three nodes with the Splunk Enterprise product. One node acts as SearchHead, one as Indexer and one for all other roles. I have a HEC on the indexer node to be able to receive data from third parties. The sourcetype configured to store the data is as follows:
[integration]
DATETIME_CONFIG = CURRENT
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Structured
description = test
disabled = false
pulldown_type = 1
INDEXED_EXTRACTIONS = none
KV_MODE = json

My problem is that when I fetch the data, there are events where the field extraction is done in duplicate and others where the field extraction is done only once.

Please, can you help me?

Best regards, thank you very much

[livehybrid]

Hi @cfernaca 

The duplicate field extractions are likely due to multiple or conflicting search-time field extraction configurations applying to the integration sourcetype. Since INDEXED_EXTRACTIONS = none is set, the issue occurs at search time. 

KV_MODE = json is generally sufficient for JSON data, but other configurations (e.g., REPORT-* or EXTRACT-* in props.conf) might be redundantly extracting the same fields.

Check for conflicting configurations usingbtool, Run this command on your Search Head's CLI to see all applied settings for your sourcetype and the source props.conf files:

splunk btool props list integration --debug

Look for REPORT-* or EXTRACT-* configurations that might be extracting fields already handled by KV_MODE = json.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

HEC sources, if writing to /event endpoint can provide own set of indexed fields beside the raw event. Also - with /event endpoint no line breaking takes place.

[cfernaca]

So, what is the solution you propose?

[PickleRick]

I can't propose any solution because I have no idea where the problem is. I don't even know which endpoint you're using. The remark about line breaking is just something worth knowing.