@vinayakwagh Please if below post helps you. We had faced similar issue and is resolved now https://answers.splunk.com/answers/768573/why-are-json-fields-extracted-and-displayed-twice.html ... View more

We ended up doing below which works the way we want i.e. no duplicate json values. On UF, do NOT define any props. On Indexers, nothing specific to JSON props, but we had defined props related time field [my_sourcetype] NO_BINARY_CHECK=true CHARSET=UTF-8 MAX_TIMESTAMP_LOOKAHEAD=14000 TIME_PREFIX=timestamp":"? On SH, do NOT define any props With this set up, The JSON values are by default extracted in Indexing layer. Because on Indexers, this property is set up in system/default location. [default] AUTO_KV_JSON = true ... View more

I checked that on SH. ... View more

Thanks for your response. I checked above steps and props are coming/used from where I defined. They are same as what you mentioned in step-4. Still same issue. $/opt/splunk/bin/splunk btool props list --debug | grep "my_sourcetype" /data/splunk/etc/apps/my_app/local/props.conf [my_sourcetype] ... View more

Anyone has anymore clues as how to debug this?. I have also run the query on CM, there also I see the duplicate JSON values. ... View more

It shows this [root@lgpbdus4101 bin]# ./splunk btool props list --debug my_sourcetype /data/splunk/etc/apps/my_app/local/props.conf [my_sourcetype] /data/splunk/etc/system/default/props.conf ADD_EXTRA_TIME_FIELDS = True /data/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True /data/splunk/etc/apps/my_app/local/props.conf AUTO_KV_JSON = false /data/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE = /data/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True /data/splunk/etc/system/default/props.conf CHARSET = UTF-8 /data/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml /data/splunk/etc/system/default/props.conf DEPTH_LIMIT = 1000 /data/splunk/etc/system/default/props.conf HEADER_MODE = /data/splunk/etc/apps/my_app/local/props.conf KV_MODE = none /data/splunk/etc/system/default/props.conf LEARN_MODEL = true /data/splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true /data/splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100 /data/splunk/etc/system/default/props.conf MATCH_LIMIT = 100000 /data/splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000 /data/splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2 /data/splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600 /data/splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800 /data/splunk/etc/system/default/props.conf MAX_EVENTS = 256 /data/splunk/etc/system/default/props.conf MAX_TIMESTAMP_LOOKAHEAD = 128 /data/splunk/etc/system/default/props.conf MUST_BREAK_AFTER = /data/splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER = /data/splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE = /data/splunk/etc/system/default/props.conf SEGMENTATION = indexing /data/splunk/etc/system/default/props.conf SEGMENTATION-all = full /data/splunk/etc/system/default/props.conf SEGMENTATION-inner = inner /data/splunk/etc/system/default/props.conf SEGMENTATION-outer = outer /data/splunk/etc/system/default/props.conf SEGMENTATION-raw = none /data/splunk/etc/system/default/props.conf SEGMENTATION-standard = standard /data/splunk/etc/system/default/props.conf SHOULD_LINEMERGE = True /data/splunk/etc/system/default/props.conf TRANSFORMS = /data/splunk/etc/system/default/props.conf TRUNCATE = 10000 /data/splunk/etc/system/default/props.conf detect_trailing_nulls = false /data/splunk/etc/system/default/props.conf maxDist = 100 /data/splunk/etc/system/default/props.conf priority = /data/splunk/etc/system/default/props.conf sourcetype = ... View more

Correct. I tried below as well on SH. [my_sourcetype] KV_MODE=none AUTO_KV_JSON = false ... View more

Restarted, No luck. ... View more

We have Similar issue (json fields are extracted twice) On Universal forwarder (7.0.3) the settings are like this [my_sourcetype] SHOULD_LINEMERGE=true LINE_BREAKER=([\r\n]+) NO_BINARY_CHECK=true CHARSET=UTF-8 INDEXED_EXTRACTIONS=json KV_MODE=none category=Structured description=JavaScript Object Notation format. For more information, visit http://json.org/ disabled=false pulldown_type=true TIMESTAMP_FIELDS=timestamp On Search Head(7.2.6), tried all combinations of below [my_sourcetype] INDEXED_EXTRACTIONS=json KV_MODE=none AUTO_KV_JSON = false Does anyone have a working solution? Also when we apply props on SH member, do we have to restart Splunk on it? We just did _debug/refresh. ... View more

Have the wildcards for the file name. But I think Splunk adds the WATCH on that path, which means it might look for sub-directories by default? Anyways, we will add the setting recursive = false and monitor for a few days. ... View more

Yeah, sure. Just 22 files in the monitored directory with the name nodemanager. There is a sub-directory and it contains around 57 sub-directories, but the name does not contain nodemanager. So probably around 57*5*3=855 unwanted files. Should we try adding recursive = false , just to avoid scanning sub-directory? ... View more

@witski But we are already on forwarder version 7.0.3. The issue still seem to exist. Are we missing anything? ... View more

Yes, I agree. For this exact reason, we had put to monitor all files with additional setting of ignoring older files (>2h). We had seen issue of missing last few events active file, before getting rolled in the past. ... View more