We ended up doing below which works the way we want i.e. no duplicate json values. On UF, do NOT define any props. On Indexers, nothing specific to JSON props, but we had defined props related time field [my_sourcetype] NO_BINARY_CHECK=true CHARSET=UTF-8 MAX_TIMESTAMP_LOOKAHEAD=14000 TIME_PREFIX=timestamp":"? On SH, do NOT define any props With this set up, The JSON values are by default extracted in Indexing layer. Because on Indexers, this property is set up in system/default location. [default] AUTO_KV_JSON = true ... View more

I checked that on SH. ... View more

Thanks for your response. I checked above steps and props are coming/used from where I defined. They are same as what you mentioned in step-4. Still same issue. $/opt/splunk/bin/splunk btool props list --debug | grep "my_sourcetype" /data/splunk/etc/apps/my_app/local/props.conf [my_sourcetype] ... View more

Anyone has anymore clues as how to debug this?. I have also run the query on CM, there also I see the duplicate JSON values. ... View more

It shows this [root@lgpbdus4101 bin]# ./splunk btool props list --debug my_sourcetype /data/splunk/etc/apps/my_app/local/props.conf [my_sourcetype] /data/splunk/etc/system/default/props.conf ADD_EXTRA_TIME_FIELDS = True /data/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True /data/splunk/etc/apps/my_app/local/props.conf AUTO_KV_JSON = false /data/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE = /data/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True /data/splunk/etc/system/default/props.conf CHARSET = UTF-8 /data/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml /data/splunk/etc/system/default/props.conf DEPTH_LIMIT = 1000 /data/splunk/etc/system/default/props.conf HEADER_MODE = /data/splunk/etc/apps/my_app/local/props.conf KV_MODE = none /data/splunk/etc/system/default/props.conf LEARN_MODEL = true /data/splunk/etc/system/default/props.conf LEARN_SOURCETYPE = true /data/splunk/etc/system/default/props.conf LINE_BREAKER_LOOKBEHIND = 100 /data/splunk/etc/system/default/props.conf MATCH_LIMIT = 100000 /data/splunk/etc/system/default/props.conf MAX_DAYS_AGO = 2000 /data/splunk/etc/system/default/props.conf MAX_DAYS_HENCE = 2 /data/splunk/etc/system/default/props.conf MAX_DIFF_SECS_AGO = 3600 /data/splunk/etc/system/default/props.conf MAX_DIFF_SECS_HENCE = 604800 /data/splunk/etc/system/default/props.conf MAX_EVENTS = 256 /data/splunk/etc/system/default/props.conf MAX_TIMESTAMP_LOOKAHEAD = 128 /data/splunk/etc/system/default/props.conf MUST_BREAK_AFTER = /data/splunk/etc/system/default/props.conf MUST_NOT_BREAK_AFTER = /data/splunk/etc/system/default/props.conf MUST_NOT_BREAK_BEFORE = /data/splunk/etc/system/default/props.conf SEGMENTATION = indexing /data/splunk/etc/system/default/props.conf SEGMENTATION-all = full /data/splunk/etc/system/default/props.conf SEGMENTATION-inner = inner /data/splunk/etc/system/default/props.conf SEGMENTATION-outer = outer /data/splunk/etc/system/default/props.conf SEGMENTATION-raw = none /data/splunk/etc/system/default/props.conf SEGMENTATION-standard = standard /data/splunk/etc/system/default/props.conf SHOULD_LINEMERGE = True /data/splunk/etc/system/default/props.conf TRANSFORMS = /data/splunk/etc/system/default/props.conf TRUNCATE = 10000 /data/splunk/etc/system/default/props.conf detect_trailing_nulls = false /data/splunk/etc/system/default/props.conf maxDist = 100 /data/splunk/etc/system/default/props.conf priority = /data/splunk/etc/system/default/props.conf sourcetype = ... View more

Correct. I tried below as well on SH. [my_sourcetype] KV_MODE=none AUTO_KV_JSON = false ... View more

@witski But we are already on forwarder version 7.0.3. The issue still seem to exist. Are we missing anything? ... View more

Yes, I agree. For this exact reason, we had put to monitor all files with additional setting of ignoring older files (>2h). We had seen issue of missing last few events active file, before getting rolled in the past. ... View more

We are on forwarder 7.0.3. We seem to have similar issue. What should be the ideal fix from Splunk monitoring side? Below is the monitoring stanza [monitor:///opt/mapr/hadoop/hadoop/logs/nodemanager] sourcetype = my_st index = my_index disabled = 0 ignoreOlderThan = 2h The latest file will be with yarn-mapr-nodemanager-host_name.log and the latest archived file be with yarn-mapr-nodemanager-host_name.log.1 . What is interesting is intermittently on certain servers, the current file gets indexed only at the time of its roll/archival i.e. lets say after 6 hours. And the issue of live/current file not getting indexed on time does not happen all the time. The next live file might get indexed on time. To me, it sounds like what @sjohnson_splunk has mentioned. But there should be ideal settings to avoid this. Any insights on this will be helpful. ... View more