JSON fields duplicating in new Splunk Cloud instance.

[mpeterson]

I am trying to ingest some json data into a new Splunk Cloud instance, with a custom sourcetype, but I keep getting duplicate data in the search results. This seems to be an extremely common problem, based on the number of old posts, but none of them seem to address the Cloud version.

I have a JSON file that looks like this:

{

"RowNumber": 1,

"ApplicationName": "177525278",

"ClientProcessID": 114889,

"DatabaseName": "1539703986",

"StartTime": "2024-07-30 12:15:13"

}

• I have a Windows 2022 server with a 9.2.2 universal forwarder installed.
• I manually added a very simple app to the C:\Program Files\SplunkUniversalForwarder\etc\apps folder.
  
  
  • inputs.conf contains this monitor
    [batch://C:\splunk_test_files\ErrorMaster\*.json]
    move_policy=sinkhole
    index=centraladmin_errormaster
    sourcetype=errormaster
    
    
  • props.conf contains this type  (copied from _json)
    [errormaster]
    pulldown_type = true
    INDEXED_EXTRACTIONS = json
    KV_MODE = none
    category = Structured
    description = JavaScript Object Notation format. For more information, visit http://json.org/
    
    
• On the cloud side I created (from the UI) a new sourcetype called 'errormaster' as just a direct clone of the existing _json type.

When I add a .json file to the folder, it is ingested and the events show up in the cloud instance, under the right correct centraladmin_errormaster index, and with the sourcetype=errormaster.

However, the fields all have duplicate values.

If it switch it to the built-in _json type it works fine. I have some field extractions I want to add, which is why I wanted a custom type.

I'm guessing this is something obvious to the Cloud experts, but I am an accidental Splunk Admin with very little experience, so any help you can offer would be appreciated.

[PickleRick]

This behaviour is typically a result of having both INDEXED_EXTRACTIONS as well as search-time extractions active. Can't say what your effective config is so don't know for sure why it's happening.

[mpeterson · Accepted solution]

Our partner had me delete the props.conf file on the Universal Forwarder and that seems to have solved the issue. I still don't understand why  the _json type works correctly while a copy of the _json type does not, but for now this issue is resolved.