Getting Data In

Automatic extraction of JSON fields doesn't work and have to use spath

Motoko89
Path Finder

I have a distributed Splunk deployment and need to index JSON data, 1 object per row. Objects are serialized using NewtonSoft.JSON .NET library. I already configured both INDEXED_EXTRACTIONS = json and KV_MODE = json for my custom source type in props.conf for deployment app of forwarders, indexers, and heads.

Yet, when I search, I still need to specify spath else no result is returned. Our event text length does not exceed 5000 chars (~ 2500 chars) so I don't think the problem here https://answers.splunk.com/answers/177410/why-are-only-some-json-fields-extracted-as-data-in.html affect us. What do I miss here? Any suggestion?

My props.conf

[myblob]
DATETIME_CONFIG = 
INDEXED_EXTRACTIONS = json
KV_MODE = json
AUTO_KV_JSON = true
NO_BINARY_CHECK = true
category = Structured
description = my blob description
disabled = false
TIME_PREFIX=\"EventInfo\.Time\":\s*
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%7NZ
0 Karma

ssadanala1
Contributor

You cant specify both INDEXED_EXTRACTIONS and KV_MODE in the props .
You need to specify either of one .

Try this

[myblob]
DATETIME_CONFIG =
KV_MODE = json
AUTO_KV_JSON = true
NO_BINARY_CHECK = true
category = Structured
description = my blob description
disabled = false
TIME_PREFIX=\"EventInfo.Time\":\s*
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%7NZ

0 Karma

Motoko89
Path Finder

So there are 3 places the props.conf go to: the forwarder, the indexer, and the head. Could you advice more specifically which one I should change into what? Do I keep AUTO_KV_JSON at all 3 places?

0 Karma

MuS
Legend

AUTO_KV_JSON is not needed anywhere, because it defaults to true anyway 😉

AUTO_KV_JSON = [true|false]
* Used for search-time field extractions only.
* Specifies whether to try json extraction automatically.
* Defaults to true.

If you use KV_MODE=json it will only go on the search head. And adding to this, your JSON format must be true and valid JSON otherwise Splunk will not do anything ...

* The 'xml' and 'json' modes will not extract any fields when used on data
  that isn't of the correct format (JSON or XML).

Hope this helps ...

cheers, MuS

xpac
SplunkTrust
SplunkTrust

...and you most likely want to use KV_MODE and not indexed extractions. Remember that in case 9d KV_MODE, this has to go on the search head, not on the indexer. In case of indexed extractions, it has to go on the indexer, and maybe on the search head (not sure about that).

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...