Getting Data In

Automatic extraction of JSON fields doesn't work and have to use spath

Motoko89
Path Finder

I have a distributed Splunk deployment and need to index JSON data, 1 object per row. Objects are serialized using NewtonSoft.JSON .NET library. I already configured both INDEXED_EXTRACTIONS = json and KV_MODE = json for my custom source type in props.conf for deployment app of forwarders, indexers, and heads.

Yet, when I search, I still need to specify spath else no result is returned. Our event text length does not exceed 5000 chars (~ 2500 chars) so I don't think the problem here https://answers.splunk.com/answers/177410/why-are-only-some-json-fields-extracted-as-data-in.html affect us. What do I miss here? Any suggestion?

My props.conf

[myblob]
DATETIME_CONFIG = 
INDEXED_EXTRACTIONS = json
KV_MODE = json
AUTO_KV_JSON = true
NO_BINARY_CHECK = true
category = Structured
description = my blob description
disabled = false
TIME_PREFIX=\"EventInfo\.Time\":\s*
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%7NZ
0 Karma

ssadanala1
Contributor

You cant specify both INDEXED_EXTRACTIONS and KV_MODE in the props .
You need to specify either of one .

Try this

[myblob]
DATETIME_CONFIG =
KV_MODE = json
AUTO_KV_JSON = true
NO_BINARY_CHECK = true
category = Structured
description = my blob description
disabled = false
TIME_PREFIX=\"EventInfo.Time\":\s*
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%7NZ

0 Karma

Motoko89
Path Finder

So there are 3 places the props.conf go to: the forwarder, the indexer, and the head. Could you advice more specifically which one I should change into what? Do I keep AUTO_KV_JSON at all 3 places?

0 Karma

MuS
SplunkTrust
SplunkTrust

AUTO_KV_JSON is not needed anywhere, because it defaults to true anyway 😉

AUTO_KV_JSON = [true|false]
* Used for search-time field extractions only.
* Specifies whether to try json extraction automatically.
* Defaults to true.

If you use KV_MODE=json it will only go on the search head. And adding to this, your JSON format must be true and valid JSON otherwise Splunk will not do anything ...

* The 'xml' and 'json' modes will not extract any fields when used on data
  that isn't of the correct format (JSON or XML).

Hope this helps ...

cheers, MuS

xpac
SplunkTrust
SplunkTrust

...and you most likely want to use KV_MODE and not indexed extractions. Remember that in case 9d KV_MODE, this has to go on the search head, not on the indexer. In case of indexed extractions, it has to go on the indexer, and maybe on the search head (not sure about that).

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...