Getting Data In

What are the requirements for a perfect Splunk JSON document?

Ultra Champion

Our Sales Engineer told us that the Splunk json parser requires several specific things in the json document, in order to be interpreted as json. What are they?

We would like to avoid hard-coded solutions such as How do we assign each JSON document to a distinct event?

Tags (2)
1 Solution

SplunkTrust
SplunkTrust

Why not just apply base configs to your JSON file and have it break correctly rather than trying to format the log to Splunk?

If you let Splunk try to figure out the linebreaking, it will add additional overhead to your indexing and slow it down.

Adding this will give you correct linebreaking and timestamping along with avoiding the merging pipeline which increases your indexing speed

[sourcetype]
TIME_PREFIX = 
TIME_FORMAT = 
LINE_BREAKER = 
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 
TRUNCATE = 

https://wiki.splunk.com/Community:HowIndexingWorks

View solution in original post

SplunkTrust
SplunkTrust

Why not just apply base configs to your JSON file and have it break correctly rather than trying to format the log to Splunk?

If you let Splunk try to figure out the linebreaking, it will add additional overhead to your indexing and slow it down.

Adding this will give you correct linebreaking and timestamping along with avoiding the merging pipeline which increases your indexing speed

[sourcetype]
TIME_PREFIX = 
TIME_FORMAT = 
LINE_BREAKER = 
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 
TRUNCATE = 

https://wiki.splunk.com/Community:HowIndexingWorks

View solution in original post

Ultra Champion

@skoelpin, good question.

We have teams that can form their json logs per the Splunk's needs. So, we are lucky in this sense.

0 Karma

Ultra Champion

We were told by the Sales Engineer that as long as it's proper JSON, all we need to do is set -

INDEXED_EXTRACTIONS = json
category = Structured

in props.conf.

0 Karma

Ultra Champion

For the record, the predefined _json sourcetype has these two defined config variables -

 INDEXED_EXTRACTIONS = json
 category = Structured
0 Karma

Ultra Champion

This solution works!!!

SplunkTrust
SplunkTrust

Your sales engineer is partially right, but you should ALWAYS apply base configs to lessen the indexer load when indexing data. This is a big part of the SCC2 bootcamp

Ultra Champion

Much appreciated @skoelpin.