Getting Data In

Why are the JSON event lines missing?

splunkLPN
Path Finder

the line format is :

{"tim":"2018-07-12 15:23:16","pre":"ayisha.udam","fir":"Ayisha","las":"UDAM","pe1":false}

Some lines present in the source file if I look at it with a text editor don't appear in a search or in a raw export.
in a json export :

{"preview":false,"result":{"_raw":"{\"tim\":\"2018-07-12 15:23:46\",\"pre\":\"ayisha.adam\",\"fir\":\"Ayisha\",\"las\":\"UDAM\" ...}

The sourcetype (I don't think it's the problem) is like that :

INDEXED_EXTRACTIONS:json
KV_MODE:json
NO_BINARY_CHECK:true
SHOULD_LINEMERGE:false
category:Structured
description:JavaScript Object Notation format. For more information, visit http://json.org/
disabled:false
pulldown_type:true

I've checked for differences in the source : line breaks, quote, I can't see any differences.

What else can I check?

thank's

0 Karma
1 Solution

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 

View solution in original post

0 Karma

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 
0 Karma

splunkLPN
Path Finder

Thank you for made me discover btool. I must now investigate. Config Quest app will help me.
Your suggestion solved my problems !

0 Karma

akocak
Contributor

Can you pick my solution as answer then 😄 ? No problem, I learned a lot here from other people

0 Karma

splunkLPN
Path Finder

That was my intention ! I don't see how change the "accepted answer"

Get Updates on the Splunk Community!

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Announcing the General Availability of Splunk Enterprise Security 8.1!

We are pleased to announce the general availability of Splunk Enterprise Security 8.1. Splunk becomes the only ...

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...