Getting Data In

Why are the JSON event lines missing?

splunkLPN
Path Finder

the line format is :

{"tim":"2018-07-12 15:23:16","pre":"ayisha.udam","fir":"Ayisha","las":"UDAM","pe1":false}

Some lines present in the source file if I look at it with a text editor don't appear in a search or in a raw export.
in a json export :

{"preview":false,"result":{"_raw":"{\"tim\":\"2018-07-12 15:23:46\",\"pre\":\"ayisha.adam\",\"fir\":\"Ayisha\",\"las\":\"UDAM\" ...}

The sourcetype (I don't think it's the problem) is like that :

INDEXED_EXTRACTIONS:json
KV_MODE:json
NO_BINARY_CHECK:true
SHOULD_LINEMERGE:false
category:Structured
description:JavaScript Object Notation format. For more information, visit http://json.org/
disabled:false
pulldown_type:true

I've checked for differences in the source : line breaks, quote, I can't see any differences.

What else can I check?

thank's

0 Karma
1 Solution

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 

View solution in original post

0 Karma

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 
0 Karma

splunkLPN
Path Finder

Thank you for made me discover btool. I must now investigate. Config Quest app will help me.
Your suggestion solved my problems !

0 Karma

akocak
Contributor

Can you pick my solution as answer then 😄 ? No problem, I learned a lot here from other people

0 Karma

splunkLPN
Path Finder

That was my intention ! I don't see how change the "accepted answer"

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...